DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Cover image for Speed up your Odoo 12.0 code
Apruzzese Francesco
Apruzzese Francesco

Posted on • Updated on

Speed up your Odoo 12.0 code

Introduction

I love Python and I work with Odoo since 10 years.
In all these years I wrote a lot of code for Odoo, both modules and migrations scripts. Every time you need to move a lot of data from a source to a destination, time is an important point.

For example, I once wrote a script to copy more than 20 millions of products from an external software to Odoo database. Odoo executes a lot of checks every time you create a new record, so it's important to understand that every seconds is vital.

With so many records, every millisecond is a resource.

When we're writing the code, what we do is create the necessary data and run the tests. These data are often some products, partners, orders and invoices. When everything works you can create automatic tests.
I don't think every time we develop something we create a stress test for a large amount of data.

So, I always study and try new tips and tricks to increase speed of my code. A code is never perfect and it can be improved, always!

Using this basic script I want to show you some snippets to speed up code effortlessly.

import timeit

# Here an exemple of common but slow code
<CODE SLOW>

# Here an exemple of fast code
<CODE FAST>

setup = "records = env['res.partner'].search([], limit=1000)"
repetitions = (1, 10, 100)
globals_vals = {'env': env}

for repetition in repetitions:
    slow_time = timeit.repeat(
        code_slow,
        setup=setup,
        number=repetition,
        globals={'env': env}
        )

    fast_time = timeit.repeat(
        code_fast,
        setup=setup,
        number=repetition,
        globals=globals_vals
        )

    print(f'=============== loops {repetition}')
    print(slow_time)
    print(fast_time)

set vs recordset

Working with Odoo means using a lots of recordsets. A recordset is an object with datas, functions and methods. It is a set of data, either from a database or calculated, on steroids. If you know other ORMs, you know what I'm talking about.

Sometimes, however, using the operations that the framework makes available is not convenient and it's more advantageous to use directly what python offers.

Code

code_slow = """
result = env['res.partner'].browse()
for record in records:
    result |= record
"""

code_fast = """
result = set()
for record in records:
    result.add(record.id)
env['res.partner'].browse(result)
"""

Results

=============== loops 1
[0.13306364299933193, 0.10838474300544476, 0.10796952999953646, 0.10946986099588685, 0.10623473400482908]
[0.0008965670058387332, 0.000925485001062043, 0.0009349410029244609, 0.0009159730034298263, 0.0009303710030508228]
=============== loops 10
[1.106517462998454, 1.1065070289987489, 1.1934943989981548, 1.1571569709994947, 1.1170753799960949]
[0.009077082999283448, 0.009004210005514324, 0.009018140997795854, 0.009551764000207186, 0.009227231996192131]
=============== loops 100
[11.516509660999873, 11.74818367199623, 11.85345141900325, 12.856167154001014, 13.63239839200105]
[0.09894232600345276, 0.09653759599314071, 0.09699099600402405, 0.10923265099700075, 0.10307022200140636]

write vs field settings

When you have a recordset, it's possible to change a value with a simple assignment. This is handy but every time you use it, Odoo prepare a query and change this value in cache. When you need to write more values ​​in the same recordset or the same value on more recordsets, you must use write function.

Code

code_slow = """
for record in records:
    record.ref = 'test_speed'
"""

code_fast = """
records.write({'ref': 'test_speed'})
"""

Results

=============== loops 1
[2.9913442569959443, 0.9559976780001307, 0.900353463999636, 0.8353457880002679, 1.0130259670040687]
[0.13637410899536917, 0.1379215750057483, 0.14272761800384615, 0.12462920800317079, 0.13321589499537367]
=============== loops 10
[10.402880988003744, 11.000012251999578, 11.697913458003313, 11.106917288998375, 11.169360947998939]
[2.0335192759957863, 2.4452823780011386, 2.2474471349996747, 2.5181185900000855, 2.776447412004927]

list comprehension vs mapped

Among the recordset methods, we have mapped(). It creates a set of values from a recordset. Values got from mapped() can be navigated through Odoo relations. In simple cases, mapped() is as convenient as list comprehension. In complex cases (for example, a navigation of relations of 3 levels) mapped() is more convenient to use.

It gives always better readability and this is good for maintainability.

Code

code_slow = """
states_set = set([r.state_id.id for r in records if r.state_id])
states = env['res.country.state'].browse(states_set)
"""

code_fast = """
states = records.mapped('state_id')
"""

Results

=============== loops 1
[0.31179145100031747, 0.004711384994152468, 0.004819055000552908, 0.004665278996981215, 0.0044039049971615896]
[0.0036316699988674372, 0.003607042999647092, 0.003627748999861069, 0.0036114609974902123, 0.003590111999073997]
=============== loops 10
[0.047915923001710325, 0.04522175599413458, 0.044370734998665284, 0.04409689400199568, 0.04411724000237882]
[0.035760099002800416, 0.0357158859987976, 0.03603034999832744, 0.036463224998442456, 0.03686660800303798]
=============== loops 100
[0.45273306199669605, 0.44231640599900857, 0.43908101499982877, 0.44457509399944684, 0.4423183210019488]
[0.36031395699683344, 0.3654527219987358, 0.38619487999676494, 0.3619795150007121, 0.3626547140011098]

loop vs filtered

Another recordset function is filtered(). You can use to filter recordset with a function or a value. Like mapped(), filtered() is as convenient as loop for simple cases but can be more convenient for complex case.

It gives always better readability and this is good for maintainability.

Code

code_slow = """
partners_start_with_a = set()
for record in records:
    if record.name.lower().startswith('a'):
        partners_start_with_a.add(record.id)
env['res.partner'].browse(partners_start_with_a)
"""

code_fast = """
records.filtered(lambda r: r.name.lower().startswith('a'))
"""

Results

=============== loops 1
[0.3316459549969295, 0.002682621001440566, 0.0026461579982424155, 0.002589060997706838, 0.002434752997942269]
[0.002538305998314172, 0.002414796006632969, 0.0024050630017882213, 0.0024614639987703413, 0.002459411000018008]
=============== loops 10
[0.02490358999784803, 0.027465703002235387, 0.02418504800152732, 0.023117996999644674, 0.023439353004505392]
[0.02508747999672778, 0.024697505999938585, 0.02418924599624006, 0.024244852000265382, 0.024165777998859994]
=============== loops 100
[0.23826791400642833, 0.22961928599397652, 0.23434631299460307, 0.2344872630055761, 0.2336325560027035]
[0.24336347499775002, 0.24536530500336085, 0.2900283189956099, 0.24434635799843818, 0.24427578799804905]

single browse vs multi browse

In some cases we have a list of ids and we want to get the relative recordsets. In Odoo we use browse(). browse() require an id or a list of ids and return a single recordset or a recordset of recordsets. If you need to iterate them you must loop directly on browse() return.

Code

code_slow = """
record_ids = records.ids
for record_id in record_ids:
    record = env['res.partner'].browse(record_id)
    record.name
"""

code_fast = """
record_ids = records.ids
for record in env['res.partner'].browse(record_ids):
    record.name
"""

Results

=============== loops 1
[1.6325390929996502, 0.004575414001010358, 0.0046123150023049675, 0.005507702997419983, 0.004666433000238612]
[0.0024330550004378892, 0.002370230002270546, 0.0023654250035178848, 0.0024355540008400567, 0.0023724880011286587]
=============== loops 10
[0.049219961001654156, 0.04274192699813284, 0.0399551490045269, 0.039992154997889884, 0.04065825999714434]
[0.020162856999377254, 0.02014190399495419, 0.021769888000562787, 0.021047277994512115, 0.020693995997135062]
=============== loops 100
[0.41175051799655193, 0.4071878250033478, 0.4059581019973848, 0.40895022099721245, 0.40670431699982146]
[0.20786928899906343, 0.2082131749994005, 0.2086394680009107, 0.23048232900328003, 0.20772191799915163]

loop vs map

This is a python pure tip but I use it in my script especially with math calculations. map() call a function on an iterable object and return an object with results.

Code

code_slow = """
def increment_id(record_id):
    return record_id + 1

for record_id in records.ids:
    increment_id(record_id)
"""

code_fast = """
def increment_id(record_id):
    return record_id + 1

map(increment_id, records.ids)
"""

Results

=============== loops 10
[0.0014327949975267984, 0.0014315969965537079, 0.0014401990047190338, 0.0015307379944715649, 0.0014989250048529357]
[0.0004444369988050312, 0.00044178399548400193, 0.000439752999227494, 0.0004845689982175827, 0.0004615380021277815]
=============== loops 100
[0.016216568001254927, 0.013593563002359588, 0.012723464002192486, 0.01124695000180509, 0.011159162997500971]
[0.003178275001118891, 0.0032086909995996393, 0.003190825998899527, 0.0031917309970594943, 0.003197609999915585]
=============== loops 1000
[0.11258809099672362, 0.11201509100646945, 0.11202597500232514, 0.11326640100014629, 0.11274601000332041]
[0.034097837000445, 0.03186929500225233, 0.031756801006849855, 0.031629047996830195, 0.0315622540001641]

Conclusions

I know you're thinking that 0.07 seconds on a line of code aren't so important but if you remember my introduction you can see that on 20 millions of repetations we can earn a lot of time!

See you soon. I'm going away... very fast!

Support

Buy Me A Coffee

Top comments (6)

Collapse
 
zernox profile image
ZerNox

What would have been interesting in this topic is to actually see your approach and comparasion of batch creation of records > 1000 records , this seems like a real bottleneck at least for odoo10, interesting article <3

Collapse
 
themacharia profile image
Eric Macharia

Hey, am starting out in odoo development and I want to pick your thoughts on something.
How would you track changes in ir.attachment from form view, Like the moment somebody uploads a file, a record is updated in the form. Thank you in advance

Collapse
 
opencode profile image
Apruzzese Francesco

Hi Eric. I think that other sites can be used to ask questions. This isn't the right place. I don't want to generate noise under a post about other arguments. Please, consider to use official odoo help site. Thanks.

Collapse
 
opencode profile image
Apruzzese Francesco

It'a a simple example to filter recordset. Of course, there is a better way to write this but my focus is on "Odoo code" and not "Pure Python Code" ;)

Thank you :)

Image description

Join the One Year Club

You can earn this badge by being a registered member of the DEV Community for at least one year. Create an account and get started today.