loading...
Cover image for Speed up your Odoo 12.0 code

Speed up your Odoo 12.0 code

opencode profile image Apruzzese Francesco Updated on ・5 min read

Introduction

I love Python and I work with Odoo since 10 years.
In all these years I wrote a lot of code for Odoo, both modules and migrations scripts. Every time you need to move a lot of data from a source to a destination, time is an important point.

For example, I once wrote a script to copy more than 20 millions of products from an external software to Odoo database. Odoo executes a lot of checks every time you create a new record, so it's important to understand that every seconds is vital.

With so many records, every millisecond is a resource.

When we're writing the code, what we do is create the necessary data and run the tests. These data are often some products, partners, orders and invoices. When everything works you can create automatic tests.
I don't think every time we develop something we create a stress test for a large amount of data.

So, I always study and try new tips and tricks to increase speed of my code. A code is never perfect and it can be improved, always!

Using this basic script I want to show you some snippets to speed up code effortlessly.

import timeit

# Here an exemple of common but slow code
<CODE SLOW>

# Here an exemple of fast code
<CODE FAST>

setup = "records = env['res.partner'].search([], limit=1000)"
repetitions = (1, 10, 100)
globals_vals = {'env': env}

for repetition in repetitions:
    slow_time = timeit.repeat(
        code_slow,
        setup=setup,
        number=repetition,
        globals={'env': env}
        )

    fast_time = timeit.repeat(
        code_fast,
        setup=setup,
        number=repetition,
        globals=globals_vals
        )

    print(f'=============== loops {repetition}')
    print(slow_time)
    print(fast_time)

set vs recordset

Working with Odoo means using a lots of recordsets. A recordset is an object with datas, functions and methods. It is a set of data, either from a database or calculated, on steroids. If you know other ORMs, you know what I'm talking about.

Sometimes, however, using the operations that the framework makes available is not convenient and it's more advantageous to use directly what python offers.

Code

code_slow = """
result = env['res.partner'].browse()
for record in records:
    result |= record
"""

code_fast = """
result = set()
for record in records:
    result.add(record.id)
env['res.partner'].browse(result)
"""

Results

=============== loops 1
[0.13306364299933193, 0.10838474300544476, 0.10796952999953646, 0.10946986099588685, 0.10623473400482908]
[0.0008965670058387332, 0.000925485001062043, 0.0009349410029244609, 0.0009159730034298263, 0.0009303710030508228]
=============== loops 10
[1.106517462998454, 1.1065070289987489, 1.1934943989981548, 1.1571569709994947, 1.1170753799960949]
[0.009077082999283448, 0.009004210005514324, 0.009018140997795854, 0.009551764000207186, 0.009227231996192131]
=============== loops 100
[11.516509660999873, 11.74818367199623, 11.85345141900325, 12.856167154001014, 13.63239839200105]
[0.09894232600345276, 0.09653759599314071, 0.09699099600402405, 0.10923265099700075, 0.10307022200140636]

write vs field settings

When you have a recordset, it's possible to change a value with a simple assignment. This is handy but every time you use it, Odoo prepare a query and change this value in cache. When you need to write more values ​​in the same recordset or the same value on more recordsets, you must use write function.

Code

code_slow = """
for record in records:
    record.ref = 'test_speed'
"""

code_fast = """
records.write({'ref': 'test_speed'})
"""

Results

=============== loops 1
[2.9913442569959443, 0.9559976780001307, 0.900353463999636, 0.8353457880002679, 1.0130259670040687]
[0.13637410899536917, 0.1379215750057483, 0.14272761800384615, 0.12462920800317079, 0.13321589499537367]
=============== loops 10
[10.402880988003744, 11.000012251999578, 11.697913458003313, 11.106917288998375, 11.169360947998939]
[2.0335192759957863, 2.4452823780011386, 2.2474471349996747, 2.5181185900000855, 2.776447412004927]

list comprehension vs mapped

Among the recordset methods, we have mapped(). It creates a set of values from a recordset. Values got from mapped() can be navigated through Odoo relations. In simple cases, mapped() is as convenient as list comprehension. In complex cases (for example, a navigation of relations of 3 levels) mapped() is more convenient to use.

It gives always better readability and this is good for maintainability.

Code

code_slow = """
states_set = set([r.state_id.id for r in records if r.state_id])
states = env['res.country.state'].browse(states_set)
"""

code_fast = """
states = records.mapped('state_id')
"""

Results

=============== loops 1
[0.31179145100031747, 0.004711384994152468, 0.004819055000552908, 0.004665278996981215, 0.0044039049971615896]
[0.0036316699988674372, 0.003607042999647092, 0.003627748999861069, 0.0036114609974902123, 0.003590111999073997]
=============== loops 10
[0.047915923001710325, 0.04522175599413458, 0.044370734998665284, 0.04409689400199568, 0.04411724000237882]
[0.035760099002800416, 0.0357158859987976, 0.03603034999832744, 0.036463224998442456, 0.03686660800303798]
=============== loops 100
[0.45273306199669605, 0.44231640599900857, 0.43908101499982877, 0.44457509399944684, 0.4423183210019488]
[0.36031395699683344, 0.3654527219987358, 0.38619487999676494, 0.3619795150007121, 0.3626547140011098]

loop vs filtered

Another recordset function is filtered(). You can use to filter recordset with a function or a value. Like mapped(), filtered() is as convenient as loop for simple cases but can be more convenient for complex case.

It gives always better readability and this is good for maintainability.

Code

code_slow = """
partners_start_with_a = set()
for record in records:
    if record.name.lower().startswith('a'):
        partners_start_with_a.add(record.id)
env['res.partner'].browse(partners_start_with_a)
"""

code_fast = """
records.filtered(lambda r: r.name.lower().startswith('a'))
"""

Results

=============== loops 1
[0.3316459549969295, 0.002682621001440566, 0.0026461579982424155, 0.002589060997706838, 0.002434752997942269]
[0.002538305998314172, 0.002414796006632969, 0.0024050630017882213, 0.0024614639987703413, 0.002459411000018008]
=============== loops 10
[0.02490358999784803, 0.027465703002235387, 0.02418504800152732, 0.023117996999644674, 0.023439353004505392]
[0.02508747999672778, 0.024697505999938585, 0.02418924599624006, 0.024244852000265382, 0.024165777998859994]
=============== loops 100
[0.23826791400642833, 0.22961928599397652, 0.23434631299460307, 0.2344872630055761, 0.2336325560027035]
[0.24336347499775002, 0.24536530500336085, 0.2900283189956099, 0.24434635799843818, 0.24427578799804905]

single browse vs multi browse

In some cases we have a list of ids and we want to get the relative recordsets. In Odoo we use browse(). browse() require an id or a list of ids and return a single recordset or a recordset of recordsets. If you need to iterate them you must loop directly on browse() return.

Code

code_slow = """
record_ids = records.ids
for record_id in record_ids:
    record = env['res.partner'].browse(record_id)
    record.name
"""

code_fast = """
record_ids = records.ids
for record in env['res.partner'].browse(record_ids):
    record.name
"""

Results

=============== loops 1
[1.6325390929996502, 0.004575414001010358, 0.0046123150023049675, 0.005507702997419983, 0.004666433000238612]
[0.0024330550004378892, 0.002370230002270546, 0.0023654250035178848, 0.0024355540008400567, 0.0023724880011286587]
=============== loops 10
[0.049219961001654156, 0.04274192699813284, 0.0399551490045269, 0.039992154997889884, 0.04065825999714434]
[0.020162856999377254, 0.02014190399495419, 0.021769888000562787, 0.021047277994512115, 0.020693995997135062]
=============== loops 100
[0.41175051799655193, 0.4071878250033478, 0.4059581019973848, 0.40895022099721245, 0.40670431699982146]
[0.20786928899906343, 0.2082131749994005, 0.2086394680009107, 0.23048232900328003, 0.20772191799915163]

loop vs map

This is a python pure tip but I use it in my script especially with math calculations. map() call a function on an iterable object and return an object with results.

Code

code_slow = """
def increment_id(record_id):
    return record_id + 1

for record_id in records.ids:
    increment_id(record_id)
"""

code_fast = """
def increment_id(record_id):
    return record_id + 1

map(increment_id, records.ids)
"""

Results

=============== loops 10
[0.0014327949975267984, 0.0014315969965537079, 0.0014401990047190338, 0.0015307379944715649, 0.0014989250048529357]
[0.0004444369988050312, 0.00044178399548400193, 0.000439752999227494, 0.0004845689982175827, 0.0004615380021277815]
=============== loops 100
[0.016216568001254927, 0.013593563002359588, 0.012723464002192486, 0.01124695000180509, 0.011159162997500971]
[0.003178275001118891, 0.0032086909995996393, 0.003190825998899527, 0.0031917309970594943, 0.003197609999915585]
=============== loops 1000
[0.11258809099672362, 0.11201509100646945, 0.11202597500232514, 0.11326640100014629, 0.11274601000332041]
[0.034097837000445, 0.03186929500225233, 0.031756801006849855, 0.031629047996830195, 0.0315622540001641]

Conclusions

I know you're thinking that 0.07 seconds on a line of code aren't so important but if you remember my introduction you can see that on 20 millions of repetations we can earn a lot of time!

See you soon. I'm going away... very fast!

Support

Buy Me A Coffee

Discussion

markdown guide
 

Hey, am starting out in odoo development and I want to pick your thoughts on something.
How would you track changes in ir.attachment from form view, Like the moment somebody uploads a file, a record is updated in the form. Thank you in advance

 

Hi Eric. I think that other sites can be used to ask questions. This isn't the right place. I don't want to generate noise under a post about other arguments. Please, consider to use official odoo help site. Thanks.

 
code_fast = """
records.filtered(lambda r: r.name.lower().startswith('a'))
"""

Would not be faster to write:

lambda name: name[0] in ("aA")

It is nittypicking but you dont need to apply lower to the whole name.

Furthermore, probably depending on implementation of Python,

the fastest may be:

char = name[0]
char is "a" or char is "A"

But of course I would not rely on this :)

Great article, anyway.

 

It'a a simple example to filter recordset. Of course, there is a better way to write this but my focus is on "Odoo code" and not "Pure Python Code" ;)

Thank you :)

 

I know 😄 and but still noteworthy.

WORDS = ("Ahaha asa sas s sas aa AA A sd df df fdfdf  sdfs ds fsf fsf sfs fsdfs fsfsf sfsf sf" 
         "SAS D DA A SDAD ADD ADAD ADAD ADA DASDSDAFDEAA AD AD D  SD  ad sds saAA   AADADDdas  a a AA SAD"*1000).split()

%%timeit
res1 = " ".join(filter(lambda r: r.lower().startswith('a'), WORDS))
12.8 ms ± 107 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
def my_filter(name, lower="a", upper="A"):
    first = name[0]
    return first is lower or first is upper
%%timeit 
res2 = " ".join(filter(my_filter, WORDS))

6.08 ms ± 26.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
6/13
0.46153846153846156

 

What would have been interesting in this topic is to actually see your approach and comparasion of batch creation of records > 1000 records , this seems like a real bottleneck at least for odoo10, interesting article <3