Performance

Performance

Performance tests of livejson comparing basic livejson usage to usage under the context manager.

Tests were performed on a 2010 MacBook Pro, with a 2.4 GHz Intel Core 2 Duo processor and 8GB of RAM.

List comprehension on 10,000 items

Say I have a ListDatabase with 10,000 items:

>>> import livejson
>>> a = livejson.Database("a.json")
>>> a.set_data(range(10000))

Let’s try a list comprehension, with a timer on it:

>>> def test_comprehension():
...     t1 = time.time()
...     b = [x + 1 for x in a]
...     print(time.time()-t1)
... 
>>> test_comprehension()
45.1499710083

This takes 45 seconds because under the current implementation of list comprehensions, the database is queried for each iteration. This means that livejson opens, reads, and closes the file 10,000 times.

However, with the context manager:

>>> def test_comprehension_2():
...     t1 = time.time()
...     with a:
...         b = [x + 1 for x in a]
...     print(time.time()-t1)
... 
>>> test_comprehension_2()
0.00205993652344

Only 1/500 of a second! This is because under the context manager, livejson will cache all the data at the beginning of the with block. Then, all subsequent operations are directed to an in-memory object. All your changes will be committed at the end of the with block.