-
Notifications
You must be signed in to change notification settings - Fork 20
Queries are slow (v1) #101
Comments
These times are not from a query - there are several levels of anidated foreachs inside every iteration each doing queries plus a complete parse on each iteration. Timing the queries and parsing individually in the same script gives 0.07 secs for the initial parsing and 0.05-0.06 for each individual xpath query. The memleak is real trough (I'll add more info soon on #100). |
I don't know it 0.05 per query in this case over a non trivial UAST can be considered slow or not so I'll leave this open until we decide, @smola what do you think? |
I don't think it's a problem of query efficiency so much as the libuast calling to some function pointers (like callback) that are implemented on the cpython side which, even while being written in C, use python data structures and runtime that are slow. But I could be wrong and this merit a further investigation. |
XPath might be slower than well-written ad hoc code, this is usually true for SQL too. @EgorBu I guess the guide you linked is actually applicable directly. |
@smola, definitely, libxml does a lot of allocations when starting up and so does libuast when creating the pseudo-xml. Ad-hoc code on Python works on already allocated and initialized nodes it's not so strange that is faster (specially using Also, as explained above, @EgorBu could you try again this benchmark after the memleak fix? I'm curious about the results but I don't have your system to have comparable times. Would be also nice if you had a timing for individual xpath queries. |
The new client was released, we need to check the performance again. |
Would be glad to hear news about changes!) |
Hi,
I measured time for the same query that I used in #100 - I think it's very suspicious that it's so slow.
Measurements:
The text was updated successfully, but these errors were encountered: