Commits on Aug 29, 2023

1. [RFC] Cache Expr objects …

   Much of our overhead comes from doing computational work in other
   libraries (pandas, arrow, ...) that could be cached.  We do cache a lot
   of stuff today, but we store these caches on the object itself.  When we
   then recreate objects (for example in optimization) then we lose those
   caches.
   
   One solution here is to cache the objects themselves, so that
   `Op(...) is Op(...)`.  This technique is a bit magical, but is used in
   other projects like SymPy where it has had good performance impacts
   (although they use it because they make many more very small objects).
   
   Maybe this isn't relevant for us.  Ideally we wouldn't recreate objects
   often in optimization (this is why we return the original object if
   arguments match).  But maybe it's hard to be careful.  If so, this
   might provide a bit of a sledgehammer approach.
   
   THis isn't done yet, in particular there are open questions about
   non-hashable inputs like pandas dataframes.  Hopefully it is a useful
   proof of concept.

   

   mrocklin committed Aug 29, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for f897d3b
   • Browse repository at this point

   Copy the full SHA

   f897d3b View commit details

   Browse the repository at this point in the history