lib_profiler

Datamart profiling library

This library can profile datasets for use with Auctus, NYU's dataset search engine. You can use it to profile datasets on your side and send that to the server for search, instead of uploading the whole dataset. It is also used internally by the service to process search-by-example queries (when sending a file to the /search endpoint) and to add datasets to the index (to be queried against later).

See also:

• The datamart-rest library for search/augmentation
• The datamart-materialize library, used to materialize dataset from search results
• The datamart-augmentation library, used to performs data augmentation with a dataset from Auctus
• Auctus, NYU's dataset search engine
• Our project on GitLab