Low-code framework for building custom LLMs, neural networks, and other AI models

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

A curated, but incomplete, list of data-centric AI resources.

Deita: Data-Efficient Instruction Tuning for Alignment [ICLR2024]

The toolkit to test, validate, and evaluate your models and surface, curate, and prioritize the most valuable data for labeling.

DataCLUE: 数据为中心的NLP基准和工具包

Rust implementation of the Data Distribution Service (DDS)

[ICLR'23] Implementation of "Empowering Graph Representation Learning with Test-Time Graph Transformation"

Simulator framework for analysis of performance, energy consumption, area and cost of multi-node multi-chiplet tile-based manycore designs

A Data Centric NER annotation tool for your Named Entity Recognition projects

Vue Form with Laravel Inspired Validation and Simply Enjoyable Error Messages Api. (Form Api, Validator Api, Rules Api, Error Messages Api)

🔥🔥🔥 KDD2024 Best Student Paper

An observer is a wrapper over JSON data, that provides an interface to know when data is changed, with a focus on performance and memory efficiency.

Codes for a Top 5% finish in the Data-Centric AI Competition organized by Andrew Ng and DeepLearning.AI

From local functions to cloud deployed pipelines

Jaehyung Kim et al's ACL 2023 paper on "infoVerse: A Universal Framework for Dataset Characterization with Multidimensional Meta-information"

Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data (NeurIPS 2022)

Data-SUITE: Data-centric identification of in-distribution incongruous examples (ICML 2022)

Quickly set up an image labelling web application for manually tagging images for machine learning tasks.

Open-source Data Backend written in Java and based on PostgreSQL & GraphQL.