Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data

This repository contains the code accompanying our paper: Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data

Repository Structure

The codebase is organized into subdirectories for each task (the functions directory contains code for both Functions and Mixture of Functions). Please refer to the individual subdirectories for specific instructions on running the code for each task.

We provide evaluation results and a Jupyter notebook with code for plotting and generating baselines for the Parity Learning task in the parity directory.

Citation

@inproceedings{NEURIPS2024_fe489a28,
 author = {Treutlein, Johannes and Choi, Dami and Betley, Jan and Marks, Sam and Anil, Cem and Grosse, Roger and Evans, Owain},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang},
 pages = {140667--140730},
 publisher = {Curran Associates, Inc.},
 title = {Connecting the Dots: {LLM}s can Infer and Verbalize Latent Structure from Disparate Training Data},
 url = {https://proceedings.neurips.cc/paper_files/paper/2024/file/fe489a28a54583ee802b8e2955c024c2-Paper-Conference.pdf},
 volume = {37},
 year = {2024}
}