pretraining-data

Here are 3 public repositories matching this topic...

Yaxin9Luo / LLMSurgeon

(ACL 2026 Main) LLMSurgeon recovers the pretraining data mixture of any LLM from only its generated text — no weights, no training data. A calibrated domain classifier plus label-shift correction de-blurs biased predictions. Ships with LLMScan, a benchmark on 8 open-source LLMs.

data-provenance label-shift membership-inference large-language-models llm-analysis pretraining-data data-mixture acl2026

Updated May 29, 2026
Python

Rian-T / benchmark-in-a-haystack

Star

Do quality filters pull benchmark questions into pretraining corpora? Injects MMLU, GSM8K, GPQA, ARC, HellaSwag, PIQA and TruthfulQA items into a corpus and ranks them with six quality classifiers, including DCLM, FineWeb-Edu and Gaperon.

python nlp data-filtering data-curation data-contamination gsm8k mmlu llm-evaluation fineweb dataset-quality pretraining-data llm-benchmark benchmark-contamination quality-filter

Updated Aug 19, 2026
Python

emfreeburg / announcement-carries-cue

Star

Notation as an unrecorded pre-training variable: the paper, the instruments, the corpus census, the reader/writer studies, and the pure-frame format. E. M. Freeburg, 2026.

nlp corpus-linguistics language-models data-curation pretraining-data

Updated Aug 11, 2026
Python

Improve this page

Add a description, image, and links to the pretraining-data topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the pretraining-data topic, visit your repo's landing page and select "manage topics."

Learn more