Skip to content

Examples

examples/ has one runnable script per usage pattern. Each takes -m/--model, so you can point any of them at a local checkpoint, and each returns its report or result rather than printing and discarding it, so the body copies straight into your own code.

Script What it shows
01_inspect_tokenizer.py Build a TokenizerSpec and describe a tokenizer without changing anything
02_trim_tokenizer_only.py Trim from must-keep rules alone, with no corpus and no model
03_trim_with_corpus.py The full TrimPipeline over a corpus, model included
04_custom_preset.py Register your own preset with register_preset
05_low_level_api.py Drive the stages yourself: count, select, trim, verify
06_local_corpus.py Count a corpus of local files with a DatasetSpec, offline

The low-level path is worth a look if you want to slot trimbed into a larger job: CorpusCounterselect_tokenstrim_tokenizertrim_modelverify_tokenizer.

Every example runs in the test suite against tiny in-process fixtures, so none of them can drift from the API without the suite going red.