Report¶
The machine-readable and human-readable record of a trimming run.
module-attribute
¶
Name of the serialised report a run writes into its output directory.
module-attribute
¶
Name of the fully resolved configuration a run writes alongside the report.
pydantic-model
¶
Bases: _Base
What the corpus pass saw.
Show JSON schema:
{
"additionalProperties": false,
"description": "What the corpus pass saw.",
"properties": {
"documents": {
"description": "Examples read across all datasets, e.g. 200000.",
"title": "Documents",
"type": "integer"
},
"total_tokens": {
"description": "Total token occurrences counted, e.g. 91204338.",
"title": "Total Tokens",
"type": "integer"
},
"distinct_tokens": {
"description": "Distinct token ids the corpus used; the ceiling on what a corpus-only selection can keep.",
"title": "Distinct Tokens",
"type": "integer"
},
"coverage": {
"description": "Fraction of corpus occurrences covered by the kept vocabulary, e.g. 0.9993.",
"title": "Coverage",
"type": "number"
}
},
"required": [
"documents",
"total_tokens",
"distinct_tokens",
"coverage"
],
"title": "CorpusReport",
"type": "object"
}
Fields:
-
documents(int) -
total_tokens(int) -
distinct_tokens(int) -
coverage(float)
pydantic-model
¶
Bases: _Base
How the vocabulary changed.
Show JSON schema:
{
"additionalProperties": false,
"description": "How the vocabulary changed.",
"properties": {
"model_type": {
"description": "Tokenizer backend type, e.g. 'BPE'.",
"title": "Model Type",
"type": "string"
},
"original_size": {
"description": "Token count before trimming, e.g. 151669.",
"title": "Original Size",
"type": "integer"
},
"trimmed_size": {
"description": "Token count after trimming, e.g. 32000.",
"title": "Trimmed Size",
"type": "integer"
},
"structural_tokens": {
"description": "Tokens that were never eligible for removal, e.g. 282 for a byte-level BPE with 26 added tokens.",
"title": "Structural Tokens",
"type": "integer"
},
"kept_by_reason": {
"additionalProperties": {
"type": "integer"
},
"description": "Kept-token counts per provenance label, e.g. {'structural': 282, 'preset:byte_alphabet': 256, 'chat_template': 88, 'dependency': 101}. An id kept for two reasons is counted under both.",
"title": "Kept By Reason",
"type": "object"
},
"unknown_requested_tokens": {
"description": "Requested token strings absent from this vocabulary; usually a keep-list written for a different checkpoint.",
"items": {
"type": "string"
},
"title": "Unknown Requested Tokens",
"type": "array"
},
"dropped_requested_tokens": {
"additionalProperties": {
"items": {
"type": "string"
},
"type": "array"
},
"description": "Requested tokens the size cap removed anyway, mapped to why they were requested.",
"title": "Dropped Requested Tokens",
"type": "object"
}
},
"required": [
"model_type",
"original_size",
"trimmed_size",
"structural_tokens",
"kept_by_reason"
],
"title": "VocabularyReport",
"type": "object"
}
Fields:
-
model_type(str) -
original_size(int) -
trimmed_size(int) -
structural_tokens(int) -
kept_by_reason(dict[str, int]) -
unknown_requested_tokens(list[str]) -
dropped_requested_tokens(dict[str, list[str]])
pydantic-field
¶
Tokens that were never eligible for removal, e.g. 282 for a byte-level BPE with 26 added tokens.
pydantic-field
¶
Kept-token counts per provenance label, e.g. {'structural': 282, 'preset:byte_alphabet': 256, 'chat_template': 88, 'dependency': 101}. An id kept for two reasons is counted under both.
pydantic-field
¶
Requested token strings absent from this vocabulary; usually a keep-list written for a different checkpoint.
pydantic-field
¶
Requested tokens the size cap removed anyway, mapped to why they were requested.
property
¶
Return the fraction of the vocabulary that was removed, e.g. 0.789.
pydantic-model
¶
Bases: _Base
How the model changed.
This is what trimbed.model_trim.trim_model returns, so the surgery's own
result is already the thing the report serialises.
Attributes:
| Name | Type | Description |
|---|---|---|
model_class |
str
|
Class the checkpoint was loaded as, e.g. |
old_embedding_rows |
int
|
Rows in the input embedding matrix before trimming, e.g. 151,936 for Qwen3-0.6B. Read off the matrix, so not necessarily the 151,669 the tokenizer reports. |
new_embedding_rows |
int
|
Rows after trimming, including any alignment padding, so
|
old_parameters |
int
|
Total model parameters before trimming. |
new_parameters |
int
|
Total model parameters after trimming. |
tied_embeddings |
bool
|
Whether input and output embeddings share weights. This is true for most current decoders, including every checkpoint the test suite covers. |
has_output_head |
bool
|
Whether the model has a separate output embedding at all. False for an encoder loaded as a base model, e.g. codefuse-ai/F2LLM-v2-160M. |
Show JSON schema:
{
"additionalProperties": false,
"description": "How the model changed.\n\nThis is what `trimbed.model_trim.trim_model` returns, so the surgery's own\nresult is already the thing the report serialises.\n\nAttributes:\n model_class: Class the checkpoint was loaded as, e.g. `BertForMaskedLM` or\n `Qwen3ForCausalLM`.\n old_embedding_rows: Rows in the input embedding matrix before trimming, e.g.\n 151,936 for Qwen3-0.6B. Read off the matrix, so not necessarily the 151,669\n the tokenizer reports.\n new_embedding_rows: Rows after trimming, including any alignment padding, so\n `pad_to_multiple_of: 128` turns 32,000 kept tokens into 32,000 exactly and\n 32,001 into 32,128.\n old_parameters: Total model parameters before trimming.\n new_parameters: Total model parameters after trimming.\n tied_embeddings: Whether input and output embeddings share weights. This is true\n for most current decoders, including every checkpoint the test suite covers.\n has_output_head: Whether the model has a separate output embedding at all. False\n for an encoder loaded as a base model, e.g. codefuse-ai/F2LLM-v2-160M.",
"properties": {
"model_class": {
"title": "Model Class",
"type": "string"
},
"old_embedding_rows": {
"title": "Old Embedding Rows",
"type": "integer"
},
"new_embedding_rows": {
"title": "New Embedding Rows",
"type": "integer"
},
"old_parameters": {
"title": "Old Parameters",
"type": "integer"
},
"new_parameters": {
"title": "New Parameters",
"type": "integer"
},
"tied_embeddings": {
"title": "Tied Embeddings",
"type": "boolean"
},
"has_output_head": {
"title": "Has Output Head",
"type": "boolean"
}
},
"required": [
"model_class",
"old_embedding_rows",
"new_embedding_rows",
"old_parameters",
"new_parameters",
"tied_embeddings",
"has_output_head"
],
"title": "ModelReport",
"type": "object"
}
Fields:
-
model_class(str) -
old_embedding_rows(int) -
new_embedding_rows(int) -
old_parameters(int) -
new_parameters(int) -
tied_embeddings(bool) -
has_output_head(bool)
property
¶
Return how many parameters the trim eliminated.
pydantic-model
¶
Bases: _Base
How faithfully the trimmed tokenizer reproduces the original.
trimbed.verify.verify_tokenizer fills this in as it walks the sample texts,
so every field starts at its empty value.
Attributes:
| Name | Type | Description |
|---|---|---|
checked |
int
|
How many texts were compared. |
identical |
int
|
Texts whose token sequence maps exactly through the remap. |
equivalent_text |
int
|
Texts that decode back to the same string, even if the ids took a different route (possible when a merge was dropped). |
original_tokens |
int
|
Tokens the original tokenizer produced over all samples. |
trimmed_tokens |
int
|
Tokens the trimmed tokenizer produced over the same samples. |
failures |
list[str]
|
Sample texts that decoded differently, truncated for readability. |
Show JSON schema:
{
"additionalProperties": false,
"description": "How faithfully the trimmed tokenizer reproduces the original.\n\n`trimbed.verify.verify_tokenizer` fills this in as it walks the sample texts,\nso every field starts at its empty value.\n\nAttributes:\n checked: How many texts were compared.\n identical: Texts whose token sequence maps exactly through the remap.\n equivalent_text: Texts that decode back to the same string, even if the ids took a\n different route (possible when a merge was dropped).\n original_tokens: Tokens the original tokenizer produced over all samples.\n trimmed_tokens: Tokens the trimmed tokenizer produced over the same samples.\n failures: Sample texts that decoded differently, truncated for readability.",
"properties": {
"checked": {
"default": 0,
"title": "Checked",
"type": "integer"
},
"identical": {
"default": 0,
"title": "Identical",
"type": "integer"
},
"equivalent_text": {
"default": 0,
"title": "Equivalent Text",
"type": "integer"
},
"original_tokens": {
"default": 0,
"title": "Original Tokens",
"type": "integer"
},
"trimmed_tokens": {
"default": 0,
"title": "Trimmed Tokens",
"type": "integer"
},
"failures": {
"items": {
"type": "string"
},
"title": "Failures",
"type": "array"
}
},
"title": "VerificationReport",
"type": "object"
}
Fields:
-
checked(int) -
identical(int) -
equivalent_text(int) -
original_tokens(int) -
trimmed_tokens(int) -
failures(list[str])
property
¶
Return the fraction of texts that still decode to the original string.
property
¶
Return how much longer the trimmed tokenizer's output is.
Dropping a merge splits the affected words into more pieces, and every extra
piece is paid for at inference time. A ratio above 1.0 is the size of that cost,
e.g. 1.0000 when nothing the sample uses was lost, 1.03 when the trim costs
3% more tokens on the same text.
pydantic-model
¶
Bases: _Base
How closely the trimmed model reproduces the original's outputs.
Attributes:
| Name | Type | Description |
|---|---|---|
checked |
int
|
How many texts both models ran on. |
skipped |
int
|
Texts left out because their ids did not map one-to-one. |
max_hidden_diff |
float
|
Largest absolute difference between the last hidden states,
e.g. |
max_logit_diff |
float | None
|
The same over the output head's logits, or |
tolerance |
float
|
The threshold |
max_length |
int | None
|
How many tokens of each text the models were run on, e.g. |
Show JSON schema:
{
"additionalProperties": false,
"description": "How closely the trimmed model reproduces the original's outputs.\n\nAttributes:\n checked: How many texts both models ran on.\n skipped: Texts left out because their ids did not map one-to-one.\n max_hidden_diff: Largest absolute difference between the last hidden states,\n e.g. `4.8e-07` for a correct float32 trim.\n max_logit_diff: The same over the output head's logits, or `None` when the model\n has no head.\n tolerance: The threshold `ok` compares against, e.g. `1e-05`.\n max_length: How many tokens of each text the models were run on, e.g. `512` for a\n BERT, or `None` when nothing bounded the length.",
"properties": {
"checked": {
"default": 0,
"title": "Checked",
"type": "integer"
},
"skipped": {
"default": 0,
"title": "Skipped",
"type": "integer"
},
"max_hidden_diff": {
"default": 0.0,
"title": "Max Hidden Diff",
"type": "number"
},
"max_logit_diff": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"default": null,
"title": "Max Logit Diff"
},
"tolerance": {
"default": 1e-05,
"title": "Tolerance",
"type": "number"
},
"max_length": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Max Length"
}
},
"title": "ModelVerificationReport",
"type": "object"
}
Fields:
-
checked(int) -
skipped(int) -
max_hidden_diff(float) -
max_logit_diff(float | None) -
tolerance(float) -
max_length(int | None)
pydantic-model
¶
Bases: _Base
The complete record of one trimming run.
Show JSON schema:
{
"$defs": {
"CorpusReport": {
"additionalProperties": false,
"description": "What the corpus pass saw.",
"properties": {
"documents": {
"description": "Examples read across all datasets, e.g. 200000.",
"title": "Documents",
"type": "integer"
},
"total_tokens": {
"description": "Total token occurrences counted, e.g. 91204338.",
"title": "Total Tokens",
"type": "integer"
},
"distinct_tokens": {
"description": "Distinct token ids the corpus used; the ceiling on what a corpus-only selection can keep.",
"title": "Distinct Tokens",
"type": "integer"
},
"coverage": {
"description": "Fraction of corpus occurrences covered by the kept vocabulary, e.g. 0.9993.",
"title": "Coverage",
"type": "number"
}
},
"required": [
"documents",
"total_tokens",
"distinct_tokens",
"coverage"
],
"title": "CorpusReport",
"type": "object"
},
"ModelReport": {
"additionalProperties": false,
"description": "How the model changed.\n\nThis is what `trimbed.model_trim.trim_model` returns, so the surgery's own\nresult is already the thing the report serialises.\n\nAttributes:\n model_class: Class the checkpoint was loaded as, e.g. `BertForMaskedLM` or\n `Qwen3ForCausalLM`.\n old_embedding_rows: Rows in the input embedding matrix before trimming, e.g.\n 151,936 for Qwen3-0.6B. Read off the matrix, so not necessarily the 151,669\n the tokenizer reports.\n new_embedding_rows: Rows after trimming, including any alignment padding, so\n `pad_to_multiple_of: 128` turns 32,000 kept tokens into 32,000 exactly and\n 32,001 into 32,128.\n old_parameters: Total model parameters before trimming.\n new_parameters: Total model parameters after trimming.\n tied_embeddings: Whether input and output embeddings share weights. This is true\n for most current decoders, including every checkpoint the test suite covers.\n has_output_head: Whether the model has a separate output embedding at all. False\n for an encoder loaded as a base model, e.g. codefuse-ai/F2LLM-v2-160M.",
"properties": {
"model_class": {
"title": "Model Class",
"type": "string"
},
"old_embedding_rows": {
"title": "Old Embedding Rows",
"type": "integer"
},
"new_embedding_rows": {
"title": "New Embedding Rows",
"type": "integer"
},
"old_parameters": {
"title": "Old Parameters",
"type": "integer"
},
"new_parameters": {
"title": "New Parameters",
"type": "integer"
},
"tied_embeddings": {
"title": "Tied Embeddings",
"type": "boolean"
},
"has_output_head": {
"title": "Has Output Head",
"type": "boolean"
}
},
"required": [
"model_class",
"old_embedding_rows",
"new_embedding_rows",
"old_parameters",
"new_parameters",
"tied_embeddings",
"has_output_head"
],
"title": "ModelReport",
"type": "object"
},
"ModelVerificationReport": {
"additionalProperties": false,
"description": "How closely the trimmed model reproduces the original's outputs.\n\nAttributes:\n checked: How many texts both models ran on.\n skipped: Texts left out because their ids did not map one-to-one.\n max_hidden_diff: Largest absolute difference between the last hidden states,\n e.g. `4.8e-07` for a correct float32 trim.\n max_logit_diff: The same over the output head's logits, or `None` when the model\n has no head.\n tolerance: The threshold `ok` compares against, e.g. `1e-05`.\n max_length: How many tokens of each text the models were run on, e.g. `512` for a\n BERT, or `None` when nothing bounded the length.",
"properties": {
"checked": {
"default": 0,
"title": "Checked",
"type": "integer"
},
"skipped": {
"default": 0,
"title": "Skipped",
"type": "integer"
},
"max_hidden_diff": {
"default": 0.0,
"title": "Max Hidden Diff",
"type": "number"
},
"max_logit_diff": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"default": null,
"title": "Max Logit Diff"
},
"tolerance": {
"default": 1e-05,
"title": "Tolerance",
"type": "number"
},
"max_length": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Max Length"
}
},
"title": "ModelVerificationReport",
"type": "object"
},
"VerificationReport": {
"additionalProperties": false,
"description": "How faithfully the trimmed tokenizer reproduces the original.\n\n`trimbed.verify.verify_tokenizer` fills this in as it walks the sample texts,\nso every field starts at its empty value.\n\nAttributes:\n checked: How many texts were compared.\n identical: Texts whose token sequence maps exactly through the remap.\n equivalent_text: Texts that decode back to the same string, even if the ids took a\n different route (possible when a merge was dropped).\n original_tokens: Tokens the original tokenizer produced over all samples.\n trimmed_tokens: Tokens the trimmed tokenizer produced over the same samples.\n failures: Sample texts that decoded differently, truncated for readability.",
"properties": {
"checked": {
"default": 0,
"title": "Checked",
"type": "integer"
},
"identical": {
"default": 0,
"title": "Identical",
"type": "integer"
},
"equivalent_text": {
"default": 0,
"title": "Equivalent Text",
"type": "integer"
},
"original_tokens": {
"default": 0,
"title": "Original Tokens",
"type": "integer"
},
"trimmed_tokens": {
"default": 0,
"title": "Trimmed Tokens",
"type": "integer"
},
"failures": {
"items": {
"type": "string"
},
"title": "Failures",
"type": "array"
}
},
"title": "VerificationReport",
"type": "object"
},
"VocabularyReport": {
"additionalProperties": false,
"description": "How the vocabulary changed.",
"properties": {
"model_type": {
"description": "Tokenizer backend type, e.g. 'BPE'.",
"title": "Model Type",
"type": "string"
},
"original_size": {
"description": "Token count before trimming, e.g. 151669.",
"title": "Original Size",
"type": "integer"
},
"trimmed_size": {
"description": "Token count after trimming, e.g. 32000.",
"title": "Trimmed Size",
"type": "integer"
},
"structural_tokens": {
"description": "Tokens that were never eligible for removal, e.g. 282 for a byte-level BPE with 26 added tokens.",
"title": "Structural Tokens",
"type": "integer"
},
"kept_by_reason": {
"additionalProperties": {
"type": "integer"
},
"description": "Kept-token counts per provenance label, e.g. {'structural': 282, 'preset:byte_alphabet': 256, 'chat_template': 88, 'dependency': 101}. An id kept for two reasons is counted under both.",
"title": "Kept By Reason",
"type": "object"
},
"unknown_requested_tokens": {
"description": "Requested token strings absent from this vocabulary; usually a keep-list written for a different checkpoint.",
"items": {
"type": "string"
},
"title": "Unknown Requested Tokens",
"type": "array"
},
"dropped_requested_tokens": {
"additionalProperties": {
"items": {
"type": "string"
},
"type": "array"
},
"description": "Requested tokens the size cap removed anyway, mapped to why they were requested.",
"title": "Dropped Requested Tokens",
"type": "object"
}
},
"required": [
"model_type",
"original_size",
"trimmed_size",
"structural_tokens",
"kept_by_reason"
],
"title": "VocabularyReport",
"type": "object"
}
},
"additionalProperties": false,
"description": "The complete record of one trimming run.",
"properties": {
"model": {
"description": "Model the tokenizer came from, e.g. 'codefuse-ai/F2LLM-v2-160M'.",
"title": "Model",
"type": "string"
},
"output_dir": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Where artefacts were written, e.g. 'trimmed/f2llm-nl'; null for a dry run.",
"title": "Output Dir"
},
"dry_run": {
"default": false,
"description": "Whether writing was skipped.",
"title": "Dry Run",
"type": "boolean"
},
"vocabulary": {
"$ref": "#/$defs/VocabularyReport"
},
"corpus": {
"anyOf": [
{
"$ref": "#/$defs/CorpusReport"
},
{
"type": "null"
}
],
"default": null
},
"model_trim": {
"anyOf": [
{
"$ref": "#/$defs/ModelReport"
},
{
"type": "null"
}
],
"default": null
},
"verification": {
"anyOf": [
{
"$ref": "#/$defs/VerificationReport"
},
{
"type": "null"
}
],
"default": null
},
"model_verification": {
"anyOf": [
{
"$ref": "#/$defs/ModelVerificationReport"
},
{
"type": "null"
}
],
"default": null
},
"sidecar_files": {
"description": "Files copied verbatim from the source repository.",
"items": {
"type": "string"
},
"title": "Sidecar Files",
"type": "array"
}
},
"required": [
"model",
"vocabulary"
],
"title": "TrimReport",
"type": "object"
}
Fields:
-
model(str) -
output_dir(str | None) -
dry_run(bool) -
vocabulary(VocabularyReport) -
corpus(CorpusReport | None) -
model_trim(ModelReport | None) -
verification(VerificationReport | None) -
model_verification(ModelVerificationReport | None) -
sidecar_files(list[str])
pydantic-field
¶
Where artefacts were written, e.g. 'trimmed/f2llm-nl'; null for a dry run.
pydantic-field
¶
Files copied verbatim from the source repository.
Write the report as JSON into a directory.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
directory
|
str | Path
|
Destination directory, created if it does not exist yet. |
required |
Returns:
| Type | Description |
|---|---|
Path
|
The path written. |