mirror of
https://github.com/docling-project/docling-core.git
synced 2026-11-18 20:27:00 +00:00
* fix(scripts): render the current schema snapshot from the live models When CURRENT_VERSION is bumped from 1.N to 1.(N+1), the coverage check creates the missing docs/schemas/DoclingDocument_1_(N+1).json by copying docs/DoclingDocument.json. But the hook order it relies on -- check-compat-projectors <- runs first docs <- runs after, regenerates the artefact -- guarantees that artefact still holds the 1.N schema at that moment. So the copy produces a file named for 1.(N+1) that describes 1.N, and because check 9 then compares the same stale artefact against it, the diff is empty and the run passes. The wrong baseline is only noticed later, as a "breaking change without a CURRENT_VERSION bump" error on the next commit, pointing the contributor at a bump they already made. Observed while bumping to 1.11.0: the generated DoclingDocument_1_11.json was byte-identical to DoclingDocument_1_10.json. The snapshot for the current version is now rendered from the models via generate_json_schema -- the same code path and serialisation generate_docs uses, so the two cannot drift -- and check 9 compares the models rather than the artefact. Neither check now depends on whether the docs hook has run, which removes the ordering coupling instead of documenting it. Snapshots for older versions predate this system and keep the permissive bootstrap copy. CONTRIBUTING and the script docstring both asserted that the hook ordering made the copy safe, which is what made the trap hard to see; both are corrected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com> * test: let DOCLING_GEN_TEST_DATA govern every per-item fixture test_docitems' verify() helper did not consult the flag at all. It wrote groundtruth for KeyValueItem and FormItem on every run, flag or no flag, and for no other class ever. Two consequences: - Those two fixtures were never really asserted. verify() wrote the file, read it straight back and compared it with itself, so any change to either model would have silently rewritten its groundtruth and passed. - Every other fixture could only be updated by hand. Bumping FieldItem in the field-metadata work needed FieldItem.yaml edited manually, because DOCLING_GEN_TEST_DATA=1 did nothing for it. A plain run now writes nothing, and the flag regenerates any class. Two details worth keeping: the flag rewrites a fixture only when its parsed content actually differs, because yaml.safe_dump sorts keys while the committed fixtures preserve insertion order -- writing unconditionally reorders all thirteen files without changing what any of them mean. And a missing or unparseable fixture counts as outdated, so a corrupt file does not raise before the regeneration that would have fixed it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com> * refactor(scripts): render every missing snapshot from the live models Review follow-up on #842. The bootstrap path for older target minors still copied docs/DoclingDocument.json and failed when that artefact was absent. _render_model_schema() already produces the same schema in memory, so use it for every missing snapshot and drop _LIVE_SCHEMA. The script no longer touches docs/DoclingDocument.json at all, and the "bootstrap copy of CURRENT_VERSION" message is now accurate: before, the copied artefact could still hold the previous version at hook time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com> --------- Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
428 lines
19 KiB
Python
428 lines
19 KiB
Python
#!/usr/bin/env python3
|
|
"""CI enforcement script — verify projector coverage for DoclingDocument schema.
|
|
|
|
This script is run as a pre-commit hook and in CI to ensure that every
|
|
schema minor-version bump is accompanied by a registered downgrade projector
|
|
AND a versioned JSON Schema snapshot.
|
|
|
|
What it does
|
|
------------
|
|
1. Checks projector count and step coverage (same as before).
|
|
2. Checks SCHEMA_VERSION_HISTORY is up-to-date.
|
|
3. Ensures ``docs/schemas/DoclingDocument_1_{N}.json`` exists for every
|
|
registered projector's target minor version N.
|
|
4. **Auto-generates missing snapshots** by rendering the live models' JSON
|
|
Schema in memory. When CURRENT_VERSION was just bumped this produces
|
|
``docs/schemas/DoclingDocument_1_{CURRENT_MINOR}.json``; for older target
|
|
versions it produces a permissive bootstrap copy (see below).
|
|
|
|
The key invariant
|
|
-----------------
|
|
``docs/DoclingDocument.json`` is regenerated on every commit by the ``docs``
|
|
pre-commit hook (``uv run python -m docling_core.utils.generate_docs docs``).
|
|
When CURRENT_VERSION is bumped from 1.N to 1.(N+1):
|
|
|
|
1. The contributor adds the model changes.
|
|
2. The contributor bumps CURRENT_VERSION.
|
|
3. This script runs (triggered by the change to constants.py).
|
|
4. It writes ``docs/schemas/DoclingDocument_1_(N+1).json`` from the live
|
|
models, so the snapshot describes exactly the version it is named for.
|
|
|
|
The snapshot for the current version is rendered from the models rather than
|
|
copied from ``docs/DoclingDocument.json``, because that artefact still holds
|
|
the 1.N schema when this script runs: the pre-commit hook order is
|
|
|
|
check-compat-projectors ← runs first
|
|
docs ← runs after, regenerates docs/DoclingDocument.json
|
|
|
|
so copying it would produce a file named for 1.(N+1) that describes 1.N, and
|
|
every later structural diff would be taken against the wrong baseline.
|
|
Snapshots for versions older than the current one predate this system and are
|
|
bootstrapped from the same in-memory render, which is a permissive superset of
|
|
every older schema. The script never reads ``docs/DoclingDocument.json``.
|
|
|
|
Exit codes:
|
|
0 — all checks pass (snapshots may have been auto-generated)
|
|
1 — one or more checks failed (prints diagnostics to stderr)
|
|
|
|
Usage::
|
|
|
|
python scripts/check_compat_projectors.py
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
_REPO_ROOT = Path(__file__).parent.parent
|
|
_SCHEMAS_DIR = _REPO_ROOT / "docs" / "schemas"
|
|
|
|
|
|
def _ensure_schemas_dir() -> None:
|
|
_SCHEMAS_DIR.mkdir(parents=True, exist_ok=True)
|
|
|
|
|
|
def _render_model_schema() -> str:
|
|
"""Render the live models' JSON Schema, byte-identical to ``docs/DoclingDocument.json``.
|
|
|
|
Taken from the models rather than from the generated artefact, so the
|
|
snapshot is correct whatever state ``docs/DoclingDocument.json`` happens to
|
|
be in when this runs. Uses the same code path and serialisation as
|
|
``generate_docs`` so the two cannot drift.
|
|
"""
|
|
from docling_core.utils.generate_jsonschema import generate_json_schema
|
|
|
|
schema = generate_json_schema("DoclingDocument")
|
|
return json.dumps(schema, ensure_ascii=False, indent=2)
|
|
|
|
|
|
def _snapshot_path(minor: int) -> Path:
|
|
return _SCHEMAS_DIR / f"DoclingDocument_1_{minor}.json"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Semantic schema diff — detect only breaking changes
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _breaking_changes(old: dict, new: dict) -> list[str]:
|
|
"""Return a list of human-readable breaking-change descriptions.
|
|
|
|
A breaking change is any diff in the JSON Schema that would cause an old
|
|
client's ``DoclingDocument.model_validate()`` to raise when given a
|
|
document produced by the new server.
|
|
|
|
Precisely, Pydantic raises in these four situations:
|
|
|
|
1. **New enum value** — a field value that is valid under the new schema
|
|
is not a member of the old enum, so Pydantic's enum validator raises.
|
|
Detected as: a value present in a ``$defs[X].enum`` array in ``new``
|
|
that is absent from the same array in ``old``.
|
|
|
|
2. **New optional field on a strict model** — every concrete item model
|
|
(``TextItem``, ``TableItem``, …) has ``extra="forbid"`` (rendered as
|
|
``additionalProperties: false`` in the JSON Schema). A new field sent
|
|
by the server will be rejected by the old client's strict model.
|
|
Detected as: a key present in ``$defs[X].properties`` in ``new`` but
|
|
absent in ``old``, where ``old.$defs[X].additionalProperties == false``.
|
|
|
|
3. **New required field** — the old client raises ``missing`` if a field
|
|
in ``required`` is absent from the document. A new required field added
|
|
by the server may not be sent by old servers, but it will be present in
|
|
new server output and the old client must accept it. More precisely:
|
|
if the new server adds it to ``required``, old clients that receive a
|
|
doc *without* it (from any source) will break. We flag it.
|
|
Detected as: a value present in ``$defs[X].required`` in ``new`` but
|
|
absent in ``old``.
|
|
|
|
4. **New DocItem subtype in a union** — a new branch in an ``anyOf``
|
|
discriminator union means the new server can produce documents with
|
|
item types the old client's union does not recognise, causing a
|
|
discriminator-lookup failure.
|
|
Detected as: a new ``$ref`` entry in an ``anyOf`` array at any path
|
|
in ``new`` that is absent in ``old``.
|
|
|
|
Changes that are NOT breaking (and are therefore not reported):
|
|
|
|
- ``title``, ``description``, ``examples`` changes — metadata only.
|
|
- ``default`` changes on optional fields — only affects Python-side
|
|
construction, not wire deserialization.
|
|
- Enum value *removed* in ``new`` — the new server never produces that
|
|
value, so old clients never see it.
|
|
- New ``$defs`` entry that is not yet referenced anywhere — dead code.
|
|
- New ``properties`` key on a model where ``additionalProperties`` is
|
|
not ``false`` — the old client ignores extra fields.
|
|
- Any change inside ``$defs`` entries that are not reachable from the
|
|
top-level ``DoclingDocument`` properties.
|
|
"""
|
|
findings: list[str] = []
|
|
old_defs = old.get("$defs", {})
|
|
new_defs = new.get("$defs", {})
|
|
|
|
# --- 1 & 2 & 3: walk every $defs entry that exists in both schemas -------
|
|
for def_name, new_def in new_defs.items():
|
|
old_def = old_defs.get(def_name)
|
|
if old_def is None:
|
|
# Entirely new def — only breaking if it is immediately used in a
|
|
# union (covered by check 4 below via the anyOf walk).
|
|
continue
|
|
|
|
# 1. Enum values added
|
|
old_enum = set(old_def.get("enum", []))
|
|
new_enum = set(new_def.get("enum", []))
|
|
added_vals = new_enum - old_enum
|
|
if added_vals:
|
|
findings.append(
|
|
f"New enum value(s) in {def_name!r}: {sorted(added_vals)}\n"
|
|
f" Old clients will raise on documents containing these values.\n"
|
|
f" -> Bump CURRENT_VERSION and add a downgrade projector that\n"
|
|
f" remaps these values to the nearest old equivalent."
|
|
)
|
|
|
|
# 2. New properties on strict models (additionalProperties: false)
|
|
if old_def.get("additionalProperties") is False:
|
|
old_props = set(old_def.get("properties", {}).keys())
|
|
new_props = set(new_def.get("properties", {}).keys())
|
|
added_props = new_props - old_props
|
|
if added_props:
|
|
findings.append(
|
|
f"New field(s) on strict model {def_name!r}: {sorted(added_props)}\n"
|
|
f" Old clients have extra='forbid' and will raise on these fields.\n"
|
|
f" -> Bump CURRENT_VERSION and add a downgrade projector that\n"
|
|
f" strips these fields."
|
|
)
|
|
|
|
# 3. New required fields
|
|
old_required = set(old_def.get("required", []))
|
|
new_required = set(new_def.get("required", []))
|
|
added_required = new_required - old_required
|
|
if added_required:
|
|
findings.append(
|
|
f"New required field(s) in {def_name!r}: {sorted(added_required)}\n"
|
|
f" Old clients will raise if these fields are missing.\n"
|
|
f" -> Bump CURRENT_VERSION and add a downgrade projector that\n"
|
|
f" supplies safe defaults for these fields."
|
|
)
|
|
|
|
# --- 4. New branches in anyOf unions (new DocItem subtypes) ---------------
|
|
def _collect_any_of_refs(schema_obj: dict, path: str = "") -> dict[str, set[str]]:
|
|
"""Return {json_path: {$ref, ...}} for every anyOf found."""
|
|
result: dict[str, set[str]] = {}
|
|
if isinstance(schema_obj, dict):
|
|
if "anyOf" in schema_obj:
|
|
refs = {branch["$ref"] for branch in schema_obj["anyOf"] if "$ref" in branch}
|
|
if refs:
|
|
result[path] = refs
|
|
for key, val in schema_obj.items():
|
|
if key in ("title", "description", "examples", "default"):
|
|
continue
|
|
result.update(_collect_any_of_refs(val, f"{path}.{key}"))
|
|
elif isinstance(schema_obj, list):
|
|
for i, item in enumerate(schema_obj):
|
|
result.update(_collect_any_of_refs(item, f"{path}[{i}]"))
|
|
return result
|
|
|
|
old_unions = _collect_any_of_refs(old)
|
|
new_unions = _collect_any_of_refs(new)
|
|
|
|
for path, new_refs in new_unions.items():
|
|
old_refs = old_unions.get(path, set())
|
|
added_refs = new_refs - old_refs
|
|
if added_refs:
|
|
type_names = [r.split("/")[-1] for r in sorted(added_refs)]
|
|
findings.append(
|
|
f"New subtype(s) in union at {path!r}: {type_names}\n"
|
|
f" Old clients' discriminator union does not include these types\n"
|
|
f" and will raise on documents containing them.\n"
|
|
f" -> Bump CURRENT_VERSION and add a downgrade projector that\n"
|
|
f" converts or removes these items."
|
|
)
|
|
|
|
return findings
|
|
|
|
|
|
def main() -> int:
|
|
errors: list[str] = []
|
|
generated: list[str] = []
|
|
|
|
# --- 1. Load the constants module ----------------------------------------
|
|
try:
|
|
from docling_core.types.doc.common.constants import (
|
|
_CURRENT_MINOR,
|
|
CURRENT_VERSION,
|
|
FIRST_SUPPORTED_MINOR,
|
|
SCHEMA_VERSION_HISTORY,
|
|
)
|
|
except ImportError as exc:
|
|
print(f"ERROR: Could not import docling_core constants: {exc}", file=sys.stderr)
|
|
return 1
|
|
|
|
# --- 2. Load the compat module -------------------------------------------
|
|
try:
|
|
from docling_core.compat import _projectors
|
|
except ImportError as exc:
|
|
print(f"ERROR: Could not import docling_core.compat: {exc}", file=sys.stderr)
|
|
return 1
|
|
|
|
_ensure_schemas_dir()
|
|
|
|
# --- 3. Projector count --------------------------------------------------
|
|
expected_count = _CURRENT_MINOR - FIRST_SUPPORTED_MINOR
|
|
actual_count = len(_projectors)
|
|
if actual_count != expected_count:
|
|
errors.append(
|
|
f"Projector count mismatch: expected {expected_count} "
|
|
f"(CURRENT_MINOR={_CURRENT_MINOR} - FIRST_SUPPORTED_MINOR={FIRST_SUPPORTED_MINOR}), "
|
|
f"got {actual_count}.\n"
|
|
f" -> Add a @register_projector(from_minor={_CURRENT_MINOR}, to_minor={_CURRENT_MINOR - 1}) "
|
|
f"function to docling_core/compat.py."
|
|
)
|
|
|
|
# --- 4. Every step covered -----------------------------------------------
|
|
missing_steps = []
|
|
for n in range(FIRST_SUPPORTED_MINOR + 1, _CURRENT_MINOR + 1):
|
|
if (n, n - 1) not in _projectors:
|
|
missing_steps.append(f"1.{n} -> 1.{n - 1}")
|
|
if missing_steps:
|
|
errors.append(
|
|
f"Missing projectors for: {', '.join(missing_steps)}.\n"
|
|
f" -> Each step requires a @register_projector in docling_core/compat.py."
|
|
)
|
|
|
|
# --- 5. SCHEMA_VERSION_HISTORY up-to-date --------------------------------
|
|
history_count = len(SCHEMA_VERSION_HISTORY)
|
|
expected_history_count = _CURRENT_MINOR - FIRST_SUPPORTED_MINOR + 1
|
|
if history_count != expected_history_count:
|
|
errors.append(
|
|
f"SCHEMA_VERSION_HISTORY has {history_count} entries; "
|
|
f"expected {expected_history_count} "
|
|
f"(one per schema version 1.{FIRST_SUPPORTED_MINOR}..1.{_CURRENT_MINOR}).\n"
|
|
f" -> Append an entry for CURRENT_VERSION={CURRENT_VERSION!r} in "
|
|
f"docling_core/types/doc/common/constants.py."
|
|
)
|
|
|
|
# --- 6. Last history entry matches CURRENT_VERSION -----------------------
|
|
if SCHEMA_VERSION_HISTORY:
|
|
last = SCHEMA_VERSION_HISTORY[-1]["schema"]
|
|
if last != CURRENT_VERSION:
|
|
errors.append(
|
|
f"Last SCHEMA_VERSION_HISTORY entry is {last!r}, "
|
|
f"expected {CURRENT_VERSION!r}.\n"
|
|
f" -> The last entry must always match CURRENT_VERSION."
|
|
)
|
|
|
|
# --- 7. Projector keys are single-step downgrades ------------------------
|
|
for key in _projectors:
|
|
if key[0] != key[1] + 1:
|
|
errors.append(f"Projector key {key} is not a single-step downgrade (from_minor must equal to_minor + 1).")
|
|
|
|
# --- 8. JSON Schema snapshots --------------------------------------------
|
|
# For every projector (from_minor -> to_minor), a snapshot for `to_minor`
|
|
# must exist at docs/schemas/DoclingDocument_1_{to_minor}.json.
|
|
#
|
|
# Additionally, a snapshot for CURRENT_MINOR must exist (used to validate
|
|
# the output of any *future* projector that targets the current version).
|
|
#
|
|
# Auto-generation strategy: every missing snapshot is written from
|
|
# _render_model_schema(), the live models' JSON Schema rendered in memory.
|
|
# The script never reads docs/DoclingDocument.json, so it works whatever
|
|
# state that artefact is in (stale, or not yet generated).
|
|
# - For CURRENT_MINOR the render *is* the schema the file is named for.
|
|
# - For older target versions (FIRST_SUPPORTED_MINOR..CURRENT_MINOR-1)
|
|
# the exact schema predates this system and cannot be reconstructed.
|
|
# The render serves as a permissive bootstrap: the current schema is a
|
|
# superset of all older schemas, so it will accept — but not strictly
|
|
# validate — old dicts. A note is printed so maintainers know these
|
|
# are bootstrap copies to replace if the exact schema is available.
|
|
#
|
|
# Going forward, the flow is:
|
|
# 1. The contributor bumps CURRENT_VERSION from 1.N to 1.(N+1).
|
|
# 2. This script renders DoclingDocument_1_{N+1}.json from the models.
|
|
# 3. The `docs` hook regenerates docs/DoclingDocument.json afterwards.
|
|
# Each snapshot is therefore the exact schema that clients at that version saw.
|
|
|
|
all_target_minors = {to_minor for _, to_minor in _projectors} | {_CURRENT_MINOR}
|
|
|
|
for minor in sorted(all_target_minors):
|
|
snap = _snapshot_path(minor)
|
|
if snap.exists():
|
|
# Verify it is valid JSON.
|
|
try:
|
|
json.loads(snap.read_text())
|
|
except json.JSONDecodeError as exc:
|
|
errors.append(f"Snapshot {snap.name} is not valid JSON: {exc}")
|
|
continue
|
|
|
|
# Snapshot missing — render it from the live models. Never copy
|
|
# docs/DoclingDocument.json: at this point in the hook order it still
|
|
# holds the *previous* schema, so a copy would silently snapshot the
|
|
# wrong version under the right name.
|
|
snap.write_text(_render_model_schema(), encoding="utf-8")
|
|
if minor == _CURRENT_MINOR:
|
|
generated.append(
|
|
f" docs/schemas/DoclingDocument_1_{minor}.json (snapshot of current schema {CURRENT_VERSION})"
|
|
)
|
|
else:
|
|
# Older target versions predate this system, so their exact schema
|
|
# cannot be reconstructed; the render is a permissive bootstrap.
|
|
generated.append(
|
|
f" docs/schemas/DoclingDocument_1_{minor}.json "
|
|
f"[bootstrap copy of {CURRENT_VERSION} schema — "
|
|
f"replace with exact 1.{minor} schema if available]"
|
|
)
|
|
|
|
# --- 9. Breaking schema diff against current snapshot --------------------
|
|
# Compare the live models' schema against
|
|
# docs/schemas/DoclingDocument_1_{CURRENT_MINOR}.json (the snapshot taken at
|
|
# the last CURRENT_VERSION bump). Read from the models rather than from
|
|
# docs/DoclingDocument.json so the check does not depend on whether the
|
|
# `docs` hook has regenerated that artefact yet.
|
|
#
|
|
# If the live schema has breaking changes relative to the snapshot AND
|
|
# CURRENT_VERSION has not been bumped, the contributor forgot to announce
|
|
# the breaking change. We report exactly what changed and why it matters.
|
|
#
|
|
# If CURRENT_VERSION *was* bumped (the snapshot for CURRENT_MINOR was just
|
|
# auto-generated in step 8 above and reflects the new schema), this diff
|
|
# will be empty — the snapshot IS the current schema.
|
|
current_snap = _snapshot_path(_CURRENT_MINOR)
|
|
if current_snap.exists():
|
|
try:
|
|
live_schema = json.loads(_render_model_schema())
|
|
snap_schema = json.loads(current_snap.read_text())
|
|
breaking = _breaking_changes(old=snap_schema, new=live_schema)
|
|
if breaking:
|
|
msg = (
|
|
f"Breaking schema change(s) detected without a CURRENT_VERSION bump.\n"
|
|
f" The live models differ from "
|
|
f"docs/schemas/DoclingDocument_1_{_CURRENT_MINOR}.json "
|
|
f"in {len(breaking)} breaking way(s):\n\n"
|
|
)
|
|
for i, b in enumerate(breaking, 1):
|
|
msg += f" [{i}] {b}\n\n"
|
|
msg += (
|
|
" If this change is intentional, follow the checklist in\n"
|
|
" CONTRIBUTING.md §'Breaking changes and schema bumps':\n"
|
|
" bump CURRENT_VERSION, add a projector, and add a test class."
|
|
)
|
|
errors.append(msg)
|
|
except json.JSONDecodeError:
|
|
pass # corrupt snapshot — already caught in step 8
|
|
|
|
# --- Report --------------------------------------------------------------
|
|
if errors:
|
|
print("FAIL: docling_core/compat.py projector coverage check\n", file=sys.stderr)
|
|
for i, err in enumerate(errors, 1):
|
|
print(f" [{i}] {err}\n", file=sys.stderr)
|
|
print(
|
|
"See the contributor guide in CONTRIBUTING.md §'Breaking changes and schema bumps'.",
|
|
file=sys.stderr,
|
|
)
|
|
return 1
|
|
|
|
if generated:
|
|
print(
|
|
f"OK: {actual_count} projector(s) registered. Auto-generated {len(generated)} schema snapshot(s):",
|
|
file=sys.stdout,
|
|
)
|
|
for msg in generated:
|
|
print(msg, file=sys.stdout)
|
|
print(
|
|
" -> Add the new snapshot file(s) to your commit.",
|
|
file=sys.stdout,
|
|
)
|
|
else:
|
|
print(
|
|
f"OK: {actual_count} projector(s) registered for "
|
|
f"schema 1.{FIRST_SUPPORTED_MINOR}..1.{_CURRENT_MINOR}, "
|
|
f"all snapshots present.",
|
|
file=sys.stdout,
|
|
)
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|