5 Commits
Author SHA1 Message Date
Peter W. J. Staar 6804d5cf49 chore: add maintainer script to rank open PRs by merge readiness (#992)
chore: updating the classification of PR's via updated labels

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-11-09 12:40:28 +01:00
Nguyen Hoang DuongandPanos Vagenas 1707a2dda4 feat(doclang): add partial DocLang 0.8 support with multiple captions (#880)
* chore(doclang): align w doclang 0.8

Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>

* feat(doclang): support multiple captions and skip track for DocLang 0.8

DocLang 0.8 allows several captions in an element head. The serializer
now writes all captions of a floating item there, instead of the first
one in the head and the others as <text> next to it in a <group>. The
deserializer reads all head captions, and still reads <text> siblings
as captions, so that earlier output keeps loading.

<track> is not supported yet: the deserializer skips it with a warning,
instead of dropping its cues and leaking its chapter texts. The module
docstrings state the support as partial.

doclang 0.8 selects the schema by the declared version and warns when
there is none. The test helpers validate documents without a version
against _DOCLANG_VERSION. The data lint gets a dclx.version check that
warns on a missing version, and fails validation on a declared version
without schema rather than falling back to the latest one.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com>
2026-10-21 10:07:13 +02:00
Jeff WittandClaude Fable 5.1 78d954ff57 fix(scripts): correct the schema-snapshot capture and the test-data gen flag (#842)
* fix(scripts): render the current schema snapshot from the live models

When CURRENT_VERSION is bumped from 1.N to 1.(N+1), the coverage check
creates the missing docs/schemas/DoclingDocument_1_(N+1).json by copying
docs/DoclingDocument.json. But the hook order it relies on --

  check-compat-projectors  <- runs first
  docs                     <- runs after, regenerates the artefact

-- guarantees that artefact still holds the 1.N schema at that moment. So
the copy produces a file named for 1.(N+1) that describes 1.N, and because
check 9 then compares the same stale artefact against it, the diff is empty
and the run passes. The wrong baseline is only noticed later, as a
"breaking change without a CURRENT_VERSION bump" error on the next commit,
pointing the contributor at a bump they already made.

Observed while bumping to 1.11.0: the generated DoclingDocument_1_11.json
was byte-identical to DoclingDocument_1_10.json.

The snapshot for the current version is now rendered from the models via
generate_json_schema -- the same code path and serialisation generate_docs
uses, so the two cannot drift -- and check 9 compares the models rather
than the artefact. Neither check now depends on whether the docs hook has
run, which removes the ordering coupling instead of documenting it.
Snapshots for older versions predate this system and keep the permissive
bootstrap copy.

CONTRIBUTING and the script docstring both asserted that the hook ordering
made the copy safe, which is what made the trap hard to see; both are
corrected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>

* test: let DOCLING_GEN_TEST_DATA govern every per-item fixture

test_docitems' verify() helper did not consult the flag at all. It wrote
groundtruth for KeyValueItem and FormItem on every run, flag or no flag,
and for no other class ever. Two consequences:

- Those two fixtures were never really asserted. verify() wrote the file,
  read it straight back and compared it with itself, so any change to
  either model would have silently rewritten its groundtruth and passed.
- Every other fixture could only be updated by hand. Bumping FieldItem in
  the field-metadata work needed FieldItem.yaml edited manually, because
  DOCLING_GEN_TEST_DATA=1 did nothing for it.

A plain run now writes nothing, and the flag regenerates any class.

Two details worth keeping: the flag rewrites a fixture only when its
parsed content actually differs, because yaml.safe_dump sorts keys while
the committed fixtures preserve insertion order -- writing unconditionally
reorders all thirteen files without changing what any of them mean. And a
missing or unparseable fixture counts as outdated, so a corrupt file does
not raise before the regeneration that would have fixed it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>

* refactor(scripts): render every missing snapshot from the live models

Review follow-up on #842. The bootstrap path for older target minors
still copied docs/DoclingDocument.json and failed when that artefact was
absent. _render_model_schema() already produces the same schema in
memory, so use it for every missing snapshot and drop _LIVE_SCHEMA.

The script no longer touches docs/DoclingDocument.json at all, and the
"bootstrap copy of CURRENT_VERSION" message is now accurate: before, the
copied artefact could still hold the previous version at hook time.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>

---------

Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-12 17:14:46 +02:00
Panos Vagenas 2887ddfa13 chore(scripts): add private data lint for DoclingDocument JSON and DocLang (#858)
* chore(scripts): add a private data lint for DoclingDocument JSON and DocLang files

scripts/_data_lint.py checks a .json, a .dclx/.dclg/.xml, or a pair (--pair): load and model rules, schema validity, deserializability, the DocLang round trip, and JSON <-> DocLang conversion, with --only/--skip, --list-checks and --report. Includes a readme and tests.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(scripts): fail the data lint when a check cannot run

A missing Schematron backend made dclx.schematron pass as 'skipped', so a run could end OK with a check missing. It now fails with a hint (install doclang[schematron-saxon], or --skip dclx.schematron), and the skip status is removed.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-10-09 20:52:20 +02:00
Cesar Berrospi Ramis f756342105 feat(compat): DoclingDocument downgrade projector system (#703)
* feat(compat): add DoclingDocument version-downgrade projector system

Adds docling_core/compat.py with a register_projector decorator,
project_to() API, and five retroactive projectors (1.10→1.9→1.8→1.7→
1.6→1.5). Adds scripts/check_compat_projectors.py as a pre-commit hook
that detects unannounced structural breaking changes and enforces that
every schema minor bump ships a projector, a JSON Schema snapshot, and a
test class. 36 new tests in test/test_compat.py; CONTRIBUTING.md
documents the change-classification table and 7-step checklist.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore: upgrade uv.lock dependencies

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-08-18 10:29:48 +02:00