* What was removed
The legacy_doc document model and the rec (Record) types, plus all code/resources/tests that existed only to serve them.
Deleted packages/modules
- docling_core/types/legacy_doc/ (whole folder)
- docling_core/types/rec/ (whole folder)
- docling_core/utils/legacy.py (legacy conversion functions)
- docling_core/utils/validate.py (RAW/ANN/OCR CLI)
Edited production code
- types/__init__.py — dropped the Record export
- utils/validators.py — removed validate_schema/validate_raw_schema/validate_ann_schema/validate_ocr_schema and their now-unused imports; kept
validate_unique_list/validate_datetime
- utils/generate_docs.py — dropped Record from MODELS
Removed resources/artifacts
- resources/schemas/doc/ (legacy RAW/ANN/OCR/DOC), resources/schemas/generated/ccs_document_schema.json, docs/Record.json
Tests — deleted the 5 legacy/rec-only suites + 2 stale gold files; trimmed test_collection.py, test_base.py, test_doc_base.py to keep their non-legacy parts;
replaced the _create_hash import in test_code_chunker.py with a local helper. Removed test/data/legacy_doc/ and test/data/rec/.
Verification
- No dangling code references remain (only string literals and the vendored _reference/chunker_repo scratch data, intentionally untouched).
- from docling_core.types import DoclingDocument, Generic works; Record now correctly raises ImportError.
- Full project suite: 496 passed.
- generate_docs.py now emits only DoclingDocument.json and Generic.json.
The full plan is saved at _plans/remove-legacy-doc-and-rec.md.
Two notes worth flagging:
- Breaking public API: docling_core.types.Record and the docling_core.utils.legacy conversion functions were public — downstream docling uses
legacy_to_docling_document/docling_document_to_legacy. The branch name (dev/remove-legacy-document) suggests this is intended, but it'll need a coordinated
bump downstream.
- I left _reference/ and _plans/ alone (untracked scratch dirs). Nothing was committed — all changes are staged/working-tree only, per your edit-by-edit
approval flow.
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* updated the checks.yaml
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* removed docling_core/types/gen/, docling_core/types/nlp, docling_core/types/legacy_doc, docling_core/types/rec
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* removed the minimal_document_schema_flat.json
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* removed docling_core/resources
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* removed the files and updated the imports according to Cesar's suggestion
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* fixed the ruff linter
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* refactor: move WebVTT data model from docling
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* fix(webvtt): deal with HTML entities in cue text spans
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(webvtt): support more WebVTT models
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(DoclingDocument): create a new provenance model for media file types
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(webvtt): make WebVTTTimestamp public
Since WebVTTTimestamp is used in DoclingDocument, the class should be public.
Strengthen validation of cue language start tag annotation.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(webvtt): set languages to a list of strings in ProvenanceTrack
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* tests(webvtt): add test for ProvenanceTrack
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(webvtt): make all WebVTT classes public for reuse
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore(webvtt): preserve newlines as WebVTTLineTerminator
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(webvtt): set ProvenanceTrack time fields as float
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore(webvtt): ensure start time offsets are in sequence
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore(webvtt): improve regex to remove note,region,style blocks
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore(webvtt): parse the WebVTT file title
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore(webvtt): rebase to latest changes in idoctags
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* feat(webvtt): add WebVTT serializer
Add a DoclingDocument serializer to WebVTT format.
Improve WebVTT data model.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* fix(webvtt): add 'text/vtt' as extra mimetype
Add 'text/vtt' as extra MIME type to support WebVTT serialization, since it is not
supported by 'mimetypes' with python < 3.11
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(webvtt): roll back DocItem.prov as list of ProvenanceItem
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* tests(webvtt): fix test with STYLE and NOTE blocks
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* style(webvtt): apply X | Y annotation instead of Optional, Union
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(webvtt): simplify TrackProvenance model with tags
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(webvtt): align class and field names to new 'source' type
Classes and fields that are related to the new source type should aign with their names.
The term 'provenance' will identify the legacy implementation.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore(DoclingDocument): drop the validation on field assignment
Drop the validation on field assignment in NodeItem objects.
Add the 'source' argument in the convenient function 'add_text' to create TextItem with track source data.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
refactor(webvtt): drop cue span classes, 'lang' and 'c' tags
Drop WebVTT formatting features not covered by Docling across formats.
Only 'u', 'b', 'i', and 'v' are supported and without classes.
Make 'v' tag explicit as 'voice' feature in SourceTrack class.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
---------
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore: remove unnecesary copyright headers
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* reformatted, still need to fix a test ...
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* fix(test): set static typing compatible to python 3.10
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
* style: enforce some pre-commit hooks on tests
Enforce pre-commit hooks black, isort, autoflake, and mypy on test modules.
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
* fix(rec): fix definition issues in attribute, predicate, subject
Remove duplicate generic types across base and predicate modules.
Create an identifier class for subject names.
Remove unnecessary type variables in attribute model.
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
* docs: refer to Docling data objects
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
---------
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>