Peter W. J. Staar
8951e6f202
chore: removing all legacy / deepsearch code ( #644 )
...
* What was removed
The legacy_doc document model and the rec (Record) types, plus all code/resources/tests that existed only to serve them.
Deleted packages/modules
- docling_core/types/legacy_doc/ (whole folder)
- docling_core/types/rec/ (whole folder)
- docling_core/utils/legacy.py (legacy conversion functions)
- docling_core/utils/validate.py (RAW/ANN/OCR CLI)
Edited production code
- types/__init__.py — dropped the Record export
- utils/validators.py — removed validate_schema/validate_raw_schema/validate_ann_schema/validate_ocr_schema and their now-unused imports; kept
validate_unique_list/validate_datetime
- utils/generate_docs.py — dropped Record from MODELS
Removed resources/artifacts
- resources/schemas/doc/ (legacy RAW/ANN/OCR/DOC), resources/schemas/generated/ccs_document_schema.json, docs/Record.json
Tests — deleted the 5 legacy/rec-only suites + 2 stale gold files; trimmed test_collection.py, test_base.py, test_doc_base.py to keep their non-legacy parts;
replaced the _create_hash import in test_code_chunker.py with a local helper. Removed test/data/legacy_doc/ and test/data/rec/.
Verification
- No dangling code references remain (only string literals and the vendored _reference/chunker_repo scratch data, intentionally untouched).
- from docling_core.types import DoclingDocument, Generic works; Record now correctly raises ImportError.
- Full project suite: 496 passed.
- generate_docs.py now emits only DoclingDocument.json and Generic.json.
The full plan is saved at _plans/remove-legacy-doc-and-rec.md.
Two notes worth flagging:
- Breaking public API: docling_core.types.Record and the docling_core.utils.legacy conversion functions were public — downstream docling uses
legacy_to_docling_document/docling_document_to_legacy. The branch name (dev/remove-legacy-document) suggests this is intended, but it'll need a coordinated
bump downstream.
- I left _reference/ and _plans/ alone (untracked scratch dirs). Nothing was committed — all changes are staged/working-tree only, per your edit-by-edit
approval flow.
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the checks.yaml
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* removed docling_core/types/gen/, docling_core/types/nlp, docling_core/types/legacy_doc, docling_core/types/rec
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* removed the minimal_document_schema_flat.json
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* removed docling_core/resources
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* removed the files and updated the imports according to Cesar's suggestion
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* fixed the ruff linter
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
2026-06-18 17:00:52 +02:00
Michele Dolfi
01373c4aab
fix: drop python 3.9 and pin built tree-sitter versions ( #487 )
...
* drop python 3.9
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* apply linter for python 3.10 changes
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2026-01-22 19:40:35 +01:00
Michele Dolfi
1a023dad6f
chore: fix discord ( #406 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-10-17 17:08:41 +02:00
Michele Dolfi
9fa02d08e3
chore: publish release notes on Discord ( #405 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-10-17 16:31:40 +02:00
Michele Dolfi
47b70d3c24
feat: python 3.14 compatibility ( #403 )
...
update deps for python 3.14 compatibility
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-10-16 15:58:29 +02:00
Michele Dolfi
73113f6a43
ci: add dco-advisor ( #331 )
...
* add dco-advisor
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix yaml syntax
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* avoid js template syntax
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* search comment more robust
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* this commit is not signed-off on purpose
* update the logic for sha filtering
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* this commit is without sign-off
* add debug output
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* skip itself as it contains dco in name
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* restore pull_request on
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* use pr checks
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix trigger
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* new status check
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* chore: DCO remediation for failing commits
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* DCO Remediation Commit for Michele Dolfi <dol@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* more logs
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add output text to logs
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* use logic scanning commits
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* print html_url
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix map function
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add username
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix body arg
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add link to dco details
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* DCO Remediation Commit for Michele Dolfi <dol@zurich.ibm.com >
I, Michele Dolfi <dol@zurich.ibm.com >, hereby add my Signed-off-by to this commit: 5c1ef004ff
I, Michele Dolfi <dol@zurich.ibm.com >, hereby add my Signed-off-by to this commit: 761f62fc3f
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* use correct event
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-06-17 08:37:41 +02:00
1b0b39b0b4
refactor: use uv as dependencies management and packaging ( #307 )
...
* use new pyproject.toml format with uv
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* update ci/cd scripts
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* update MD files
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* build without pre-commit cache
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* run pre-commit from uv
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* small changes
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add uv package install
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Apply suggestions from code review
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com >
* docs: update README
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com >
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com >
2025-05-28 13:06:06 +02:00
Cesar Berrospi Ramis
1af07218e1
fix(codecov): fix codecov argument and yaml file ( #260 )
...
* fix(codecov): fix codecov argument and yaml file
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* ci: set the codecov status to success even if the CI fails
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
---------
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
2025-04-15 16:53:33 +02:00
Michele Dolfi
64bafa1afe
ci: add tests in ci with coverage upload ( #257 )
...
* add coverage of tests
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* use coverage from pre-commit
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-04-15 10:08:07 +02:00
Michele Dolfi
8b5dd171a5
ci: fix new pypi build ( #255 )
...
fix new pypi build
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com >
2025-04-15 06:24:33 +02:00
Michele Dolfi
37d545a1c8
ci: use trusted publishers and sign releases ( #254 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-04-14 19:15:39 +02:00
Michele Dolfi
5739033408
chore: move to docling-project gh org ( #190 )
...
* chore: move to docling-project gh org
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* retrigger ci with DCO
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-03-14 09:28:17 +01:00
Panos Vagenas
ee2d67b2ec
chore: use GH cache for HF models ( #183 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-03-04 13:12:44 +01:00
Panos Vagenas
0cd8b94ca6
chore: switch release pipeline to manual triggering ( #56 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-24 13:02:39 +02:00
Michele Dolfi
5cc6ba73f7
chore: allow forks with synchronize event ( #46 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-10-18 12:56:26 +02:00
03df97fa73
feat!: Expose DoclingDocument as main type, move old typing to legacy ( #41 )
...
* Fix area method of BoundingBox
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* add image placeholder
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* enable picture label
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* refactor captions and markdown
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add logic to skip repeated caption
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* use DocItemLabel
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Extend default export labels, add convenience mehtods
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Introduce ListItem API, with marker and enumerated properties
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* add classification and description in PictureData
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add molecule picture data
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Fixes for DoclingDocument and aligned methods on legacy doc
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* add advanced picture data content
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Many markdown export fixes, renaming BaseTableData
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Rename module paths doc->legacy_doc, experimental->doc
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* feat: imageref with pil_image
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Small fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* docs: remove documentation in markdown to support python 3.13 (#43 )
Since json-schema-for-humans dependency does not support python 3.13,
remove the generation of documentation in markdown of main docling types.
Remove 'ds' prefix from documentation scripts.
Update README.
Add python 3.13 in CI/CD workflow checks.
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* Fix TableCell model validator
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* store list of classes in classification
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Fixes for DocumentOrigin mimetype validation
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* introduce picturedata as list of annotations
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* feat: adapt hierarchical chunker to v2 DoclingDocument
[skip-ci]
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* feat: add table support in chunker, incl. captions
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* use Field constraints instead of conlist, refactor chunking types
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* revert unnecessary doc module change
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* align test data with upstream changes
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* Update __init__.py on docling_core.types.doc
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Remove DescriptionItem
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-16 15:53:44 +02:00
Michele Dolfi
1a4440a771
chore: switch to gh apps user ( #26 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-09-20 16:40:30 +02:00
Michele Dolfi and Panos Vagenas
4e5dbdf703
ci: add cicd ( #1 )
...
* single source of version
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add cicd
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* apply pre-commit validators
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Update pyproject.toml
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com >
* update lock
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-07-17 17:43:02 +02:00