* feat: add initial dlgrep implementation
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* refactor: migrate CLI to Typer
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* docs: expand README
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* docs: improve README examples
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* fix: streamline dlgrep output
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* fix: make text context output grep-compatible
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Improve XPath and inline context output
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* dlgrep support extensions
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Use git sources for doclang and docling-core
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* feat: emit native JSON records
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Remove search class filter
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Make section output addressable
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Improve README examples
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* feat: build and publish dlgrep from this repository
dlgrep becomes a uv workspace member under packages/dlgrep, published as
its own wheel and released in lockstep with docling-core: same version,
exact `docling-core==` pin, both wheels built from the same commit. This
removes the git-branch dependency dlgrep needed to track unreleased
docling-core changes.
- workspace wiring; ruff, mypy and pytest config inherited from the root,
dlgrep in the dev group so `uv sync` covers it
- release.sh writes both versions and the pin from one TARGET_VERSION
- build-packages.sh builds into per-package dist dirs; pypi.yml publishes
docling-core first, then dlgrep
- check_lockstep.py guards against version drift, which uv cannot catch on
its own: the workspace source replaces the pin with a local path, so a
wrong pin resolves and locks cleanly
- checks.yml and pypi.yml install both wheels together, which makes a
mismatched pin fail before anything is published
- declare click and lxml, previously only transitive
- read __version__ from package metadata instead of a hardcoded literal
- flag dlgrep as experimental in its help output, since the shared version
number would otherwise imply a maturity it does not have
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* ci: simplify lockstep release tooling
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* chore(dlgrep): drop handoff notes, keep only the CLI plan
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* refactor(doclang): isolate source mapping recorder
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* build(dlgrep): move dependencies to core extra
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* feat(dlq): rename CLI and add list command
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* feat(dclq): rename tool and consolidate plan
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* fix(dclq): bound archive document reads
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* What was removed
The legacy_doc document model and the rec (Record) types, plus all code/resources/tests that existed only to serve them.
Deleted packages/modules
- docling_core/types/legacy_doc/ (whole folder)
- docling_core/types/rec/ (whole folder)
- docling_core/utils/legacy.py (legacy conversion functions)
- docling_core/utils/validate.py (RAW/ANN/OCR CLI)
Edited production code
- types/__init__.py — dropped the Record export
- utils/validators.py — removed validate_schema/validate_raw_schema/validate_ann_schema/validate_ocr_schema and their now-unused imports; kept
validate_unique_list/validate_datetime
- utils/generate_docs.py — dropped Record from MODELS
Removed resources/artifacts
- resources/schemas/doc/ (legacy RAW/ANN/OCR/DOC), resources/schemas/generated/ccs_document_schema.json, docs/Record.json
Tests — deleted the 5 legacy/rec-only suites + 2 stale gold files; trimmed test_collection.py, test_base.py, test_doc_base.py to keep their non-legacy parts;
replaced the _create_hash import in test_code_chunker.py with a local helper. Removed test/data/legacy_doc/ and test/data/rec/.
Verification
- No dangling code references remain (only string literals and the vendored _reference/chunker_repo scratch data, intentionally untouched).
- from docling_core.types import DoclingDocument, Generic works; Record now correctly raises ImportError.
- Full project suite: 496 passed.
- generate_docs.py now emits only DoclingDocument.json and Generic.json.
The full plan is saved at _plans/remove-legacy-doc-and-rec.md.
Two notes worth flagging:
- Breaking public API: docling_core.types.Record and the docling_core.utils.legacy conversion functions were public — downstream docling uses
legacy_to_docling_document/docling_document_to_legacy. The branch name (dev/remove-legacy-document) suggests this is intended, but it'll need a coordinated
bump downstream.
- I left _reference/ and _plans/ alone (untracked scratch dirs). Nothing was committed — all changes are staged/working-tree only, per your edit-by-edit
approval flow.
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* updated the checks.yaml
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* removed docling_core/types/gen/, docling_core/types/nlp, docling_core/types/legacy_doc, docling_core/types/rec
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* removed the minimal_document_schema_flat.json
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* removed docling_core/resources
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* removed the files and updated the imports according to Cesar's suggestion
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* fixed the ruff linter
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com>