Commit Graph
229 Commits
Author SHA1 Message Date
github-actions[bot] 5fa6cae721 chore: bump version to 2.89.0 [skip ci] 2026-07-31 12:06:39 +00:00
Christoph Auer cf2ac70632 feat: dclq tool (#696)
* feat: add initial dlgrep implementation

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* refactor: migrate CLI to Typer

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* docs: expand README

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* docs: improve README examples

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* fix: streamline dlgrep output

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* fix: make text context output grep-compatible

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Improve XPath and inline context output

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* dlgrep support extensions

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Use git sources for doclang and docling-core

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* feat: emit native JSON records

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Remove search class filter

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Make section output addressable

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Improve README examples

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* feat: build and publish dlgrep from this repository

dlgrep becomes a uv workspace member under packages/dlgrep, published as
its own wheel and released in lockstep with docling-core: same version,
exact `docling-core==` pin, both wheels built from the same commit. This
removes the git-branch dependency dlgrep needed to track unreleased
docling-core changes.

- workspace wiring; ruff, mypy and pytest config inherited from the root,
  dlgrep in the dev group so `uv sync` covers it
- release.sh writes both versions and the pin from one TARGET_VERSION
- build-packages.sh builds into per-package dist dirs; pypi.yml publishes
  docling-core first, then dlgrep
- check_lockstep.py guards against version drift, which uv cannot catch on
  its own: the workspace source replaces the pin with a local path, so a
  wrong pin resolves and locks cleanly
- checks.yml and pypi.yml install both wheels together, which makes a
  mismatched pin fail before anything is published
- declare click and lxml, previously only transitive
- read __version__ from package metadata instead of a hardcoded literal
- flag dlgrep as experimental in its help output, since the shared version
  number would otherwise imply a maturity it does not have

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* ci: simplify lockstep release tooling

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* chore(dlgrep): drop handoff notes, keep only the CLI plan

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* refactor(doclang): isolate source mapping recorder

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* build(dlgrep): move dependencies to core extra

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* feat(dlq): rename CLI and add list command

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* feat(dclq): rename tool and consolidate plan

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* fix(dclq): bound archive document reads

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
2026-07-31 13:38:43 +02:00
Daniel KapitanandCopilot 6b8d35ff28 fix: relax typer upper bound to <0.27.0 (#697)
* Relax typer upper bound to <0.27.0

Match docling's own typer constraint (>=0.12.5,<0.27.0). The
<0.25.0 pin appears overly conservative: docling_core/cli/serialize.py
and view.py only use stable Typer/Click APIs (Typer, Exit, Argument,
Option, BadParameter, echo, typer.main.get_command), verified working
under typer 0.26.8.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* DCO Remediation Commit for Daniel Kapitan <daniel@kapitan.net>

I, Daniel Kapitan <daniel@kapitan.net>, hereby add my Signed-off-by to this commit: fcc22ce120

Signed-off-by: Daniel Kapitan <daniel@kapitan.net>

* chore: update uv.lock for relaxed typer upper bound

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* DCO Remediation Commit for Daniel Kapitan <daniel@kapitan.net>

I, Daniel Kapitan <daniel@kapitan.net>, hereby add my Signed-off-by to this commit: fcc22ce120
I, Daniel Kapitan <daniel@kapitan.net>, hereby add my Signed-off-by to this commit: cfd2fc94b1

Signed-off-by: Daniel Kapitan <daniel@kapitan.net>

---------

Signed-off-by: Daniel Kapitan <daniel@kapitan.net>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-28 14:26:05 +02:00
github-actions[bot] c357d5c6c4 chore: bump version to 2.88.0 [skip ci] 2026-07-27 11:53:54 +00:00
github-actions[bot] 0215808e9b chore: bump version to 2.87.1 [skip ci] 2026-07-15 09:10:55 +00:00
github-actions[bot] 9e9c28f309 chore: bump version to 2.87.0 [skip ci] 2026-07-11 11:36:02 +00:00
Cesar Berrospi Ramis e71da6bf05 fix(deps): constrain numpy <2.5.0 and add doclang schematron-saxon to dev (#669)
* fix(mypy): constrain numpy to '<2.5.0' to avoid syntax error

numpy 2.5.0 ships stubs that require Python 3.12 syntax, it can be installed into a 3.12 environment
that is being type-checked against a 3.10 language target.
Constraint numpy to '<2.5.0' for mypy compatibility checks.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore: upgrade dependencies in uv.lock

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* fix(dev): add doclang schematron-saxon extra to dev dependencies

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-07-03 17:36:00 +02:00
github-actions[bot] c3177448b3 chore: bump version to 2.86.0 [skip ci] 2026-07-03 11:20:50 +00:00
NintoracandCesar Berrospi Ramis be42fcea2e feat: add fsspec support (#444)
* feat: add universal-pathlib support for document to_*/from_* methods

Enable cloud storage backends (S3, GCS, Azure, etc.) for document
serialization methods via fsspec/universal-pathlib integration.

Signed-off-by: Nintorac <Nintorac@users.noreply.github.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* test(fsspec): remove universal-pathlib check

Remove the import check for universal-pathlib library and the decorators
'@pytest.mark.skipif' since it is unnecessary and adds complexity.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore: remove potentially misleading comment

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* fix(types): convert image URI path to AnyUrl for type safety

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* test(upath): add error handling for remote paths in 'relative_path()'

Catch error in case 'relative_path()' is used with remote paths and 'resolve' fails.
Consolidate the tests in test_upath_support.py module.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* fix: misplaced block, annotations, tests

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor: deduplicate code in '_with_pictures_refs'

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Nintorac <Nintorac@users.noreply.github.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-07-02 16:36:19 +02:00
github-actions[bot] cf8648b6f1 chore: bump version to 2.85.0 [skip ci] 2026-06-25 16:15:48 +00:00
Panos Vagenas 3704874312 feat(doclang): add support for .dclx archive, <description>, <summary> (#658)
* feat(doclang): add .dclx archive save and load support

- Add `DoclingDocument.save_as_doclang_archive()` and `load_from_doclang_archive()` for DocLang OPC archives (`.dclx`)
- Store markup in `document.xml` with page rasters under `pages/` and picture assets under `assets/`
- Resolve relative `<src uri="..."/>` paths when loading from an extracted archive
- Bump DocLang serialization to version `0.7` and update namespace/version golden fixtures
- Restore picture images from archive-relative URIs via `DocLangDocDeserializer` (`media_root`)
- Add safe zip extraction and archive path helpers in `docling_core.types.doc.utils`
- Promote `doclang` to a runtime dependency (`>=0.7,<0.8`) on PyPI 0.7.0
- Remove `doclang` from dev dependency group and refresh `uv.lock`
- Add `test/test_doclang_archive.py` with save, load, and roundtrip coverage
- Add two-page archive fixtures under `test/data/doc/doclang_archive/`

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* feat(doclang): add support for `<description>` and `<summary>`

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-06-25 18:06:10 +02:00
github-actions[bot] dff585385a chore: bump version to 2.84.0 [skip ci] 2026-06-23 09:44:00 +00:00
github-actions[bot] ef5091295e chore: bump version to 2.83.1 [skip ci] 2026-06-19 05:45:12 +00:00
github-actions[bot] ce0523ea84 chore: bump version to 2.83.0 [skip ci] 2026-06-18 15:33:58 +00:00
Max SwainandChristoph Auer ab3666c081 perf: Use OpenCV over PIL for PNG encoding in ImageRef.from_pil (#562)
* perf: Use OpenCV over PIL for PNG encoding

Signed-off-by: Max Swain <89113255+maxdswain@users.noreply.github.com>

* refactor: Make opencv optional and add fallback to PIL

Signed-off-by: Max Swain <89113255+maxdswain@users.noreply.github.com>

* refactor: Update opencv dependency to use dev group

Signed-off-by: Max Swain <89113255+maxdswain@users.noreply.github.com>

* Fix RGBA vs RGB handling in openCV

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Update test GT

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* empty commit to trigger DCO

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Max Swain <89113255+maxdswain@users.noreply.github.com>
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Co-authored-by: Christoph Auer <cau@zurich.ibm.com>
2026-06-17 16:44:14 +02:00
Cesar Berrospi Ramis 2a78701b3f chore: update pre-commit hooks and development dependencies (#638)
* chore: update pre-commit hooks and development dependencies

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* docs: update CONTRIBUTING after pre-commit hook changes

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-06-15 17:07:34 +02:00
github-actions[bot] e3360a0562 chore: bump version to 2.82.0 [skip ci] 2026-06-12 08:57:42 +00:00
Panos Vagenas 0b7690509e feat(doclang): align with DocLang v0.6, add field region deser. (#636)
* feat(doclang): align with DocLang v0.6 picture model

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* feat(doclang): enhance picture metadata handling with custom fields

- Introduced support for custom metadata fields in the picture model, including summary, description, and SMILES.
- Updated serializer and deserializer to handle new custom vocabulary tags.
- Added tests for round-trip serialization and deserialization of picture metadata, ensuring correct handling of custom fields.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* feat(doclang): deserialize field regions and add round-trip fixtures

- Parse `<field_region>`, `<field_item>`, `<key>`, `<value>`, `<hint>`, and checkbox markup into `FieldRegionItem` / field text items, including inline key/value bodies
- Add `test_field_region_doclang_roundtrip` and `test_kv_annot_doclang_roundtrip` with materialized `input.json`, `deserialized.json`, and `reserialized.dclg.xml` goldens under `test/data/doc/field_region_kv*/` and `test/data/doc/kv/*/`
- Assert lossless XML round-trip where supported today (`_KV_ANNOT_XML_LOSSLESS`); partial cases remain golden-locked pending list `<group>` and in-region `<heading>` handling

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-06-11 12:30:09 +02:00
github-actions[bot] d3633168ef chore: bump version to 2.81.0 [skip ci] 2026-06-10 14:25:19 +00:00
github-actions[bot] f90b48f8f3 chore: bump version to 2.80.0 [skip ci] 2026-06-09 13:38:14 +00:00
Panos Vagenas 5d740c260a feat(doclang): promote serializer and deserializer (#633)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-06-09 15:23:21 +02:00
Panos Vagenas 28342dcc6c chore(doclang): align with DocLang v0.5 foundation (#630)
* chore(doclang): switch to suppressing namespace and version by default

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): align with DocLang 0.4

- Align DocLang serialization/deserialization with the doclang 0.4 schema (`<label value>`, `<layer value>`, `<href uri>`, `<src uri>`, element-head property order).
- Auto-wrap orphan key/value markup in `<field_item>` / `<field_region>` when the document tree is missing those containers.
- Add optional `doclang-validation` extra and `verify_doclang()` so golden tests also pass the reference `doclang` validator.
- Refresh `.dclg.xml` fixtures and add coverage for hyperlinks, field-region wrapping, and known incompat output under `test/data/doc/incompat/`.
- Pin emitted DocLang version to 0.4 and defer `<layer>`-related expectations until doclang 0.5.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] add basic virtual `<text>` support

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] add virtual `<text>` support

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): extend virtual `<text>` support, unify el. head parsing

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): align code/picture labels to DocLang, add LabelMode

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): emit bare `<value>` for read-only field values

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] align with DocLang v0.5

- Bump doclang validation to 0.5 and enable <layer> emission.
- Align open-tag shapes with the recommended vocabulary: bare <heading>/<field_heading> at level 1, bare <list> for unordered lists.
- Add field_heading deserializer support and remove deferred layer xfails.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): add `<rtl>`/`<corn>`/`<xcel>` coverage

- Emit <rtl> for RTL script text via get_text_direction()
- Emit <corn> for row/column header intersection cells
- Fix OTSL span tokens so <xcel> only appears at interior continuations
- Add rtl/corn/xcel golden tests; defer multi-prov threading test to P2

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] drop legacy vocabulary

- Remove non-spec tokens (`section`, `watermark`, `time_break`, `facets`, `meta`, `uri`) and `GROUP@type` / `TABLE@class` from vocabulary
- Reject `class` on `<table>` at deserialize time; resolve label from `<table>` vs `<index>` only
- Drop legacy `class="data"` / `class="index"` table parsing paths and update deserializer tests accordingly
- Tighten `WRAP_WHEN_NEEDED` comment to include embedded newlines

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-06-08 16:42:35 +02:00
github-actions[bot] be200daff9 chore: bump version to 2.79.0 [skip ci] 2026-06-05 17:48:16 +00:00
github-actions[bot] af013afa90 chore: bump version to 2.78.1 [skip ci] 2026-06-04 08:05:00 +00:00
Cesar Berrospi Ramis d441e6ece3 fix: pin transformers<5.9.0 to solve MPS issue (#625)
Temporary workaround to avoid TypeError with transformers on Apple's MPS framework
until huggingface/transformers#46174 is merged.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-06-04 09:39:42 +02:00
github-actions[bot] d8a18a09e4 chore: bump version to 2.78.0 [skip ci] 2026-05-29 15:53:34 +00:00
Cesar Berrospi Ramis 45e54bb725 chore: upgrade dependencies (#624)
chore: upgrade uv.lock dependencies

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-05-29 14:33:52 +02:00
github-actions[bot] 756580dcd5 chore: bump version to 2.77.1 [skip ci] 2026-05-26 15:22:20 +00:00
github-actions[bot] ebee77b5a6 chore: bump version to 2.77.0 [skip ci] 2026-05-19 13:47:03 +00:00
github-actions[bot] db78e34d8c chore: bump version to 2.76.0 [skip ci] 2026-05-18 15:34:29 +00:00
github-actions[bot] 7cc62cbde7 chore: bump version to 2.75.0 [skip ci] 2026-05-12 14:54:25 +00:00
Peter W. J. Staar 014948b0e8 feat: updated the HTML serialization (#609)
* updated the HTML serialization

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the tests

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-05-12 16:49:08 +02:00
github-actions[bot] 275bd1d217 chore: bump version to 2.74.1 [skip ci] 2026-04-22 14:33:02 +00:00
Panos Vagenas 2087d0f362 fix: refine ImageRef URI handling (#595)
* fix: refine ImageRef URI handling

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* make URI handling configurable via settings

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-04-22 16:13:10 +02:00
github-actions[bot] 0425dc0c03 chore: bump version to 2.74.0 [skip ci] 2026-04-17 06:48:41 +00:00
github-actions[bot] d2e79f4a80 chore: bump version to 2.73.0 [skip ci] 2026-04-09 08:08:14 +00:00
github-actions[bot] b7d35cef4b chore: bump version to 2.72.0 [skip ci] 2026-04-07 12:35:15 +00:00
github-actions[bot] 0388a99570 chore: bump version to 2.71.0 [skip ci] 2026-03-30 15:47:39 +00:00
github-actions[bot] 2808317b3f chore: bump version to 2.70.2 [skip ci] 2026-03-20 15:37:35 +00:00
github-actions[bot] 2a4377b713 chore: bump version to 2.70.1 [skip ci] 2026-03-17 14:06:00 +00:00
github-actions[bot] 1513f7d171 chore: bump version to 2.70.0 [skip ci] 2026-03-13 15:06:05 +00:00
Cesar Berrospi Ramis 9f3c0c6757 chore: upgrade dependencies to address dependabot alerts (#543)
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-03-13 15:18:38 +01:00
github-actions[bot] 24574110d3 chore: bump version to 2.69.0 [skip ci] 2026-03-09 04:31:38 +00:00
Michele Dolfi 4eb0d20d04 feat: Loosen dependency version constraints (#534)
* update deps

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* fix code to support all versions and regen tests

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
2026-03-09 05:27:48 +01:00
github-actions[bot] eb900064c8 chore: bump version to 2.68.0 [skip ci] 2026-03-07 12:19:50 +00:00
Faiq AdzlanandMichele Dolfi 43030488c9 chore: add support for pandas==3.0.0 (#513)
* feat: add support for `pandas==3.0.0`

* DCO Remediation Commit for wanadzhar913 <adzhar.faiq@gmail.com>
I, wanadzhar913 <adzhar.faiq@gmail.com>, hereby add my Signed-off-by to this commit: bb8f81a193

Signed-off-by: wanadzhar913 <adzhar.faiq@gmail.com>

* Revert "feat: add support for `pandas==3.0.0`"

This reverts commit bb8f81a193.

* chore: add support for pandas==3.0.0

* DCO Remediation Commit for wanadzhar913 <adzhar.faiq@gmail.com>

I, wanadzhar913 <adzhar.faiq@gmail.com>, hereby add my Signed-off-by to this commit: 440484229e
I, wanadzhar913 <adzhar.faiq@gmail.com>, hereby add my Signed-off-by to this commit: b47e96c02d

Signed-off-by: wanadzhar913 <adzhar.faiq@gmail.com>

* allow pandas >3 and update lock for it

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: wanadzhar913 <adzhar.faiq@gmail.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com>
2026-03-06 11:25:56 +01:00
github-actions[bot] 1c6ae32b46 chore: bump version to 2.67.1 [skip ci] 2026-03-05 09:11:24 +00:00
github-actions[bot] 7c21840c4f chore: bump version to 2.67.0 [skip ci] 2026-03-04 15:29:11 +00:00
github-actions[bot] 9eca661df7 chore: bump version to 2.66.0 [skip ci] 2026-02-26 10:46:22 +00:00
github-actions[bot] cc4df04ac0 chore: bump version to 2.65.2 [skip ci] 2026-02-23 15:17:00 +00:00