* fix: Add missing dependency on `requests`
Add an explicit dependency on `requests` since it is used in
`docling_core.utils.file` since 752cbc3. Without it, the CLI tools fail
to run on a fresh install (e.g. via `uv pip install docling-core`):
```pytb
$ docling-view
Traceback (most recent call last):
File "/tmp/.venv/bin/docling-view", line 4, in <module>
from docling_core.cli.view import app
File "/tmp/.venv/lib/python3.14/site-packages/docling_core/cli/view.py", line 13, in <module>
from docling_core.utils.file import resolve_source_to_path
File "/tmp/.venv/lib/python3.14/site-packages/docling_core/utils/file.py", line 11, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
```
Signed-off-by: Michał Górny <mgorny@quansight.com>
* Apply batched suggestions from code review
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Sync the lockfile
Signed-off-by: Michał Górny <mgorny@quansight.com>
---------
Signed-off-by: Michał Górny <mgorny@quansight.com>
Signed-off-by: Michał Górny <mgorny@gentoo.org>
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
* fix(serializer): derive markdown table header rows from column_header flags
MarkdownTableSerializer hardcoded grid row 0 as the only header row, while
TableItem._export_to_dataframe_with_options, the HTML serializer and the OTSL
export all derive the header rows from TableCell.column_header. A table with a
stacked header therefore serialized to markdown with its lower header rows
demoted to data rows, and HybridChunker repeated only the first of them at the
top of each chunk of an oversized table.
Take the header row count from the flags and flatten those rows into the single
header row GFM allows, dropping the repeated text a cell spanning several header
rows produces. When no cell in the table carries column_header the previous
behavior is kept, since backends that never set the flag cannot be told apart
from genuinely headerless tables.
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
* test(serializer): cover tables whose header flags start below row 0
The empty-header-block branch of _flatten_header_rows was the one line of the
patch left uncovered. Pin the behavior it produces: with no leading run of
flagged rows there is no header to promote, so the header is empty and every
row stays in the body rather than one being consumed.
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
* fix(serializer): join stacked header cells with a documented separator
Review feedback: a blank space loses the boundary between header levels, so
"native backend" over "TTS" is indistinguishable from a single cell reading
"native backend TTS". Join with " - " instead, kept as a module constant, and
document the header-resolution contract on MarkdownTableSerializer.
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
* fix(serializer): only count header rows a column header starts on
TableData.grid repeats a row-spanning cell into every row it covers, so a
header cell with row_span > 1 leaves a column_header-flagged cell in the data
rows beneath it. Counting every row that holds a flagged cell therefore pulled
those data rows into the header block, where they were flattened into the
column names and dropped from the body entirely.
Require that a header cell starts on the row, matching the same fix applied to
the DataFrame export in #756.
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
* perf(serializer): avoid iterating grid twice; style and tests improvements
In '_count_header_rows' of the Markdown serializer, avoid iterating
the entire grid twice in all scenarios.
Strengthen docstrings in markdown serializer.
Add test for single header row with flags.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
---------
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
OpenCV's imencode defaults to compression level 1 while PIL's PNG encoder
defaults to 6, so PNGs from the cv2 path were 30-60% larger for identical
pixels. Regenerated the affected test ground truth.
Fixes#758
Signed-off-by: Yiorgis Gozadinos <ggozadinos@gmail.com>
Add MIME types (both standard and legacy) for Apple iWork (productivity apps):
Apple Pages, Apple Numbers, Apple Keynote
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
test: rename 'test' folder to 'tests'
To align with other Docling repositories, use 'tests' for test modules.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* style: modernize type hints with PEP 604
Enforce Ruff UP007 and UP045 to align with PEP 604 (union types as X | Y).
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore: upgrade 'uv.lock' dependencies
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
---------
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* fix: relaxing transformers versions
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* relaxed transformers across the board
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* fix(tokens): normalize inverted bbox in get_location instead of asserting
Near-degenerate elements (e.g. a thin rule whose layout-model regression
produced a slightly inverted bbox) could arrive with bbox[0] > bbox[2] or
bbox[1] > bbox[3]. The redundant asserts in DocumentToken.get_location
crashed export_to_doctags(add_location=True) on such boxes, even though the
method already normalizes ordering via min()/max() downstream. This bites
long documents in particular, where the odds of one degenerate element grow
with page count.
Sort the coordinates before normalization to mirror the existing min()/max()
behaviour, and drop the asserts. Adds regression tests for normal, partially
inverted (from the issue repro), and fully inverted bboxes.
Fixes#620
Signed-off-by: max <87073104+maxtaran2010@users.noreply.github.com>
* docs: fix get_location docstring grammar (give -> given)
Addresses review feedback on #676.
Signed-off-by: max <87073104+maxtaran2010@users.noreply.github.com>
* refactor(tokens): drop redundant min()/max() after bbox sort in get_location
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* docs(tokens): move get_location normalization note into docstring
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
---------
Signed-off-by: max <87073104+maxtaran2010@users.noreply.github.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* fix: updated the doclang serializer for groups
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* fixed the pre-commit
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
* fix(markdown): serialize newlines as GFM hard line breaks
Single \n in body text and list items is now serialized as ' \n'
(two trailing spaces), which GFM renders as a hard line break.
Double \n\n is preserved as a paragraph break.
Newlines in headings are collapsed to a space, since headings cannot
span multiple lines.
PlainTextDocSerializer is unaffected via overridable instance methods.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore(deps): upgrade uv.lock dependencies
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
---------
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* contextualize function() skip excluded fields
This change proposes that we use model_dump to skip/avoid serializing
fields that are in excluded_embed. This should reduce unnecesary
computation and is also more precisely what I believe we are intending
to do here in that we do not need to unnecessarily filter the results
below in the for-loop.
Signed-off-by: Jordan White <jordan.page.white@gmail.com>
* style(chunker): fix style issue C416
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* fix(chunker): resolve excluded_embed to field names
model_dump(exclude=...) requires Python attribute names, not
aliases. excluded_embed stores alias keys, which happen to
match today but would silently fail to exclude fields in
subclasses where they differ, leaking excluded fields into the
embedding context.
Adds BaseMeta._excluded_embed_field_names() to resolve each
entry through the alias-to-field-name map. contextualize() now
calls this helper instead of using excluded_embed directly.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* fix(chunker): fix pydantic deprecation
Deprecated in pydantic v2.11, removed in v3: model_fields can only be accessed
from class object, not the instance.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* test(chunker): add test to verify contextualize() excludes the right fields
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore: update uv.lock dependencies
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
---------
Signed-off-by: Jordan White <jordan.page.white@gmail.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* feat(compat): add DoclingDocument version-downgrade projector system
Adds docling_core/compat.py with a register_projector decorator,
project_to() API, and five retroactive projectors (1.10→1.9→1.8→1.7→
1.6→1.5). Adds scripts/check_compat_projectors.py as a pre-commit hook
that detects unannounced structural breaking changes and enforces that
every schema minor bump ships a projector, a JSON Schema snapshot, and a
test class. 36 new tests in test/test_compat.py; CONTRIBUTING.md
documents the change-classification table and 7-step checklist.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore: upgrade uv.lock dependencies
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
---------
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
A page break falling inside a ListGroup was emitted as an ordinary list
part and joined with the list separator "\n", while one falling between
top-level items was joined with "\n\n". The placeholder is substituted
into the already joined text, so the spacing was fixed by whichever
scope emitted the break and could not be renegotiated downstream.
Exporting a document globally and page by page therefore disagreed by
two bytes whenever a list straddled a page boundary, although the
content matched.
MarkdownListSerializer now separates a page break part from its
neighbours with "\n\n", so the marker forms a block in either scope. To
let it recognise such a part, DocSerializer.serialize() returns the
_PageBreakSerResult as such instead of rebuilding a plain result and
dropping the type.
Output is unchanged for callers leaving page_break_placeholder unset,
as no page break parts are produced at all in that case.
Signed-off-by: OmarFaig <omerfaiq7@gmail.com>
* feat: add initial dlgrep implementation
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* refactor: migrate CLI to Typer
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* docs: expand README
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* docs: improve README examples
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* fix: streamline dlgrep output
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* fix: make text context output grep-compatible
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Improve XPath and inline context output
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* dlgrep support extensions
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Use git sources for doclang and docling-core
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* feat: emit native JSON records
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Remove search class filter
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Make section output addressable
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* Improve README examples
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* feat: build and publish dlgrep from this repository
dlgrep becomes a uv workspace member under packages/dlgrep, published as
its own wheel and released in lockstep with docling-core: same version,
exact `docling-core==` pin, both wheels built from the same commit. This
removes the git-branch dependency dlgrep needed to track unreleased
docling-core changes.
- workspace wiring; ruff, mypy and pytest config inherited from the root,
dlgrep in the dev group so `uv sync` covers it
- release.sh writes both versions and the pin from one TARGET_VERSION
- build-packages.sh builds into per-package dist dirs; pypi.yml publishes
docling-core first, then dlgrep
- check_lockstep.py guards against version drift, which uv cannot catch on
its own: the workspace source replaces the pin with a local path, so a
wrong pin resolves and locks cleanly
- checks.yml and pypi.yml install both wheels together, which makes a
mismatched pin fail before anything is published
- declare click and lxml, previously only transitive
- read __version__ from package metadata instead of a hardcoded literal
- flag dlgrep as experimental in its help output, since the shared version
number would otherwise imply a maturity it does not have
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* ci: simplify lockstep release tooling
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* chore(dlgrep): drop handoff notes, keep only the CLI plan
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* refactor(doclang): isolate source mapping recorder
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* build(dlgrep): move dependencies to core extra
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* feat(dlq): rename CLI and add list command
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* feat(dclq): rename tool and consolidate plan
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
* fix(dclq): bound archive document reads
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
DocLangDocSerializer raised "level must be in [1, 6]" for any
SectionHeaderItem with level 6. That is reachable in normal use, not a
synthetic edge case: SectionHeaderItem.level accepts up to 100, and
docling's heading-hierarchy inference emits level-6 headings by default,
so a PDF with nested procedure lists crashes the serializer.
The +1 shift (level 1 is reserved for TitleItem) pushed level 6 to 7,
past the heading token's allowed range, with nothing clamping it back.
html.py already clamps the identical item.level + 1 computation with
min(..., 6); this applies the same treatment, which is the smallest fix
and keeps the two serializers consistent.
Fixes#685
Signed-off-by: Max Hsu <maxmilian@gmail.com>