Commit Graph
247 Commits
Author SHA1 Message Date
Michele Dolfi 8c330ab81a fix(deps): switch to tree-sitter-java-orchard and expand typer compatibility (#473)
* switch to tree-sitter-java-orchard and work with newer typer

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* update lock for typer

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* use new lib

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* ignore 3.9 issues

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* lazy import tree-splitter languages

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
2026-01-09 17:18:37 +01:00
Panos Vagenas 51a01a62c9 fix: skip annotation migration if respective meta present (#469)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-01-08 18:02:18 +01:00
SivaandPanos Vagenas 1d2e0c7ebe feat(DocItem): Add comments field for linking annotations to document… (#465)
* feat(DocItem): Add comments field for linking annotations to document items

Implements support for linking comments (from Word/PPT documents) to their
annotated content using the established FloatingItem/RefItem pattern.

Changes:
- Add `comments: List[RefItem]` field to DocItem class
- Update `_update_breadth_first_with_lookup()` to handle comment references on deletion
- Bump CURRENT_VERSION to 1.9.0
- Fix version comparison bug (string vs integer for minor version)
- Add 4 new tests for comments functionality
- Update test data files for new schema

Closes: docling-project/docling#464
Related: docling-project/docling#2834

Signed-off-by: s1v4-d <leelasaisivasubrahmanyamdurga@gmail.com>

* improve comment Pydantic serialization

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add add_comment, update tests

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* introduce fine-granular references with span ranges

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* simplify last test

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: s1v4-d <leelasaisivasubrahmanyamdurga@gmail.com>
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com>
2026-01-08 05:00:57 +01:00
Panos Vagenas 8b082e9b9d feat: enable heading-only chunks for empty-section headings (#461)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-12-18 14:35:15 +01:00
Peter W. J. Staar dda9c885a3 feat: idoctags serialization and deserialization matching the iso proposal (#457)
* Working on ISO standard for IDocTags

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the pre-commit, still tons of work to do

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* updated idoctag token class

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* need to start fixing the bugs and add proper tests for idoctags

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the tests

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* refactored the testing for dcotags and idoctags

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* updating the location tokens in idoctags

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed the DocumentToken

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the idt quot issue

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* reformatted

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* Added IDocTagsSerializationMode

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* working on the deserializer

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the captions for floating items

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* still need to fix some seserialization tests

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* finally all is working and reformatted

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* made the OTSL serialization and deserialization self-contained in IDocTags

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed some unclean code

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* did some fixes and clean up

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* added extra tests, expanded to deserialize to deal with nested lists

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* added the deserialization for formatting

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the formatting and the complex inline groups in nested lists

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* added get_category

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* make IS_SELFCLOSING a set

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed the regex

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the if ... return into if ... elif ... return

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the facets parsing

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the footnotes for Tables and Pictures in IDocTags

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* updated with latest doctags token table

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed private name

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2025-12-17 14:38:10 +01:00
Peter W. J. Staar 9a42a3c860 feat: updating the idoctags serializer (#450)
* feat: updating the idoctags serializer

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* feat: adding IDocTagsToken class

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fix the get_special_tokens

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fix the get_special_tokens (2)

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the doctags serializer: if the no_content flag is true, no double list_item, no content in tables and captions

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the list serializer in the doctags

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* added missing document serializer

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed the prints

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* added tests for content suppression

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* added extra tests for serialization

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* added extra tests

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* ran pre-commit

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* deal with differences between xml serialization in mac and linux

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* revert to old doctags serialization

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* revert to old doctags serialization (2)

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* revert to old doctags serialization (3)

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* revert to old doctags serialization (4)

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixing the tests

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed all the tests, except the no-content with lists

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the no-content in lists with doctags

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed the prints

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2025-12-10 11:39:40 +01:00
Panos Vagenas f8e09ecf69 fix: switch meta migration logging to info (#452)
* chore: switch meta migration logging to info

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* suppress deprecation warnings within migration, expand to TableItem, add tests

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-12-08 11:36:48 +01:00
Christoph AuerandPanos Vagenas a5e2d2dcc0 feat: Visualize reading-order inside pictures (#446)
* fix: robustify page filtering

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* feat: Visualize reading-order inside pictures

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Add EMPTY_VALUE to DEFAULT_EXPORT_LABELS

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com>
2025-11-28 16:29:32 +01:00
Peter W. J. Staar 76e14fdb97 feat: add latex serializer (#445)
* feat: add latex serializer

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed spurious copyright

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the multiline

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2025-11-28 14:43:37 +01:00
Michele Dolfi edda70b2cf chore: use sample_doc fixture (#442)
use sample_doc fixture

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
2025-11-27 15:59:41 +01:00
Peter W. J. StaarandPanos Vagenas 92d60b0948 feat: added the Azure Document Intelligence (#395)
* added the Azure Document Intelligence

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the formatting and MyPy, working on the tests now

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the tests

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* added test document

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* replace all  with pydantic dataclasses

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* minor azure serializer refactoring

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com>
2025-11-27 11:32:15 +01:00
Peter W. J. Staar 928aaed9ab chore: remove unnecesary copyright headers (#440)
* chore: remove unnecesary copyright headers

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* reformatted, still need to fix a test ...

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2025-11-27 11:17:23 +01:00
Panos Vagenas 8bdeaa7238 fix: robustify page filtering (#437)
* fix: robustify page filtering

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* cover validate_rules

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-11-26 13:53:15 +01:00
Cesar Berrospi Ramis 8feb09f1cf fix: markdown serialization of hyperlink with code (#434)
* chore: leverage fixtures in tests and add type annotations

Use fixtures for sample DoclingDocument in tests.
Add missing function type annotation when the body contains annotations.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* fix(markdown): serialize hyperlink as code always with single backticks

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2025-11-21 14:05:29 +01:00
Panos Vagenas c9e5fb4a1c feat(experimental): add new DocTags serializer (#412)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-11-20 13:06:38 +01:00
c80b58369c feat: convert regions into TableData (#430)
* Introduction of regions.py - convert regions into TableData

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Cleaned up bbox helper functions, reusing more of the existing code

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Small fixes

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* refactored _bbox_intersection from regions.py into a method of BoundingBox.get_intersection_bbox

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* move region-based construction into `TableData` class

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com>
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com>
2025-11-19 13:47:20 +01:00
Panos Vagenas 7bd274b84a chore: fix invalid test filenames (#431)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-11-18 11:31:29 +01:00
Cesar Berrospi Ramis 56b3c42c61 fix: DoclingDocument model validator should deal with any raw input (#419)
* fix: DoclingDocument model validator should deal with any raw input

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* docs: mention 3.14 as valid Python version for development

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2025-11-13 16:37:11 +01:00
Panos Vagenas 3d13b02756 fix: improve meta migration and warning handling (#417)
* fix: improve meta migration and warning handling

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* remove commented-out code

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-11-12 16:25:14 +01:00
3097645198 feat: add code chunking functionality (#398)
* initial code chunking for docling-core

* DCO Remediation Commit for Bridget McGinn <bridget.mcginn@ibm.com>

I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 334811a4edf0c9a8511a0e3cd31acda2df335d62

Signed-off-by: Bridget McGinn <bridget.mcginn@ibm.com>

* include language detections, add code chunking into hierarchical chunker

* add serializer, internal marking of chunkers, typing

* Update pyproject.toml

Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
Signed-off-by: Bridget <bridget.mcginn@ibm.com>

* Update docling_core/transforms/chunker/hierarchical_chunker.py

Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
Signed-off-by: Bridget <bridget.mcginn@ibm.com>

* run all pre-commit less pytest

* update test files for code ID

* DCO Remediation Commit for Bridget McGinn <bridget.mcginn@ibm.com>
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 46bb88ae3a00ebbdd33336e51b51e030c748425b
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 10e9ed8a0f28bdd4ae6156a84f19435319f4a5ce
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: d9827c749b72ecc902c66a24eb7fcb4db1b615fa
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 814dc6161134752b08f699eeef58535248893768

Signed-off-by: Bridget McGinn <bridget.mcginn@ibm.com>

* update uv.lock

Signed-off-by: Bridget McGinn <bridget.mcginn@ibm.com>

* revert to stricter treesitter versioning due to compatibility

Signed-off-by: Bridget McGinn <bridget.mcginn@ibm.com>

* DCO Remediation Commit for Bridget McGinn <bridget.mcginn@ibm.com>
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: a4a21e90d63da3aa46fd6f0681b065d6bf85305b
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 0266c6387066cd8e4d45ea02fcb417ade5eedd18
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 336dd6a2078272dd30ade58b4fc6392e7a2edf86
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 68890e9f48e6f310e27eb20eeb2d98d45c90bd36
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 3c65eef59e817e9bd8cd3c55e98e843e326dd7a2

Signed-off-by: Bridget McGinn <bridget.mcginn@ibm.com>

* remove language detection (to be run by client, i.e. docling)

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* align new dependency specs

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* address backticks, ABC, and supported languages feedback

Signed-off-by: Bridget McGinn <bridget.mcginn@ibm.com>

* remove Language class and reuse CodeLanguageLabel

Signed-off-by: Bridget McGinn <bridget.mcginn@ibm.com>

* DCO Remediation Commit for Bridget McGinn <bridget.mcginn@ibm.com>

I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 63c773907f
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 431d357053
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: f3175c2717
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 1a01de8557
I, Bridget McGinn <bridget.mcginn@ibm.com>, hereby add my Signed-off-by to this commit: 025aea31ab

Signed-off-by: Bridget McGinn <bridget.mcginn@ibm.com>

* refactoring and improvements

- encapsulated code chunking specifics to separate package
- clearly separated public vs internal API via module and method naming conventions
- simplified or removed parts not stricly necessary for public API (e.g. lang support querying, noopstrategy)
- split chunk data model to separate modules to prevent circular dependencies
- renamed DefaultCodeChunkingStrategy to Standard... for clarity as it need not be the default strategy
- fixed some issues (e.g. gen flag in test)

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Bridget McGinn <bridget.mcginn@ibm.com>
Signed-off-by: Bridget <bridget.mcginn@ibm.com>
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com>
2025-11-12 10:28:54 +01:00
Panos Vagenas 2ee3cacdd6 feat: add metadata model hierarchy (#408)
* feat: add metadata model hierarchy

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add deprecation, add first migration

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* extend annotations migration

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* update with feedback

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* expose main prediction

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* ideas on enforcing separation between standard and custom fields

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add custom field setter method

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* update Markdown serialization

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* revert description, add include_non_meta, showcase custom serializer for summaries

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* simplify customization

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* fix reference exclusion

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* eliminate serialization dupliation between meta & (legacy) annotations

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* remove old file

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* fix item used in get_parts for meta ser

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* serialize GroupItem meta prior to content, DocItem meta after content

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* restore ser order for all nodeitems

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* move meta serialization into DocSerializer.serialize() to maintain seamless chunking integration

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add allow- & block-lists for meta names, add std field name enum

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add HTML serializer, document meta field names, rename SMILES field

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* bump DoclingDocument version

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* make TabularChartMetaField.title optional, expose new classes through __init__.py, add MetaUtils

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add DocTags serialization, revert smiles to smi to prevent confusion with plural

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-10-30 11:15:14 +01:00
Michele Dolfi 47b70d3c24 feat: python 3.14 compatibility (#403)
update deps for python 3.14 compatibility

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
2025-10-16 15:58:29 +02:00
Erik abfa81f065 feat: rendering of checkboxes in markdown (#381)
* added support for rendering checkboxes

* DCO Remediation Commit for Erik <erik@apption.com>

Signed-off-by: Erik <erik@apption.com>

* DCO Remediation Commit for Erik <erik@apption.com>

I, Erik <erik@apption.com>, hereby add my Signed-off-by to this commit: 7af470451b

Signed-off-by: Erik <erik@apption.com>

* added code suggestions for better checkboxes

* updated test data

* fixed formatting

* DCO Remediation Commit for Erik <erik@apption.com>

I, Erik <erik@apption.com>, hereby add my Signed-off-by to this commit: fc7a57e1f7
I, Erik <erik@apption.com>, hereby add my Signed-off-by to this commit: 7ea737ecd4
I, Erik <erik@apption.com>, hereby add my Signed-off-by to this commit: 5c431b52c8

Signed-off-by: Erik <erik@apption.com>

---------

Signed-off-by: Erik <erik@apption.com>
2025-10-01 11:16:31 +02:00
Panos Vagenas 3e8d628eee fix: switch to safe YAML loader (#396)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-10-01 09:43:42 +02:00
Panos Vagenas 9df72083c0 fix(markdown): fix single-row table serialization (#385)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-09-10 16:11:34 +02:00
Maxim LysakandMaksym Lysak b13267f18b feat: Introduction of fillable TableCell (#384)
* New properties for TableCell, corner_header and fillable

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* Updated docs

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

* bumped docling document version to 1.7.0

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>

---------

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com>
2025-09-09 15:48:42 +02:00
Viroshaan Uthayamoorthy (KLP) 0512f4473d fix: add doc param to all export_to_dataframe() calls (#380)
* fix: add 'doc' to all export_to_dataframe calls

* DCO Remediation Commit for Viroshaan <viroshaan.uthayamoorthy@klp.no>

I, Viroshaan <viroshaan.uthayamoorthy@klp.no>, hereby add my Signed-off-by to this commit: 10317f8db7

Signed-off-by: Viroshaan <viroshaan.uthayamoorthy@klp.no>

---------

Signed-off-by: Viroshaan <viroshaan.uthayamoorthy@klp.no>
2025-09-09 14:42:57 +02:00
Panos Vagenas 2dc57c1769 fix: fix handling of generic groups in rich table cells (#383)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-09-09 09:55:55 +02:00
Panos Vagenas b60ac192ef feat: add support for heading with inline in HTML & DocTags (#379)
* test: refactor serialization tests

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* further refactor serialization tests

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* rename inline test file for markdown

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* simplify Markdown handling of inline in heading / list item

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* fix input test data & make it more interesting

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add HTML & DocTags support

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* generalize and refactor HTML ser case

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* generalize MD serializer too

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-09-03 16:56:58 +02:00
Panos Vagenas 9dc526de53 feat: add page filtering to DoclingDocument (#378)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-09-02 15:56:35 +02:00
Panos Vagenas b2095b31de fix: use in-OTSL DocTags for rich table cells (#375)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-08-28 09:46:46 +02:00
Panos Vagenas 1d04154378 feat: add rich table cells (#368)
* feat: add rich table cells

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* propagate cell text resolution, cover row deletions

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add doctags, fix referential integrity, expand tests, reenable mypy

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* bump DoclingDocument version

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* simplify / remove serialize_cell

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* update rich table cell refs in doc indexing

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* update notebook

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* expose new classes in `docling_core.types.doc`

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-08-27 10:07:47 +02:00
Panos Vagenas bbe6243833 chore: add DoclingDocument validation rules (#349)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-08-19 14:14:13 +02:00
Panos Vagenas 99eabb3b01 feat: add document concatenation (#365)
* chore: add document concatenation

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* finalize exposed API, add tests

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-08-19 12:30:33 +02:00
Cesar Berrospi Ramisand15mbp 23badf2a1d fix: add forward slashes to singleton tags (#369)
* Add forward slash to singleton tags

See issue #354 on docling-project/docling-core

Signed-off-by: 15mbp <100952997+15mbp@users.noreply.github.com>

* test(HTML): fix ground truth data files

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

---------

Signed-off-by: 15mbp <100952997+15mbp@users.noreply.github.com>
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
Co-authored-by: 15mbp <100952997+15mbp@users.noreply.github.com>
2025-08-19 10:05:34 +02:00
Panos Vagenas 807d972f05 fix(HTML): fix nested list serialization edge cases (#367)
* fix(HTML): (initial state without fix) fix nested list serialization edge cases

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* actual fix

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-08-13 15:06:54 +02:00
Michele DolfiandCesar Berrospi Ramis 5afa99e028 fix: referenced artifacts relative to the document location (#361)
* fix relative references

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* Apply suggestions from code review

Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com>

---------

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com>
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
2025-07-30 13:00:03 +02:00
Christoph Auer f9b3b499de fix: LayoutVisualizer should traverse pictures (#358)
LayoutVisualizer should traverse pictures

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
2025-07-23 09:56:00 +02:00
Cesar Berrospi Ramis 5a7883cee5 fix: HTML serialization of nested lists (#359)
* fix(serializer): the HTML serializer should append nested lists to list items

For a valid HTML document, the serializer should ensure that sub-list items are indented
under their respective main list items.

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

* test: address several warning messages

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

---------

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
2025-07-23 09:31:02 +02:00
Panos Vagenas 3a0b7471ba feat: add page chunking (#337)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-07-15 11:03:11 +02:00
Clément Doumouro ebd9147ced fix: only save applicable page images (#226)
fix(document): save document images page by page

Signed-off-by: Clément Doumouro <clement.doumouro@gmail.com>
2025-07-15 09:52:09 +02:00
Ryan Fernandes cb59fd39bc feat: Additional DoclingDocument methods for use in MCP document manipulation (#344)
* insert_* methods and range manipulation methods

* DCO Remediation Commit for Ryan Fernandes <ryan@fernandes.us>

I, Ryan Fernandes <ryan@fernandes.us>, hereby add my Signed-off-by to this commit: cc0c4b283f

Signed-off-by: Ryan Fernandes <ryan@fernandes.us>

* insert_node_items method with recursive _append_item_copies helper for cross-document manipulation

Signed-off-by: Ryan Fernandes <ryan@fernandes.us>

* add_node_items complement to inser_node_items, refactored extract_items_range to be more robust, added insert/add_document

Signed-off-by: Ryan Fernandes <ryan@fernandes.us>

* Extended table data editing functionality

Signed-off-by: Ryan Fernandes <ryan@fernandes.us>

* Added new document manipulation tests

* Extra gt files

* Refactor for reusable code - table methods

* Private methods for more concise insert_* methods

Signed-off-by: Ryan Fernandes <ryan@fernandes.us>

* Docstring changes

Signed-off-by: Ryan Fernandes <ryan@fernandes.us>

* ValueError for failed insertions

Signed-off-by: Ryan Fernandes <ryan@fernandes.us>

* DCO Remediation Commit for Ryan Fernandes <ryan@fernandes.us>

I, Ryan Fernandes <ryan@fernandes.us>, hereby add my Signed-off-by to this commit: 01e8b9cb03
I, Ryan Fernandes <ryan@fernandes.us>, hereby add my Signed-off-by to this commit: 8c016d9d32
I, Ryan Fernandes <ryan@fernandes.us>, hereby add my Signed-off-by to this commit: 308900270a

Signed-off-by: Ryan Fernandes <ryan@fernandes.us>

---------

Signed-off-by: Ryan Fernandes <ryan@fernandes.us>
2025-07-09 10:53:04 +02:00
Panos Vagenas baa2cc3714 feat: enable precision control in float serialization (#352)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-07-08 17:00:29 +02:00
Clément Doumouro 9fa0c9f8be fix: BoundingRectangle angle computation when in CoordOrigin.TOPLEFT (#347)
* fix(page): angle computation when in `CoordOrigin.TOPLEFT`

* DCO Remediation Commit for Clément Doumouro <clement.doumouro@gmail.com>

I, Clément Doumouro <clement.doumouro@gmail.com>, hereby add my Signed-off-by to this commit: 742ccf3d9c

Signed-off-by: Clément Doumouro <clement.doumouro@gmail.com>

---------

Signed-off-by: Clément Doumouro <clement.doumouro@gmail.com>
2025-07-02 17:53:03 +02:00
Panos Vagenas 14a4fdee87 feat: remodel lists, add MD & HTML ser. params, enable unset marker (#339)
* feat: remodel lists

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* prepare test document separate markers

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add MD/HTML serializer options, expand test data

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* create list groups where not in place

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add auto-increment logic

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* restore UnorderedList as deprecated alias (for backwards compatibility)

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* enable unset marker case

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* make ordered markers able to be "unset" (empty) too

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* rename default marker mode to auto

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-06-27 14:52:25 +02:00
Panos Vagenas c9374e8fe4 fix(markdown): add heading formatting, fix code & formula formatting (#336)
* feat(markdown): add formatted headings (show behavior before change)

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add change and updated test data

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* fix when formatting & escaping are applied (depending on type)

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-06-20 14:10:48 +02:00
Panos Vagenas 78b796221d feat(viz): add reading order branch numbering, fix cross-page lists (#334)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-06-18 14:18:48 +02:00
Panos Vagenas 574779f4c7 chore: add ref number normalization (#322)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-06-17 12:31:39 +02:00
Panos Vagenas 5c99722b38 fix(HybridChunker): improve long heading handling (#333)
* fix(HybridChunker): improve long heading handling

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* fix warning wording

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-06-17 12:21:51 +02:00
Riccardo Maggioni 0fd3c1cbe6 feat: add parameter to choose of which pages export the doctags (#290)
* feat: add parameter to choose of which pages export the doctags

Signed-off-by: rickymaggio02 <riccardo.maggioni@googlemail.com>

* feat: added test and better description of the method

Signed-off-by: rickymaggio02 <riccardo.maggioni@googlemail.com>

---------

Signed-off-by: rickymaggio02 <riccardo.maggioni@googlemail.com>
2025-06-16 15:30:09 +02:00