Commit Graph
596 Commits
Author SHA1 Message Date
Christoph Auer 7b47b5a506 fix: externalize page images in REFERENCED mode for JSON/YAML export (#634)
On JSON export, control inclusion of page images for ImageRefMode.REFERENCED

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
2026-06-10 10:32:40 +02:00
github-actions[bot] f90b48f8f3 chore: bump version to 2.80.0 [skip ci] v2.80.0 2026-06-09 13:38:14 +00:00
Panos Vagenas 5d740c260a feat(doclang): promote serializer and deserializer (#633)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-06-09 15:23:21 +02:00
Panos Vagenas ba110e8671 chore(doclang): add v0.5 container-level threading (#631)
* test(doclang): add simple multi-page roundtrip test

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] add container-level threading

- Emit threaded fragments with page breaks for multi-prov text, list groups, and tables
- Merge continuation fragments on deserialize using a (thread_id, host) registry
- Allocate document-scoped auto-increment thread IDs to satisfy spec uniqueness rules
- Add serialization and round-trip fixtures for paragraph, list, and table threading
- Suppress duplicate document-level page breaks when list/table serializers embed their own

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): align with v0.5 threading and heads

- Emit fragment-scoped OTSL for multi-prov tables using continuation tokens (`lcel`/`ucel`/`xcel`) instead of duplicating full grids, and merge fragments on deserialize with correct offsets and `TableData` dimensions.
- Fix element-head order to match the XSD (`label` → `thread` → `href` → `layer` → `location` → `caption` → `custom`) and add coverage for href/layer ordering.
- Drop legacy `unknown` label handling in favor of the v0.5 `undefined` default, adjusting both serializer and deserializer behavior plus goldens.
- Update cross-page/column table factories and fixtures to use proper 2×2 layouts and the new slice-based threading semantics.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-06-09 11:17:14 +02:00
Panos Vagenas 28342dcc6c chore(doclang): align with DocLang v0.5 foundation (#630)
* chore(doclang): switch to suppressing namespace and version by default

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): align with DocLang 0.4

- Align DocLang serialization/deserialization with the doclang 0.4 schema (`<label value>`, `<layer value>`, `<href uri>`, `<src uri>`, element-head property order).
- Auto-wrap orphan key/value markup in `<field_item>` / `<field_region>` when the document tree is missing those containers.
- Add optional `doclang-validation` extra and `verify_doclang()` so golden tests also pass the reference `doclang` validator.
- Refresh `.dclg.xml` fixtures and add coverage for hyperlinks, field-region wrapping, and known incompat output under `test/data/doc/incompat/`.
- Pin emitted DocLang version to 0.4 and defer `<layer>`-related expectations until doclang 0.5.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] add basic virtual `<text>` support

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] add virtual `<text>` support

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): extend virtual `<text>` support, unify el. head parsing

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): align code/picture labels to DocLang, add LabelMode

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): emit bare `<value>` for read-only field values

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] align with DocLang v0.5

- Bump doclang validation to 0.5 and enable <layer> emission.
- Align open-tag shapes with the recommended vocabulary: bare <heading>/<field_heading> at level 1, bare <list> for unordered lists.
- Add field_heading deserializer support and remove deferred layer xfails.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): add `<rtl>`/`<corn>`/`<xcel>` coverage

- Emit <rtl> for RTL script text via get_text_direction()
- Emit <corn> for row/column header intersection cells
- Fix OTSL span tokens so <xcel> only appears at interior continuations
- Add rtl/corn/xcel golden tests; defer multi-prov threading test to P2

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] drop legacy vocabulary

- Remove non-spec tokens (`section`, `watermark`, `time_break`, `facets`, `meta`, `uri`) and `GROUP@type` / `TABLE@class` from vocabulary
- Reject `class` on `<table>` at deserialize time; resolve label from `<table>` vs `<index>` only
- Drop legacy `class="data"` / `class="index"` table parsing paths and update deserializer tests accordingly
- Tighten `WRAP_WHEN_NEEDED` comment to include embedded newlines

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-06-08 16:42:35 +02:00
github-actions[bot] be200daff9 chore: bump version to 2.79.0 [skip ci] v2.79.0 2026-06-05 17:48:16 +00:00
129507ebd2 feat(meta): add keywords and topics metadata fields (#617)
* feat: add keywords and topics meta fields

Introduce two simple, additive meta fields:
- KeywordsMetaField — unique list of keyword strings
- TopicsMetaField — unique list of topic / subject strings

Both attach to BaseMeta (node-level) and to the new DocumentMeta
(document-level), which is wired into DoclingDocument.meta. HTML,
Markdown, and Doclang serializers render the new fields as comma-
separated values. Schema bump is intentionally avoided since the
fields are optional and backward-compatible.

Signed-off-by: Ana Daniele <ana.daniele@ibm.com>

* test: cover keywords and topics meta fields

Add round-trip, dedup, has_content, HTML escape, and Markdown
rendering tests for KeywordsMetaField, TopicsMetaField, and
DocumentMeta. Document-level meta absent-by-default is also covered.

Signed-off-by: Ana Daniele <ana.daniele@ibm.com>

* style: satisfy ruff UP038 and import order

Convert tuple-form isinstance checks to PEP 604 unions and
reorder a misplaced import flagged by ruff.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Ana Daniele <ana.daniele@ibm.com>

* fix: align with HTML attribute format from #613

Post-rebase onto main, the HTML metadata serializer now emits
`data-meta-name="<field>"` instead of valueless `data-meta-<field>`
and applies a single trailing html.escape pass. Drop the per-type
escape from the Keywords/Topics branch (was producing double
escaping after rebase) and update the two test assertions to the
new attribute shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Ana Daniele <ana.daniele@ibm.com>

* revert: remove DocLang keywords/topics serialization

Per discussion, DocLang is a standard whose extension is out of
scope for this PR. Drop the KeywordsMetaField/TopicsMetaField
branches from DoclangMetaSerializer and their imports; HTML and
Markdown serialization remain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Ana Daniele <ana.daniele@ibm.com>

* refactor: address review feedback on keywords/topics meta fields

- Replace MetaFieldName inline comments with Attributes docstring (TableFormerMode pattern)
- Swap list[str]+manual dedup → UniqueList[str] in KeywordsMetaField and TopicsMetaField
- Add Field(description, examples) to all BaseMeta fields distinguishing entities/keywords/topics
- Remove DocumentMeta class and DoclingDocument.meta field; document-level meta goes via body
- Consolidate keywords/topics tests into test_semantic_base_meta_fields_roundtrip_and_html_rendering
- Regenerate docs/DoclingDocument.json

Signed-off-by: Ana Daniele <ana.daniele@ibm.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(meta): reuse validator between keywords and topics

Revert the deduplication as part of a validator.
Share the validator between keywords and topics meta classes.
Adopt the standard Google style for enum classes.
Adapt the regression tests to the changes above.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Ana Daniele <ana.daniele@ibm.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Ana Daniele <ana.daniele@ibm.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-05 13:06:37 +02:00
github-actions[bot] af013afa90 chore: bump version to 2.78.1 [skip ci] v2.78.1 2026-06-04 08:05:00 +00:00
Cesar Berrospi Ramis d441e6ece3 fix: pin transformers<5.9.0 to solve MPS issue (#625)
Temporary workaround to avoid TypeError with transformers on Apple's MPS framework
until huggingface/transformers#46174 is merged.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-06-04 09:39:42 +02:00
Pragnya Khandelwal 701c0ce5d1 fix: reject unsupported URL schemes (#604)
* Reject unsupported URL schemes

Signed-off-by: Pragnya <prag1704@gmail.com>

* Remove PR template

Signed-off-by: Pragnya <prag1704@gmail.com>

* fix: satisfy lint checks for unsupported url scheme

Signed-off-by: Pragnya <prag1704@gmail.com>

---------

Signed-off-by: Pragnya <prag1704@gmail.com>
2026-06-03 12:40:16 +02:00
github-actions[bot] d8a18a09e4 chore: bump version to 2.78.0 [skip ci] v2.78.0 2026-05-29 15:53:34 +00:00
Peter W. J. Staar c1c9ea80a5 feat: support vertical orientation in TableData row/column bounding boxes (#622)
* fix: vertical-rows-and-columns

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* ran pre-commit

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* added Orientation to TableData to deal with vertical tables

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* updated docs

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* moved to ROT_0, etc

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* updated the convention of the rotation

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-05-29 17:16:08 +02:00
Cesar Berrospi Ramis 45e54bb725 chore: upgrade dependencies (#624)
chore: upgrade uv.lock dependencies

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-05-29 14:33:52 +02:00
b20b5677c0 fix(html-serializer): emit XHTML-compatible metadata attributes (#613)
* fix(html-serializer): emit XHTML-compatible metadata attributes

HTMLMetaSerializer previously rendered each metadata field as a valueless
HTML5 attribute (e.g. `<div data-meta-classification>`). Lenient HTML
parsers accept this, but the output is not valid XHTML/XML, so downstream
consumers that parse exported HTML as XML — including TEDS table
evaluation — fail with `xml.etree.ElementTree.ParseError: not well-formed
(invalid token)`.

Switch the attribute shape to a standard name=value pair
(`data-meta-name="<field>"`), HTML-escape both the field name and the
text body (`TabularChartMetaField` is left unescaped because its value is
an intentionally embedded HTML table), and update the embedded CSS
selectors and serialization ground-truth fixtures accordingly. Adds a
regression test that round-trips the output of `export_to_html` through
`xml.etree.ElementTree.fromstring`.

Fixes #598.

Signed-off-by: Brian Wang <20699847+BrianWang1990@users.noreply.github.com>

* fix(html-serializer): rebase changes to 'main' and add tests

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* test(html-serializer): remove redundant CSS rule

Remove duplicate CSS rules in 'html_styles.py' and ground-truth files.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Brian Wang <20699847+BrianWang1990@users.noreply.github.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Brian Wang <20699847+BrianWang1990@users.noreply.github.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-05-29 10:01:30 +02:00
Manohar Mallipudi 507df717c1 fix(chunking): handle empty TripletTableSerializer output without consuming children (#596)
When a `TableItem` produced empty serialized text, `TripletTableSerializer`
was still populating the shared `visited` set via `RichTableCell._get_text()`.
Downstream items were then treated as already chunked and the chunker
emitted 0 chunks (e.g. on DOCX layout/header-only tables).

This change:
- Isolates `visited` while exporting the table dataframe and only merges
  the local copy back into the shared set when non-empty text is produced.
- Promotes the header row into a data row when the dataframe has 0 data
  rows but non-empty column names, so header-only tables still emit text.
- Handles single-row single-column tables by emitting the cell text
  directly instead of an empty triplet list.
- Adds `_flatten_table_text` as a fallback for tables whose triplet form
  collapses to empty text.
- Guards against appending empty text to serialization parts.

Adds regression tests covering header-only tables, single-row
single-column tables, end-to-end `HierarchicalChunker`/`HybridChunker`
runs on header-only / single-cell rich tables, and verifies that empty
tables do not consume their children's refs.

Closes docling-project/docling#3335

Signed-off-by: Manohar Mallipudi <manohar.m4475@yahoo.com>
2026-05-28 15:41:47 +02:00
github-actions[bot] 756580dcd5 chore: bump version to 2.77.1 [skip ci] v2.77.1 2026-05-26 15:22:20 +00:00
Peter W. J. Staar 01f8d03cda fix: render RichTableCell descendants inside the cell (#621)
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-05-26 16:46:10 +02:00
github-actions[bot] ebee77b5a6 chore: bump version to 2.77.0 [skip ci] v2.77.0 2026-05-19 13:47:03 +00:00
Peter W. J. Staar 9187158ef8 feat: extending BaseMeta with language and entities for docling-agent (#611)
* feat: extending BaseMeta with language, entities, etc for docling-agent

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* ignore serialization of empty Meta in HTML

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* addressing the good comments from Cesar

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-05-19 15:43:35 +02:00
github-actions[bot] db78e34d8c chore: bump version to 2.76.0 [skip ci] v2.76.0 2026-05-18 15:34:29 +00:00
Panos Vagenas 430473ae54 chore(doclang): add document index support (#616)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-05-18 16:51:58 +02:00
Panos Vagenas 9d4b288a33 chore(doclang): align lists, tables, groups, headings (#610)
* chore(doclang): align lists, tables, groups, headings

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add aligned virtual text support to lists & tables

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-05-18 12:36:02 +02:00
Peter W. J. Staar b6c3d6fb14 feat: updated the table visualization (#615)
updated the table visualization

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-05-18 07:30:34 +02:00
github-actions[bot] 7cc62cbde7 chore: bump version to 2.75.0 [skip ci] v2.75.0 2026-05-12 14:54:25 +00:00
Peter W. J. Staar 014948b0e8 feat: updated the HTML serialization (#609)
* updated the HTML serialization

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the tests

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-05-12 16:49:08 +02:00
48c5b97593 fix(DocLang): fix chemistry serialization. (#607)
* fix chemistry serialization. use molecule meta instead of classification

* DCO Remediation Commit for VWE@zurich.ibm.com;1G0002848;Valery Weber <vwe@login4.bluevela.rmf.ibm.com>

I, VWE@zurich.ibm.com;1G0002848;Valery Weber <vwe@login4.bluevela.rmf.ibm.com>, hereby add my Signed-off-by to this commit: e77b7e9478

Signed-off-by: VWE@zurich.ibm.com;1G0002848;Valery Weber <vwe@login3.bluevela.rmf.ibm.com>

---------

Signed-off-by: VWE@zurich.ibm.com;1G0002848;Valery Weber <vwe@login3.bluevela.rmf.ibm.com>
Co-authored-by: VWE@zurich.ibm.com;1G0002848;Valery Weber <vwe@login4.bluevela.rmf.ibm.com>
Co-authored-by: VWE@zurich.ibm.com;1G0002848;Valery Weber <vwe@login3.bluevela.rmf.ibm.com>
2026-05-08 13:43:26 +02:00
Michele Dolfi 6a6512ebe4 docs(security): Document security processes (#606)
add SECURITY.md file

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
2026-05-08 10:00:26 +02:00
github-actions[bot] 275bd1d217 chore: bump version to 2.74.1 [skip ci] v2.74.1 2026-04-22 14:33:02 +00:00
Panos Vagenas 2087d0f362 fix: refine ImageRef URI handling (#595)
* fix: refine ImageRef URI handling

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* make URI handling configurable via settings

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-04-22 16:13:10 +02:00
Ahmed NassarandAhmed Nassar AHN@zurich.ibm.com 048f1720f6 fix(doclang): default DoclangDeserializer to page 1 (#590)
* fix(doclang): default DoclangDeserializer to page 1 and allow start offset

Doclang VLM output was assigning provenance to page 0 while DoclingDocument and ground truth use 1-based page numbers, which broke page-aligned matching.

- Default internal _page_no to 1 and add deserialize(..., page_no=1).
- Replace hardcoded self._page_no = 0 with the page_no argument.
- Update roundtrip_list_item_with_inline deserialized fixture for page 1.

* DCO Remediation Commit for Ahmed Nassar AHN@zurich.ibm.com <AHN@zurich.ibm.com>

I, Ahmed Nassar AHN@zurich.ibm.com <AHN@zurich.ibm.com>, hereby add my Signed-off-by to this commit: 0c34abd5c3

Signed-off-by: Ahmed Nassar AHN@zurich.ibm.com <AHN@zurich.ibm.com>

---------

Signed-off-by: Ahmed Nassar AHN@zurich.ibm.com <AHN@zurich.ibm.com>
Co-authored-by: Ahmed Nassar AHN@zurich.ibm.com <AHN@zurich.ibm.com>
2026-04-21 17:54:48 +02:00
Panos Vagenas 473fbacfb9 fix: refine remote filename handling (#591)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-04-21 16:37:04 +02:00
github-actions[bot] 0425dc0c03 chore: bump version to 2.74.0 [skip ci] v2.74.0 2026-04-17 06:48:41 +00:00
Matteo b72af126b7 fix(DocLang): fix chemistry serialization (#584)
added correct serialization of chemistry into doclang

Signed-off-by: MatteoOmenetti <omenetti.matteo@gmail.com>
2026-04-17 07:33:45 +02:00
Smeet AgrawalandClaude Sonnet 4.6 9dc882dc48 feat(serializer): add MsExcelMarkdownDocSerializer for sheet-name headings (#587)
* feat(serializer): add MsExcelMarkdownDocSerializer for sheet-name headings

Add `MsExcelMarkdownFallbackSerializer` and `MsExcelMarkdownDocSerializer`
to the serializer package so that `GroupLabel.SHEET` groups are rendered as
level-2 Markdown headings when exporting Excel-sourced documents.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* DCO Remediation Commit for Smeet23 <smeetagrawal2003@gmail.com>

I, Smeet23 <smeetagrawal2003@gmail.com>, hereby add my Signed-off-by to this commit: 2a3808e5dc

Signed-off-by: Smeet23 <smeetagrawal2003@gmail.com>

* style: apply ruff formatter to markdown_excel.py

Signed-off-by: Smeet23 <smeetagrawal2003@gmail.com>

---------

Signed-off-by: Smeet23 <smeetagrawal2003@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 07:01:13 +02:00
Cesar Berrospi Ramis 6cbdee9626 fix: prevent numeric precision loss in Markdown table serialization (#588)
* fix: preserve numeric precision in markdown serialization

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* test: apply style conventions to test_serialization.py module

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-04-15 17:16:57 +02:00
f2a61868d4 feat: DocChunk expansion (#549)
* line_chunker

* split table to header and body

* duplicat table headers

* Revert "duplicat table headers"

This reverts commit 5d17bdacf2.

* Revert "split table to header and body"

This reverts commit 91b43f97e4.

* Revert "line_chunker"

This reverts commit 5cc61d93fb.

* chunk expansion + test

Signed-off-by: odelliab <odelliab@il.ibm.com>

* small fixes

Signed-off-by: odelliab <odelliab@il.ibm.com>

* small fixes

Signed-off-by: odelliab <odelliab@il.ibm.com>

* remove unnecessary code

Signed-off-by: odelliab <odelliab@il.ibm.com>

* DCO Remediation Commit for odelliab <odelliab@il.ibm.com>

I, odelliab <odelliab@il.ibm.com>, hereby add my Signed-off-by to this commit: 5cc61d93fb
I, odelliab <odelliab@il.ibm.com>, hereby add my Signed-off-by to this commit: 91b43f97e4
I, odelliab <odelliab@il.ibm.com>, hereby add my Signed-off-by to this commit: 5d17bdacf2
I, odelliab <odelliab@il.ibm.com>, hereby add my Signed-off-by to this commit: a50392e53c
I, odelliab <odelliab@il.ibm.com>, hereby add my Signed-off-by to this commit: e5894290d5
I, odelliab <odelliab@il.ibm.com>, hereby add my Signed-off-by to this commit: 30c72a99be
I, odelliab <91875866+odelliab@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 6aa0019fe0

Signed-off-by: odelliab <odelliab@il.ibm.com>

* change names

Signed-off-by: odelliab <odelliab@il.ibm.com>

* remove some tests

Signed-off-by: odelliab <odelliab@il.ibm.com>

* address review comments

Signed-off-by: odelliab <odelliab@il.ibm.com>

* Apply suggestions from code review

Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
Signed-off-by: odelliab <91875866+odelliab@users.noreply.github.com>

* consolidate tests

Signed-off-by: odelliab <odelliab@il.ibm.com>

* fix failing test

Signed-off-by: odelliab <odelliab@il.ibm.com>

* test: regenerate ground truth for hybrid chunker test

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor: extract chunk expansion into TreeChunkExpander and PageChunkExpander

- Create chunk_expander.py with TreeChunkExpander and PageChunkExpander classes
- Remove DocChunk expansion methods
- Improve docstrings
- Refactor tests: test_doc_chunk_expansion.py → test_chunk_expander.py

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: odelliab <odelliab@il.ibm.com>
Signed-off-by: odelliab <91875866+odelliab@users.noreply.github.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-04-15 17:16:28 +02:00
github-actions[bot] d2e79f4a80 chore: bump version to 2.73.0 [skip ci] v2.73.0 2026-04-09 08:08:14 +00:00
Cesar Berrospi Ramis 18f573899b feat(ouline): extend OutlineDocSerializer with filtering capabilities (#580)
* feat(outline): add filtering params to OutlineDocSerializer

Add start_item and max_level parameters for flexible outline generation.
Refactor get_parts to reduce complexity below Ruff C901 threshold.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* fix(outline): preserve spans in serialization results

Pass span_source to create_ser_result in all outline serializers.
Add test_outline_serialization_spans to verify get_unique_doc_items().

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(outline): reduce complexity and improve code quality

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-04-09 09:56:02 +02:00
Jordan White 7d8c9db2a7 docs: Fixes a typo in CONTRIBUTING.md (#582)
* Fixes a typo in CONTRIBUTING.md

I spotted this while checkign the contributing docs to create my
previous MR and thought it'd be nice to just push up a quick fix.

* DCO Remediation Commit for Jordan White <jordan.page.white@gmail.com>

I, Jordan White <jordan.page.white@gmail.com>, hereby add my Signed-off-by to this commit: b4026f63dd

Signed-off-by: Jordan White <jordan.page.white@gmail.com>

---------

Signed-off-by: Jordan White <jordan.page.white@gmail.com>
2026-04-09 07:57:11 +02:00
Peter W. J. Staar 46a9b5a329 feat: add latex and Tikz as codelabels (#579)
* feat: add latex and Tikz as codelabels

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fix the docs

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-04-09 06:46:18 +02:00
github-actions[bot] b7d35cef4b chore: bump version to 2.72.0 [skip ci] v2.72.0 2026-04-07 12:35:15 +00:00
mergify[bot] aaa1fbb031 ci(mergify): upgrade configuration to current format (#576)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-06 09:19:25 +02:00
Panos Vagenas 00c3bb223d feat(Doclang): add newline handling (#575)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-04-01 17:13:37 +02:00
Peter W. J. StaarandPanos Vagenas f20068db91 feat: add transforms in the hierarchy (#572)
* feat: add transforms in the hierarchy

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* implement hierachizing & flattening based on shift/tree operations

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* address conflict

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com>
2026-04-01 17:12:06 +02:00
github-actions[bot] 0388a99570 chore: bump version to 2.71.0 [skip ci] v2.71.0 2026-03-30 15:47:39 +00:00
Panos Vagenas 0bd5d8e649 feat: add code representation meta field (#573)
* feat: add code representation meta field

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add code language

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* restore custom field API changes

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-03-30 17:43:39 +02:00
Panos Vagenas c9b51520d2 fix(Doclang): improve checkbox serialization & deserialization (#570)
Wrap in `<text>` if needed.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-03-30 12:31:07 +02:00
Panos Vagenas a1535bc1d8 fix(Doclang): fix serialization order in text items (#571)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-03-30 10:44:45 +02:00
Panos Vagenas 57187b2364 chore: address subtree moving within same parent (#564)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-03-30 10:41:54 +02:00
Panos Vagenas fe9bbfbb0f feat(Doclang): add content layer support (#568)
* feat(Doclang): add content layer support

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* rename layer attribute

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-03-30 10:40:49 +02:00