* test(doclang): add serialization test for multiple captions & footnotes
Add DoclingDocument fixtures with pictures, a table and a code block
carrying multiple captions and footnotes: above, below and beside their
host, interleaved, and without provenance. A mirrored fixture has them at
body level instead of as children of their host. Add a golden-file test
for their DocLang serialization.
Multiple captions are currently merged into a single <caption> element
with several locations, which fails DocLang schema validation and breaks
the deserialize/reserialize roundtrip. Until that is fixed, the test
compares against the golden file without schema validation, and the
roundtrip steps are commented out.
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
* fix(doclang): serialize multiple captions and footnotes of a float as satellites
All captions of a picture/table were merged into its single <caption>,
yielding invalid DocLang, while those of a code block were escaped into
its content. Keep only the first caption in the element head and emit any
further captions as <text> and footnotes as <footnote> next to the host,
placed before or after it by bbox position, all wrapped in a <group>.
On deserialization, read such a group back as the float with its
captions and footnotes (a convention until DocLang supports this).
Enable schema validation and the roundtrip in the corresponding test.
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
---------
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
* fix(serializer): derive markdown table header rows from column_header flags
MarkdownTableSerializer hardcoded grid row 0 as the only header row, while
TableItem._export_to_dataframe_with_options, the HTML serializer and the OTSL
export all derive the header rows from TableCell.column_header. A table with a
stacked header therefore serialized to markdown with its lower header rows
demoted to data rows, and HybridChunker repeated only the first of them at the
top of each chunk of an oversized table.
Take the header row count from the flags and flatten those rows into the single
header row GFM allows, dropping the repeated text a cell spanning several header
rows produces. When no cell in the table carries column_header the previous
behavior is kept, since backends that never set the flag cannot be told apart
from genuinely headerless tables.
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
* test(serializer): cover tables whose header flags start below row 0
The empty-header-block branch of _flatten_header_rows was the one line of the
patch left uncovered. Pin the behavior it produces: with no leading run of
flagged rows there is no header to promote, so the header is empty and every
row stays in the body rather than one being consumed.
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
* fix(serializer): join stacked header cells with a documented separator
Review feedback: a blank space loses the boundary between header levels, so
"native backend" over "TTS" is indistinguishable from a single cell reading
"native backend TTS". Join with " - " instead, kept as a module constant, and
document the header-resolution contract on MarkdownTableSerializer.
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
* fix(serializer): only count header rows a column header starts on
TableData.grid repeats a row-spanning cell into every row it covers, so a
header cell with row_span > 1 leaves a column_header-flagged cell in the data
rows beneath it. Counting every row that holds a flagged cell therefore pulled
those data rows into the header block, where they were flattened into the
column names and dropped from the body entirely.
Require that a header cell starts on the row, matching the same fix applied to
the DataFrame export in #756.
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
* perf(serializer): avoid iterating grid twice; style and tests improvements
In '_count_header_rows' of the Markdown serializer, avoid iterating
the entire grid twice in all scenarios.
Strengthen docstrings in markdown serializer.
Add test for single header row with flags.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
---------
Signed-off-by: Anusha Acharya <acharyaanusha123@gmail.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
OpenCV's imencode defaults to compression level 1 while PIL's PNG encoder
defaults to 6, so PNGs from the cv2 path were 30-60% larger for identical
pixels. Regenerated the affected test ground truth.
Fixes#758
Signed-off-by: Yiorgis Gozadinos <ggozadinos@gmail.com>
test: rename 'test' folder to 'tests'
To align with other Docling repositories, use 'tests' for test modules.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>