Files
docling-core/test/data/doc/flattened.json
Panos Vagenas 28342dcc6c chore(doclang): align with DocLang v0.5 foundation (#630)
* chore(doclang): switch to suppressing namespace and version by default

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): align with DocLang 0.4

- Align DocLang serialization/deserialization with the doclang 0.4 schema (`<label value>`, `<layer value>`, `<href uri>`, `<src uri>`, element-head property order).
- Auto-wrap orphan key/value markup in `<field_item>` / `<field_region>` when the document tree is missing those containers.
- Add optional `doclang-validation` extra and `verify_doclang()` so golden tests also pass the reference `doclang` validator.
- Refresh `.dclg.xml` fixtures and add coverage for hyperlinks, field-region wrapping, and known incompat output under `test/data/doc/incompat/`.
- Pin emitted DocLang version to 0.4 and defer `<layer>`-related expectations until doclang 0.5.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] add basic virtual `<text>` support

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] add virtual `<text>` support

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): extend virtual `<text>` support, unify el. head parsing

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): align code/picture labels to DocLang, add LabelMode

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): emit bare `<value>` for read-only field values

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] align with DocLang v0.5

- Bump doclang validation to 0.5 and enable <layer> emission.
- Align open-tag shapes with the recommended vocabulary: bare <heading>/<field_heading> at level 1, bare <list> for unordered lists.
- Add field_heading deserializer support and remove deferred layer xfails.

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): add `<rtl>`/`<corn>`/`<xcel>` coverage

- Emit <rtl> for RTL script text via get_text_direction()
- Emit <corn> for row/column header intersection cells
- Fix OTSL span tokens so <xcel> only appears at interior continuations
- Add rtl/corn/xcel golden tests; defer multi-prov threading test to P2

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* chore(doclang): [WIP] drop legacy vocabulary

- Remove non-spec tokens (`section`, `watermark`, `time_break`, `facets`, `meta`, `uri`) and `GROUP@type` / `TABLE@class` from vocabulary
- Reject `class` on `<table>` at deserialize time; resolve label from `<table>` vs `<index>` only
- Drop legacy `class="data"` / `class="index"` table parsing paths and update deserializer tests accordingly
- Tighten `WRAP_WHEN_NEEDED` comment to include embedded newlines

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-06-08 16:42:35 +02:00

936 lines
20 KiB
JSON
Vendored

{
"schema_name": "DoclingDocument",
"version": "1.10.0",
"name": "",
"furniture": {
"self_ref": "#/furniture",
"children": [],
"content_layer": "furniture",
"name": "_root_",
"label": "unspecified"
},
"body": {
"self_ref": "#/body",
"children": [
{
"$ref": "#/texts/0"
},
{
"$ref": "#/texts/1"
},
{
"$ref": "#/texts/2"
},
{
"$ref": "#/texts/3"
},
{
"$ref": "#/texts/4"
},
{
"$ref": "#/texts/5"
},
{
"$ref": "#/texts/6"
},
{
"$ref": "#/texts/7"
},
{
"$ref": "#/texts/11"
},
{
"$ref": "#/texts/12"
},
{
"$ref": "#/texts/13"
},
{
"$ref": "#/texts/14"
},
{
"$ref": "#/texts/18"
},
{
"$ref": "#/texts/19"
},
{
"$ref": "#/texts/20"
},
{
"$ref": "#/texts/21"
},
{
"$ref": "#/groups/2"
},
{
"$ref": "#/texts/26"
},
{
"$ref": "#/tables/0"
},
{
"$ref": "#/texts/29"
},
{
"$ref": "#/field_regions/0"
}
],
"content_layer": "body",
"name": "_root_",
"label": "unspecified"
},
"groups": [
{
"self_ref": "#/groups/0",
"parent": {
"$ref": "#/texts/7"
},
"children": [
{
"$ref": "#/texts/8"
},
{
"$ref": "#/texts/9"
},
{
"$ref": "#/texts/10"
}
],
"content_layer": "body",
"name": "group",
"label": "inline"
},
{
"self_ref": "#/groups/1",
"parent": {
"$ref": "#/texts/14"
},
"children": [
{
"$ref": "#/texts/15"
},
{
"$ref": "#/texts/16"
},
{
"$ref": "#/texts/17"
}
],
"content_layer": "body",
"name": "group",
"label": "inline"
},
{
"self_ref": "#/groups/2",
"parent": {
"$ref": "#/body"
},
"children": [
{
"$ref": "#/texts/22"
},
{
"$ref": "#/texts/23"
}
],
"content_layer": "body",
"name": "group",
"label": "list"
},
{
"self_ref": "#/groups/3",
"parent": {
"$ref": "#/texts/23"
},
"children": [
{
"$ref": "#/texts/24"
},
{
"$ref": "#/texts/25"
}
],
"content_layer": "body",
"name": "group",
"label": "list"
},
{
"self_ref": "#/groups/4",
"parent": {
"$ref": "#/tables/0"
},
"children": [
{
"$ref": "#/texts/27"
},
{
"$ref": "#/texts/28"
}
],
"content_layer": "body",
"name": "group",
"label": "inline"
}
],
"texts": [
{
"self_ref": "#/texts/0",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "title",
"prov": [],
"orig": "Title",
"text": "Title"
},
{
"self_ref": "#/texts/1",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Some intro",
"text": "Some intro"
},
{
"self_ref": "#/texts/2",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "section_header",
"prov": [],
"orig": "Foo",
"text": "Foo",
"level": 1
},
{
"self_ref": "#/texts/3",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Foo stuff",
"text": "Foo stuff"
},
{
"self_ref": "#/texts/4",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "section_header",
"prov": [],
"orig": "Bar",
"text": "Bar",
"level": 2
},
{
"self_ref": "#/texts/5",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Bar stuff",
"text": "Bar stuff"
},
{
"self_ref": "#/texts/6",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "More stuff",
"text": "More stuff"
},
{
"self_ref": "#/texts/7",
"parent": {
"$ref": "#/body"
},
"children": [
{
"$ref": "#/groups/0"
}
],
"content_layer": "body",
"label": "section_header",
"prov": [],
"orig": "",
"text": "",
"level": 1
},
{
"self_ref": "#/texts/8",
"parent": {
"$ref": "#/groups/0"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Rich heading",
"text": "Rich heading"
},
{
"self_ref": "#/texts/9",
"parent": {
"$ref": "#/groups/0"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "without",
"text": "without",
"formatting": {
"bold": false,
"italic": true,
"underline": false,
"strikethrough": false,
"script": "baseline"
}
},
{
"self_ref": "#/texts/10",
"parent": {
"$ref": "#/groups/0"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "other children besides the inline",
"text": "other children besides the inline"
},
{
"self_ref": "#/texts/11",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Section content as sibling of the heading.",
"text": "Section content as sibling of the heading."
},
{
"self_ref": "#/texts/12",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "section_header",
"prov": [],
"orig": "Subheading",
"text": "Subheading",
"level": 2
},
{
"self_ref": "#/texts/13",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Subsection content.",
"text": "Subsection content."
},
{
"self_ref": "#/texts/14",
"parent": {
"$ref": "#/body"
},
"children": [
{
"$ref": "#/groups/1"
}
],
"content_layer": "body",
"label": "section_header",
"prov": [],
"orig": "",
"text": "",
"level": 1
},
{
"self_ref": "#/texts/15",
"parent": {
"$ref": "#/groups/1"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Rich heading",
"text": "Rich heading"
},
{
"self_ref": "#/texts/16",
"parent": {
"$ref": "#/groups/1"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "with",
"text": "with",
"formatting": {
"bold": false,
"italic": true,
"underline": false,
"strikethrough": false,
"script": "baseline"
}
},
{
"self_ref": "#/texts/17",
"parent": {
"$ref": "#/groups/1"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "other children besides the inline",
"text": "other children besides the inline"
},
{
"self_ref": "#/texts/18",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Section content as child of the heading.",
"text": "Section content as child of the heading."
},
{
"self_ref": "#/texts/19",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Section content as sibling of the heading.",
"text": "Section content as sibling of the heading."
},
{
"self_ref": "#/texts/20",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "section_header",
"prov": [],
"orig": "Heading",
"text": "Heading",
"level": 1
},
{
"self_ref": "#/texts/21",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Bar",
"text": "Bar"
},
{
"self_ref": "#/texts/22",
"parent": {
"$ref": "#/groups/2"
},
"children": [],
"content_layer": "body",
"label": "list_item",
"prov": [],
"orig": "List item",
"text": "List item",
"enumerated": false,
"marker": ""
},
{
"self_ref": "#/texts/23",
"parent": {
"$ref": "#/groups/2"
},
"children": [
{
"$ref": "#/groups/3"
}
],
"content_layer": "body",
"label": "list_item",
"prov": [],
"orig": "List item",
"text": "List item",
"enumerated": false,
"marker": ""
},
{
"self_ref": "#/texts/24",
"parent": {
"$ref": "#/groups/3"
},
"children": [],
"content_layer": "body",
"label": "list_item",
"prov": [],
"orig": "List item",
"text": "List item",
"enumerated": false,
"marker": ""
},
{
"self_ref": "#/texts/25",
"parent": {
"$ref": "#/groups/3"
},
"children": [],
"content_layer": "body",
"label": "list_item",
"prov": [],
"orig": "List item",
"text": "List item",
"enumerated": false,
"marker": ""
},
{
"self_ref": "#/texts/26",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "section_header",
"prov": [],
"orig": "Heading",
"text": "Heading",
"level": 2
},
{
"self_ref": "#/texts/27",
"parent": {
"$ref": "#/groups/4"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "text in italic ",
"text": "text in italic ",
"formatting": {
"bold": false,
"italic": true,
"underline": false,
"strikethrough": false,
"script": "baseline"
}
},
{
"self_ref": "#/texts/28",
"parent": {
"$ref": "#/groups/4"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "text in bold",
"text": "text in bold",
"formatting": {
"bold": true,
"italic": false,
"underline": false,
"strikethrough": false,
"script": "baseline"
}
},
{
"self_ref": "#/texts/29",
"parent": {
"$ref": "#/body"
},
"children": [],
"content_layer": "body",
"label": "section_header",
"prov": [],
"orig": "Heading",
"text": "Heading",
"level": 1
},
{
"self_ref": "#/texts/30",
"parent": {
"$ref": "#/field_regions/0"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Some text",
"text": "Some text"
},
{
"self_ref": "#/texts/31",
"parent": {
"$ref": "#/field_items/0"
},
"children": [],
"content_layer": "body",
"label": "text",
"prov": [],
"orig": "Some text",
"text": "Some text"
},
{
"self_ref": "#/texts/32",
"parent": {
"$ref": "#/field_items/0"
},
"children": [],
"content_layer": "body",
"label": "field_key",
"prov": [],
"orig": "Key",
"text": "Key"
},
{
"self_ref": "#/texts/33",
"parent": {
"$ref": "#/field_items/0"
},
"children": [],
"content_layer": "body",
"label": "field_value",
"prov": [],
"orig": "Value",
"text": "Value",
"kind": "read_only"
}
],
"pictures": [],
"tables": [
{
"self_ref": "#/tables/0",
"parent": {
"$ref": "#/body"
},
"children": [
{
"$ref": "#/groups/4"
}
],
"content_layer": "body",
"label": "table",
"prov": [],
"captions": [],
"references": [],
"footnotes": [],
"data": {
"table_cells": [
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 0,
"end_row_offset_idx": 1,
"start_col_offset_idx": 0,
"end_col_offset_idx": 1,
"text": "cell 0,0",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 0,
"end_row_offset_idx": 1,
"start_col_offset_idx": 1,
"end_col_offset_idx": 2,
"text": "cell 0,1",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 1,
"end_row_offset_idx": 2,
"start_col_offset_idx": 0,
"end_col_offset_idx": 1,
"text": "cell 1,0",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 1,
"end_row_offset_idx": 2,
"start_col_offset_idx": 1,
"end_col_offset_idx": 2,
"text": "",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false,
"ref": {
"$ref": "#/groups/4"
}
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 2,
"end_row_offset_idx": 3,
"start_col_offset_idx": 0,
"end_col_offset_idx": 1,
"text": "cell 2,0",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 2,
"end_row_offset_idx": 3,
"start_col_offset_idx": 1,
"end_col_offset_idx": 2,
"text": "cell 2,1",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 3,
"end_row_offset_idx": 4,
"start_col_offset_idx": 0,
"end_col_offset_idx": 1,
"text": "cell 3,0",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 3,
"end_row_offset_idx": 4,
"start_col_offset_idx": 1,
"end_col_offset_idx": 2,
"text": "cell 3,1",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
}
],
"num_rows": 4,
"num_cols": 2,
"orientation": "rot_0",
"grid": [
[
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 0,
"end_row_offset_idx": 1,
"start_col_offset_idx": 0,
"end_col_offset_idx": 1,
"text": "cell 0,0",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 0,
"end_row_offset_idx": 1,
"start_col_offset_idx": 1,
"end_col_offset_idx": 2,
"text": "cell 0,1",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
}
],
[
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 1,
"end_row_offset_idx": 2,
"start_col_offset_idx": 0,
"end_col_offset_idx": 1,
"text": "cell 1,0",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 1,
"end_row_offset_idx": 2,
"start_col_offset_idx": 1,
"end_col_offset_idx": 2,
"text": "",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
}
],
[
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 2,
"end_row_offset_idx": 3,
"start_col_offset_idx": 0,
"end_col_offset_idx": 1,
"text": "cell 2,0",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 2,
"end_row_offset_idx": 3,
"start_col_offset_idx": 1,
"end_col_offset_idx": 2,
"text": "cell 2,1",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
}
],
[
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 3,
"end_row_offset_idx": 4,
"start_col_offset_idx": 0,
"end_col_offset_idx": 1,
"text": "cell 3,0",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
},
{
"row_span": 1,
"col_span": 1,
"start_row_offset_idx": 3,
"end_row_offset_idx": 4,
"start_col_offset_idx": 1,
"end_col_offset_idx": 2,
"text": "cell 3,1",
"column_header": false,
"row_header": false,
"row_section": false,
"fillable": false
}
]
]
},
"annotations": []
}
],
"key_value_items": [],
"form_items": [],
"field_regions": [
{
"self_ref": "#/field_regions/0",
"parent": {
"$ref": "#/body"
},
"children": [
{
"$ref": "#/texts/30"
},
{
"$ref": "#/field_items/0"
}
],
"content_layer": "body",
"label": "field_region",
"prov": []
}
],
"field_items": [
{
"self_ref": "#/field_items/0",
"parent": {
"$ref": "#/field_regions/0"
},
"children": [
{
"$ref": "#/texts/31"
},
{
"$ref": "#/texts/32"
},
{
"$ref": "#/texts/33"
}
],
"content_layer": "body",
"label": "field_item",
"prov": []
}
],
"pages": {}
}