mirror of
https://github.com/docling-project/docling-core.git
synced 2026-11-18 20:27:00 +00:00
* feat: add 'meta' field to DoclingDocument at root level
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* test(metadata): reuse DoclingDocument objects with fixtures
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* chore(serializer): add document-level meta to the outline serialization
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* test: add helper function to compare or regenerate ground truth files
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* test: regenerate (upgrade) ground truth files
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(serializer): set default indent to 2 spaces in outline serializer
To be aligned with other indented text serializations, the outline serializer takes 2 spaces
as default indent for serialization.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor(serializer): add line break after ref in outline md serialization
To render a line break after the reference element in the outline markdown serializer, add 2 trailing spaces.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
* refactor: rollback adding 'meta' field to DoclingDocument
Revert the changes in commit 7e17c52 since DoclingDocument meta-information can
alreadby be achieved by leveraging the 'body' field, which is of type 'NodeItem'
and thus it has a 'meta' field.
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
---------
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
23 lines
726 B
Python
23 lines
726 B
Python
from pathlib import Path
|
|
|
|
from docling_core.transforms.chunker.hierarchical_chunker import DocChunk
|
|
from docling_core.transforms.chunker.page_chunker import PageChunker
|
|
from docling_core.types.doc.document import DoclingDocument
|
|
|
|
from .test_utils import assert_or_generate_json_ground_truth
|
|
|
|
|
|
def test_page_chunks():
|
|
src = Path("./test/data/doc/cross_page_lists.json")
|
|
doc = DoclingDocument.load_from_json(src)
|
|
|
|
chunker = PageChunker()
|
|
|
|
chunk_iter = chunker.chunk(dl_doc=doc)
|
|
chunks = list(chunk_iter)
|
|
act_data = dict(root=[DocChunk.model_validate(n).export_json_dict() for n in chunks])
|
|
assert_or_generate_json_ground_truth(
|
|
act_data,
|
|
src.parent / f"{src.stem}_chunks.json",
|
|
)
|