Yusik Kim
4708f933a7
fix: properly handle missing page image case for export_to_html ( #166 )
...
* fix: properly handle missing image case
Signed-off-by: Yusik Kim <kmyusk@gmail.com >
* chore: pre-commit changes
Signed-off-by: Yusik Kim <kmyusk@gmail.com >
* fix: simplify test and address broken test
Signed-off-by: Yusik Kim <kmyusk@gmail.com >
---------
Signed-off-by: Yusik Kim <kmyusk@gmail.com >
2025-02-25 09:43:39 +01:00
d622800750
feat: Introduce Key-Value and Forms items ( #158 )
...
* Draft KeyValueItem content
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* added add_key_value_item method
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* add KeyValueLink
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* update an add_key_value_item argument
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* remove KeyOrValueCellType and KeyValueLinkType
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* update tests for KeyValueItem
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* add union method to create bbox that covers all the given bboxes
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* added the , and rewrote it to make it more general with a GraphItem, as well as having cell- and link-labels
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the code due to MyPy and reformatted
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the reference docs with form-key
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated image
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* removed the circle
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* fixed the figure, testing now ...
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* added square in image
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* removed square in image
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* tests should go through
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* update poetry.lock versions
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* fix import issue
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* add validator for links in GraphData
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* added field_validator
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* rename add_key_values and add_form
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* rename union to enclosingbbox
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
Co-authored-by: Christoph Auer <cau@zurich.ibm.com >
Co-authored-by: Peter Staar <taa@zurich.ibm.com >
2025-02-19 09:50:45 +01:00
Peter W. J. Staar and Michele Dolfi
916323fb55
feat: Redefine CodeItem as floating object with captions ( #160 )
...
* updated the CodeItem with captions
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* use FloatItem and add captions to add_code interface
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com >
2025-02-14 11:34:40 +01:00
f751b45b62
feat: Implementation of doc tags ( #138 )
...
* Migration to new DocTags, gt missing
Signed-off-by: Ahmed Nassar <ahn@zurich.ibm.com >
* fixed the tests
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* overloaded export to markdown in CodeItem to include code language
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* minor change
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* implemented page break token instead of page number
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added figure classification in doc tags
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* changed table repr. to otsl
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* changed table repr. to otsl fixed new line
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* fixed new line bug
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* fixed new line bug
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* removed print in test
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* new normalization to 500 and top left coord
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* export of FOOTER AND HEADER
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* export of FOOTER AND HEADER
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* fix code bug
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added export of standalon captions
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added export of standalon captions
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* changed generate to False
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added export of furnitures and fixed bug in iterate_items function
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
---------
Signed-off-by: Ahmed Nassar <ahn@zurich.ibm.com >
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
Co-authored-by: Ahmed Nassar <ahn@zurich.ibm.com >
Co-authored-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
2025-02-13 13:26:28 +00:00
Christoph Auer and Panos Vagenas
786f0c6833
feat: Add ContentLayer attribute to designate items to body or furniture ( #148 )
...
* feat: Add ContentLayer attribute to designate items to body or furniture
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* introduce safer data gen mechanism, update chunking test data
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* Do not make test rely on order in yaml
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* chore: format fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* fix: legacy_to_docling_doc must use content_layer
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add content_layer in iterate_items
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Bump format version, add model_validator for old page_header,page_footer in body
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* fix: Change to before model_validator
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Update tests
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Address review comments
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2025-02-10 10:58:41 +01:00
Michele Dolfi
23477f7674
feat(HTML): Fallback showing formulas as images ( #146 )
...
* remove un-needed logic
the labels allowlist is checked before
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* textitem cannot have label code
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* display formulas with mathml in exported html
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* expose argument in save_as_html
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* rename sanitize in prepare and add \n
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix mypy parsing
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* remove unused/impossible elif
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* remove strip()
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add display none for latex annotation
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fallback showing equations as image
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add html placeholder
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* markdown placeholder
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-02-03 09:29:30 +01:00
Michele Dolfi
ed36437346
feat(HTML): Export formulas with mathml ( #144 )
...
* remove un-needed logic
the labels allowlist is checked before
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* textitem cannot have label code
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* display formulas with mathml in exported html
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* expose argument in save_as_html
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* rename sanitize in prepare and add \n
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix mypy parsing
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* remove unused/impossible elif
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* remove strip()
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add display none for latex annotation
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* make mathml the default
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* revert wrong commit
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-01-31 15:43:43 +01:00
Michele Dolfi
c6590e83e2
fix: add html escape in md export and fix formula escapes ( #143 )
...
* add html escape in md export and fix formula escapes
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* escape also html export
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-01-31 11:31:35 +01:00
Peter W. J. Staar
f02bbaea47
feat: added the geometric operations to BoundingBox ( #136 )
...
* added the geometric operations
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the geometric operations and added deprecation warnings to scaled and normalize
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* finalised the geometric operations
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated tests and functionality
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
2025-01-29 11:11:46 +01:00
Panos Vagenas
b741eeaab4
fix: fix hybrid chunker token constraint ( #131 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2025-01-17 17:22:26 +01:00
c940aa5ca9
feat: Add CodeItem as pydantic type, update export methods and APIs ( #129 )
...
* added code item
* added code item
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* add constraints to allow numpy > 2.1.0 on python3.13 and others
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Add CodeItem to ContentItem
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* added CodeItem in ContentItem tagged union.
* added enum for programming languages
* removed double CodeItem in ContentItem Union
* fixed type of code_language in CodeItem class
* fixed sorting of programming languages, not sorted anymore by value of string but variable name
---------
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Co-authored-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Christoph Auer <cau@zurich.ibm.com >
2025-01-17 16:59:33 +01:00
Michele Dolfi
268c294cc9
fix: always write with utf8 encoding ( #111 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-12-17 09:37:01 +01:00
Michele Dolfi
b31e0a3d05
feat: add legacy to DoclingDocument utility ( #108 )
...
* add legacy_to_docling_document
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix for python < 3.12
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add test
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add docs with not-preserved components
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-12-13 13:28:41 +01:00
Panos Vagenas
047a1960af
fix: improve doc item typing ( #105 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-12-13 12:51:53 +01:00
Panos Vagenas
b546c0a50d
fix: set origin when merging chunks ( #109 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-12-13 09:05:56 +01:00
Panos Vagenas
2591c70c66
feat: add default tokenizer to HybridChunker ( #107 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-12-12 16:51:08 +01:00
Michele Dolfi and Christoph Auer
437c498f77
feat: utilities converting document formats ( #91 )
...
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Christoph Auer <cau@zurich.ibm.com >
2024-12-09 14:14:01 +01:00
Eric Musa
e8f775569b
Fix: solved embedded image serialization/deserialization with Pydantic. ( #95 )
...
Signed-off-by: Eric Musa <eric.musa17@gmail.com >
2024-12-09 14:12:59 +01:00
Boyce
fd7529f409
fix(markdown): preserve underscores in image URLs during markdown export ( #98 )
...
Signed-off-by: Boyce <aboyskyi@gmail.com >
2024-12-09 11:38:23 +01:00
628ab679cb
feat: add hybrid chunker ( #68 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Bill Murdock <bmurdock@redhat.com >
Co-authored-by: Ben Rood <bash99@gmail.com >
2024-12-06 14:56:10 +01:00
Cesar Berrospi Ramis
266e7fc603
build: set pydantic version to 2.10.3 ( #93 )
...
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
2024-12-04 14:26:22 +01:00
Maxim Lysak and Maksym Lysak
180e294aad
feat: Export to OTSL method for docling doc tables ( #86 )
...
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com >
2024-12-03 16:31:14 +01:00
Michele Dolfi and Panos Vagenas
9a74d13fd6
feat: extend source resolution with streams and workdir ( #79 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-12-02 14:12:55 +01:00
Maxim Lysak and Maksym Lysak
8422ad4fbf
fix: Even better style for HTML export ( #78 )
...
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com >
2024-11-28 16:54:01 +01:00
Maxim Lysak and Maksym Lysak
a94ba273b1
chore: Improving styling for HTML export ( #77 )
...
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com >
2024-11-27 20:19:43 +01:00
ef49fd3f34
feat: adding HTML export to DoclingDocument, adding export of images in png with links to Markdown & HTML ( #69 )
...
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Christoph Auer <cau@zurich.ibm.com >
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com >
2024-11-27 05:27:23 +01:00
Shubham Gupta and Shubham Gupta
9d7e831fb2
feat: add get_image for all DocItem ( #67 )
...
* Moved image attribute from FloatingItem to DocItem
Signed-off-by: Shubham Gupta <26436285+sh-gupta@users.noreply.github.com >
* Revert "Moved image attribute from FloatingItem to DocItem"
This reverts commit e48cd47d3743a25ce17bbaf31b3152ed53365fba.
Signed-off-by: Shubham Gupta <26436285+sh-gupta@users.noreply.github.com >
* Added get_image to DocItem and FloatingItem
Signed-off-by: Shubham Gupta <26436285+sh-gupta@users.noreply.github.com >
* Added tests for get_image in DocItem and FloatingItem
Signed-off-by: Shubham Gupta <26436285+sh-gupta@users.noreply.github.com >
* Updated get_image docstring in DocItem and FloatingItem
Signed-off-by: Shubham Gupta <26436285+sh-gupta@users.noreply.github.com >
---------
Signed-off-by: Shubham Gupta <26436285+sh-gupta@users.noreply.github.com >
Co-authored-by: Shubham Gupta <Shubham.Gupta1@ibm.com >
2024-11-18 13:34:16 +01:00
Panos Vagenas
bfeb2db24b
fix: include titles to chunk heading metadata ( #62 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-11-01 13:41:16 +01:00
Panos Vagenas
2c88e56f8c
fix: fix resolution in case of URL without path ( #55 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-24 13:07:59 +02:00
Panos Vagenas
d09fe7ed44
feat: extend chunk meta with schema, version, origin ( #49 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-22 11:08:04 +02:00
Panos Vagenas
e12d6a70c3
fix: fix legacy doc ref ( #48 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-18 15:21:01 +02:00
03df97fa73
feat!: Expose DoclingDocument as main type, move old typing to legacy ( #41 )
...
* Fix area method of BoundingBox
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* add image placeholder
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* enable picture label
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* refactor captions and markdown
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add logic to skip repeated caption
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* use DocItemLabel
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Extend default export labels, add convenience mehtods
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Introduce ListItem API, with marker and enumerated properties
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* add classification and description in PictureData
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add molecule picture data
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Fixes for DoclingDocument and aligned methods on legacy doc
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* add advanced picture data content
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Many markdown export fixes, renaming BaseTableData
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Rename module paths doc->legacy_doc, experimental->doc
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* feat: imageref with pil_image
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Small fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* docs: remove documentation in markdown to support python 3.13 (#43 )
Since json-schema-for-humans dependency does not support python 3.13,
remove the generation of documentation in markdown of main docling types.
Remove 'ds' prefix from documentation scripts.
Update README.
Add python 3.13 in CI/CD workflow checks.
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* Fix TableCell model validator
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* store list of classes in classification
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Fixes for DocumentOrigin mimetype validation
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* introduce picturedata as list of annotations
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* feat: adapt hierarchical chunker to v2 DoclingDocument
[skip-ci]
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* feat: add table support in chunker, incl. captions
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* use Field constraints instead of conlist, refactor chunking types
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* revert unnecessary doc module change
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* align test data with upstream changes
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* Update __init__.py on docling_core.types.doc
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Remove DescriptionItem
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-16 15:53:44 +02:00
Panos Vagenas
39ae6cd253
chore: minor refactor incl. explicit typing ( #39 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-07 13:59:40 +02:00
Panos Vagenas
b5592ad747
fix: align chunk ref format with one used in Document ( #37 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-07 09:07:08 +02:00
688789ea75
feat: (experimental) introduce new document format ( #21 )
...
* Draft new docling document format, pydantic model and tests
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Fix tests to have unique document_hashes per test
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Manual update from main
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Move new-format to experimental path
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Updates for document construction API and format
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add comments
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add BaseTableData and table cell typing
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Tree element iterator, several API fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Turn captions into list field
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add export methods to DoclingDocument and types
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Change DoclingDocument.iterate_elements and add print tree function
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Introduce label enum types, apply everywhere
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Introduce provenance info, use enum labels
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Update formatting
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Docstrings and linter fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Lockfile rollback, since updating breaks tests
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Cleanup
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Several improvements and cleanup
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Format fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Big redesign for usage of hashes, several other fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Fix flake8 config
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Remove hash, renamings
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* updating the tests
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* added some unit tests for DocItem
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* Add tree validation API and test case
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add extra=Forbid to NodeItem
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* feat: set DoclingDocument version as SemanticVersion with default
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* Cleanup
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Simpler literal enforcement
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Fix static document version
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Rollback changes to allow for semver<3
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* build: rollback changes to include python-semantic-release as dev dependency
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* feat: set version field as string with pattern and check compatibility
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* add JSON Pointer validation to refs, fix test data
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Peter Staar <taa@zurich.ibm.com >
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-01 09:41:16 +02:00
Panos Vagenas
4bde51528d
feat: support heading as chunk metadata ( #36 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-09-30 11:08:09 +02:00
Cesar Berrospi Ramis
a4ddd142ee
fix: change order of JSON Schema to search mapper transformations ( #32 )
...
* fix: change order of JSON Schema to search mapper transformations
In the JSON Schema to search mapper, the suppression flags need to be addressed first,
otherwise the jsonref.replace_refs function may remove them.
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* build: update dependencies
Updating pydantic from 2.8.2 to 2.9.2 triggers a change in JSON Schema from models:
the lists with only 1 element get flatten.
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* chore: improve verbose in JSON Schema to search mapper test
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
---------
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
2024-09-26 13:49:18 +02:00
Michele Dolfi
a334b9fc72
fix: remove duplicate captions in markdown ( #31 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-09-24 14:33:56 +02:00
Michele Dolfi
b843ae6688
feat: add figures in markdown export ( #27 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-09-23 20:12:06 +02:00
Peter W. J. Staar
891530f595
feat: add export to doctags for document components ( #25 )
...
* feat: refactoring doc-tokens in new file and add new export function to table
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* reformatted and fixed bugs
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* working on table exporting in document tokens
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated code and tests, need to decide what to commit as test-cases
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the test cases
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored the test-files
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored duplicate code
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored duplicate code (boundingbox to figure and table)
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored duplicate code (making BaseCell the base-class of BaseText)
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored duplicate code (re-use of function in BaseCell)
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* fixed issues from review (2)
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
2024-09-20 14:16:33 +02:00
Panos Vagenas
9698d30288
feat: add hierarchical chunker ( #18 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-09-11 12:57:56 +02:00
Peter W. J. Staar
acdf816081
feat: added the XML export ( #16 )
...
* added the XML export
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* reformatted all
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* fixed tests
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* added the DocumentTokens class
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updating the to-xml method
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updating the to-xml method
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* fixed the to-md method
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* added the strict-text in the to-md method
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* added page-tokens
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the location/page tokens
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
2024-09-09 17:41:58 +02:00
Michele Dolfi
f450c8cbfd
fix: use same base type for all components ( #10 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-08-28 11:21:05 +02:00
Cesar Berrospi Ramis
faf472c168
fix: set type to optional ( #7 )
...
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
2024-07-23 14:07:42 +02:00
Panos Vagenas
d0ffc85e0c
feat: add document Markdown export ( #4 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-07-18 10:12:43 +02:00
Cesar Berrospi Ramis
656f56380f
fix: fix definition issues in record type ( #2 )
...
* fix(test): set static typing compatible to python 3.10
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* style: enforce some pre-commit hooks on tests
Enforce pre-commit hooks black, isort, autoflake, and mypy on test modules.
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* fix(rec): fix definition issues in attribute, predicate, subject
Remove duplicate generic types across base and predicate modules.
Create an identifier class for subject names.
Remove unnecessary type variables in attribute model.
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* docs: refer to Docling data objects
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
---------
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
2024-07-17 17:24:31 +02:00
Cesar Berrospi Ramis
9134ffee93
Initial commit
...
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
2024-07-15 12:55:20 +02:00