Cesar Berrospi Ramis
8c55233d74
feat: add document question and answer annotations
...
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
2025-04-10 09:04:54 +02:00
Panos Vagenas and GitHub
5e4c0fd51f
feat: integrate serialization API into chunkers ( #221 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-04-08 14:04:40 +02:00
e9259a5f87
feat: Support of DocTags charts (serialization and deserialization) ( #229 )
...
* Doctags charts deserialization into Docling Document, added PictureTabularChartData
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Correction for chart label mapping
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* simplified label mapping
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Improved docummentation comment
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Added stacked_bar_chart to labels and tokens, as well as adjusted doctags_load
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* doctags serializer for charts
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* - Fixed DocTags table deserialization
- Updated tests
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* cleaning
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Allowing DocItemLabel.CHART as a label for PictureItem
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
---------
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com >
2025-04-07 12:27:48 +02:00
f2fe1c1bc6
feat: added initial delete and insert methods in DoclingDocument ( #220 )
...
* added intial delete and merge
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* work-in-progress for the delete option
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* working on the delete_document_item
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* reformatted code
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* verified the model-validation
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* reformatted the code
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* fixed the delete item
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* reproducible error in tests
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* before rebase with main
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the tests
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the tests
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the tests (2) and reformatted
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the iterate_items to reduce code duplication
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* mapped path method to _split_ref_to_path
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* made get_parent_ref private method
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated with new public functions
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* reformatted code
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* working on the comments of the PR
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the test
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* factored out the internal tests
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored the code and updated the tests
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* update return types, extend unit test
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* add replace_item test
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* revert unnecessary comments
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com >
2025-04-04 14:34:25 +02:00
a055e1a9f0
fix: DocTags import location fix for tables, pictures, captions ( #227 )
...
* Fix for the table with captions bboxes
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* cleaned up
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Correcting caption bbox logic for pictures
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Added test, small fix with adding caption to the picture when image not available
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* test cases (pictures + png)
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* fixed comment
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Corrected caption spans for tables and pictures, added check of captions in the tests
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
---------
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com >
2025-04-02 09:52:54 +02:00
Panos Vagenas and GitHub
2efb71a0ca
fix: allow captions without holding item ( #215 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-03-25 11:13:59 +01:00
Panos Vagenas and GitHub
ff13a9385d
feat: expose MD page break & DocTags minification ( #213 )
...
* feat: expose MD page break & DocTags minification
Also:
- aligned DocTags method naming
- fixed mutable default arguments
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* cover newly exposed params in testing
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* drop DocTags-specific post-processing
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
---------
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-03-25 10:52:09 +01:00
db119f4665
feat: Add document tokens from key value items ( #170 )
...
* add export_to_document_tokens method for KeyValueItem
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* fix export_to_document_tokens kv-item
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* update key-link representations in document tokens
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* fix key-value cell location for doctags
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* set default page_no to 1
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* fix get_location call with to_top_left_origin
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* integrate export_to_document_tokens to doctag serializer
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* Add DocTags serializer dispatching, deprecate new_line param (#212 )
* updates for key value region
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* deprecate "new_line" parameter
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
---------
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
---------
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2025-03-24 12:24:40 +01:00
Panos Vagenas and GitHub
1f4d57ed18
feat: add DocTags serializers ( #192 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-03-21 14:49:28 +01:00
Saidgurbuz and GitHub
2371c11b8f
feat: add kv_item support for doctag to docling_document ( #188 )
...
* add kv_item support for doctag to docling_document
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* use resize_by_scale to save locations
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* add kv region to tag_to_doclabel
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* add test for doctags_load_for_kv_region
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* update the naming to .dt for consistency
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
---------
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
2025-03-19 14:07:13 +01:00
Panos Vagenas and GitHub
7ed4d225b6
fix(markdown): fix ordered list numbering ( #200 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-03-18 17:20:07 +01:00
Panos Vagenas and GitHub
65a82a158c
chore: update text label for test doc creation ( #195 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-03-17 14:14:03 +01:00
Michele Dolfi and GitHub
11eb1dc5aa
fix: new favicon url ( #191 )
...
fix favicon url
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-03-17 13:16:42 +01:00
Panos Vagenas and GitHub
a7cdc87411
feat: add serializers, text formatting, update Markdown export ( #182 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-03-13 17:54:32 +01:00
c065c4c7c6
feat: Add DoclingDocument.load_from_doctags method and DocTags data models ( #187 )
...
* Adding load_from_document_tokens method to Docling document
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Added skeleton for the unit test
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Processing ordered and unordered lists
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Define new types for DocTags containers
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Implement load_from_doctags with container types
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Update tests
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* update test data
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Cleanup commented lines
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Fix for proper list groups
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
* Check for number of page doctags must be equal to page images
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
---------
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com >
Co-authored-by: Christoph Auer <cau@zurich.ibm.com >
2025-03-12 16:11:56 +01:00
Lucas Morin and GitHub
32398b8ac0
feat: add document tokens for SMILES ( #176 )
...
Signed-off-by: lum <lum@zurich.ibm.com >
2025-03-10 11:04:21 +01:00
Panos Vagenas and GitHub
53f6d099b0
fix(doctags): fix code export ( #181 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-03-04 09:41:32 +01:00
Panos Vagenas and GitHub
834db4bc66
fix(markdown): fix escaping in case of nesting ( #180 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-03-03 14:51:59 -05:00
Panos Vagenas and GitHub
349f7daa0c
fix(markdown): fix handling of ordered lists ( #175 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-02-27 16:08:05 +01:00
Panos Vagenas and GitHub
c77c59bec0
fix(markdown): fix case of leading list ( #174 )
...
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-02-27 11:10:42 +01:00
Panos Vagenas and GitHub
2abaf9b537
feat: add inline groups, revamp Markdown export incl. list groups ( #156 )
...
* feat: add inline groups, update markdown export
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* rename parameter for clarity
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* drop redundant `traverse_inline`, use only `visited`
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* update API usage after rebase
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* minor refactor
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* extend nesting logic to list groups
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* (MD export) add inline formulas, placeholders for `KeyValueItem`, `FormItem`
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* include 3rd-level list item in test
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* update test file with more recent conversion result
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
* update list construction pattern so that sublists are added to actual ListItems
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
---------
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com >
2025-02-25 09:48:10 +01:00
d622800750
feat: Introduce Key-Value and Forms items ( #158 )
...
* Draft KeyValueItem content
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* added add_key_value_item method
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* add KeyValueLink
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* update an add_key_value_item argument
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* remove KeyOrValueCellType and KeyValueLinkType
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* update tests for KeyValueItem
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* add union method to create bbox that covers all the given bboxes
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* added the , and rewrote it to make it more general with a GraphItem, as well as having cell- and link-labels
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the code due to MyPy and reformatted
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the reference docs with form-key
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated image
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* removed the circle
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* fixed the figure, testing now ...
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* added square in image
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* removed square in image
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* tests should go through
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* update poetry.lock versions
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* fix import issue
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* add validator for links in GraphData
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* added field_validator
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* rename add_key_values and add_form
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
* rename union to enclosingbbox
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch >
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
Co-authored-by: Christoph Auer <cau@zurich.ibm.com >
Co-authored-by: Peter Staar <taa@zurich.ibm.com >
2025-02-19 09:50:45 +01:00
916323fb55
feat: Redefine CodeItem as floating object with captions ( #160 )
...
* updated the CodeItem with captions
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* use FloatItem and add captions to add_code interface
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com >
2025-02-14 11:34:40 +01:00
f751b45b62
feat: Implementation of doc tags ( #138 )
...
* Migration to new DocTags, gt missing
Signed-off-by: Ahmed Nassar <ahn@zurich.ibm.com >
* fixed the tests
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* overloaded export to markdown in CodeItem to include code language
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* minor change
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* implemented page break token instead of page number
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added figure classification in doc tags
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* changed table repr. to otsl
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* changed table repr. to otsl fixed new line
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* fixed new line bug
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* fixed new line bug
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* removed print in test
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* new normalization to 500 and top left coord
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* export of FOOTER AND HEADER
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* export of FOOTER AND HEADER
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* fix code bug
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added export of standalon captions
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added export of standalon captions
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* changed generate to False
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added export of furnitures and fixed bug in iterate_items function
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
---------
Signed-off-by: Ahmed Nassar <ahn@zurich.ibm.com >
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
Co-authored-by: Ahmed Nassar <ahn@zurich.ibm.com >
Co-authored-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
2025-02-13 13:26:28 +00:00
786f0c6833
feat: Add ContentLayer attribute to designate items to body or furniture ( #148 )
...
* feat: Add ContentLayer attribute to designate items to body or furniture
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* introduce safer data gen mechanism, update chunking test data
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* Do not make test rely on order in yaml
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* chore: format fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* fix: legacy_to_docling_doc must use content_layer
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add content_layer in iterate_items
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Bump format version, add model_validator for old page_header,page_footer in body
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* fix: Change to before model_validator
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Update tests
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Address review comments
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2025-02-10 10:58:41 +01:00
Michele Dolfi and GitHub
23477f7674
feat(HTML): Fallback showing formulas as images ( #146 )
...
* remove un-needed logic
the labels allowlist is checked before
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* textitem cannot have label code
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* display formulas with mathml in exported html
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* expose argument in save_as_html
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* rename sanitize in prepare and add \n
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix mypy parsing
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* remove unused/impossible elif
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* remove strip()
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add display none for latex annotation
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fallback showing equations as image
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add html placeholder
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* markdown placeholder
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-02-03 09:29:30 +01:00
Michele Dolfi and GitHub
ed36437346
feat(HTML): Export formulas with mathml ( #144 )
...
* remove un-needed logic
the labels allowlist is checked before
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* textitem cannot have label code
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* display formulas with mathml in exported html
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* expose argument in save_as_html
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* rename sanitize in prepare and add \n
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix mypy parsing
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* remove unused/impossible elif
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* remove strip()
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add display none for latex annotation
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* make mathml the default
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* revert wrong commit
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-01-31 15:43:43 +01:00
Michele Dolfi and GitHub
c6590e83e2
fix: add html escape in md export and fix formula escapes ( #143 )
...
* add html escape in md export and fix formula escapes
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* escape also html export
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2025-01-31 11:31:35 +01:00
Panos Vagenas and GitHub
b741eeaab4
fix: fix hybrid chunker token constraint ( #131 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2025-01-17 17:22:26 +01:00
c940aa5ca9
feat: Add CodeItem as pydantic type, update export methods and APIs ( #129 )
...
* added code item
* added code item
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* added code item
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
* add constraints to allow numpy > 2.1.0 on python3.13 and others
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Add CodeItem to ContentItem
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* added CodeItem in ContentItem tagged union.
* added enum for programming languages
* removed double CodeItem in ContentItem Union
* fixed type of code_language in CodeItem class
* fixed sorting of programming languages, not sorted anymore by value of string but variable name
---------
Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Co-authored-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com >
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Christoph Auer <cau@zurich.ibm.com >
2025-01-17 16:59:33 +01:00
Michele Dolfi and GitHub
b31e0a3d05
feat: add legacy to DoclingDocument utility ( #108 )
...
* add legacy_to_docling_document
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* fix for python < 3.12
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add test
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add docs with not-preserved components
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
---------
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-12-13 13:28:41 +01:00
Panos Vagenas and GitHub
047a1960af
fix: improve doc item typing ( #105 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-12-13 12:51:53 +01:00
Panos Vagenas and GitHub
b546c0a50d
fix: set origin when merging chunks ( #109 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-12-13 09:05:56 +01:00
Panos Vagenas and GitHub
2591c70c66
feat: add default tokenizer to HybridChunker ( #107 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-12-12 16:51:08 +01:00
437c498f77
feat: utilities converting document formats ( #91 )
...
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Christoph Auer <cau@zurich.ibm.com >
2024-12-09 14:14:01 +01:00
Boyce and GitHub
fd7529f409
fix(markdown): preserve underscores in image URLs during markdown export ( #98 )
...
Signed-off-by: Boyce <aboyskyi@gmail.com >
2024-12-09 11:38:23 +01:00
628ab679cb
feat: add hybrid chunker ( #68 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Bill Murdock <bmurdock@redhat.com >
Co-authored-by: Ben Rood <bash99@gmail.com >
2024-12-06 14:56:10 +01:00
Cesar Berrospi Ramis and GitHub
266e7fc603
build: set pydantic version to 2.10.3 ( #93 )
...
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
2024-12-04 14:26:22 +01:00
8422ad4fbf
fix: Even better style for HTML export ( #78 )
...
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com >
2024-11-28 16:54:01 +01:00
a94ba273b1
chore: Improving styling for HTML export ( #77 )
...
Signed-off-by: Maksym Lysak <mly@zurich.ibm.com >
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com >
2024-11-27 20:19:43 +01:00
ef49fd3f34
feat: adding HTML export to DoclingDocument, adding export of images in png with links to Markdown & HTML ( #69 )
...
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Christoph Auer <cau@zurich.ibm.com >
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com >
2024-11-27 05:27:23 +01:00
Panos Vagenas and GitHub
bfeb2db24b
fix: include titles to chunk heading metadata ( #62 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-11-01 13:41:16 +01:00
Panos Vagenas and GitHub
d09fe7ed44
feat: extend chunk meta with schema, version, origin ( #49 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-22 11:08:04 +02:00
03df97fa73
feat!: Expose DoclingDocument as main type, move old typing to legacy ( #41 )
...
* Fix area method of BoundingBox
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* add image placeholder
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* enable picture label
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* refactor captions and markdown
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add logic to skip repeated caption
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* use DocItemLabel
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Extend default export labels, add convenience mehtods
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Introduce ListItem API, with marker and enumerated properties
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* add classification and description in PictureData
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* add molecule picture data
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Fixes for DoclingDocument and aligned methods on legacy doc
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* add advanced picture data content
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Many markdown export fixes, renaming BaseTableData
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Rename module paths doc->legacy_doc, experimental->doc
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* feat: imageref with pil_image
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Small fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* docs: remove documentation in markdown to support python 3.13 (#43 )
Since json-schema-for-humans dependency does not support python 3.13,
remove the generation of documentation in markdown of main docling types.
Remove 'ds' prefix from documentation scripts.
Update README.
Add python 3.13 in CI/CD workflow checks.
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* Fix TableCell model validator
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* store list of classes in classification
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* Fixes for DocumentOrigin mimetype validation
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* introduce picturedata as list of annotations
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
* feat: adapt hierarchical chunker to v2 DoclingDocument
[skip-ci]
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* feat: add table support in chunker, incl. captions
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* use Field constraints instead of conlist, refactor chunking types
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* revert unnecessary doc module change
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* align test data with upstream changes
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
* Update __init__.py on docling_core.types.doc
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Remove DescriptionItem
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com >
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-16 15:53:44 +02:00
Panos Vagenas and GitHub
b5592ad747
fix: align chunk ref format with one used in Document ( #37 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-07 09:07:08 +02:00
688789ea75
feat: (experimental) introduce new document format ( #21 )
...
* Draft new docling document format, pydantic model and tests
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Fix tests to have unique document_hashes per test
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Manual update from main
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Move new-format to experimental path
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Updates for document construction API and format
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add comments
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add BaseTableData and table cell typing
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Tree element iterator, several API fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Turn captions into list field
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add export methods to DoclingDocument and types
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Change DoclingDocument.iterate_elements and add print tree function
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Introduce label enum types, apply everywhere
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Introduce provenance info, use enum labels
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Update formatting
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Docstrings and linter fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Lockfile rollback, since updating breaks tests
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Cleanup
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Several improvements and cleanup
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Format fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Big redesign for usage of hashes, several other fixes
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Fix flake8 config
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Remove hash, renamings
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* updating the tests
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* added some unit tests for DocItem
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* Add tree validation API and test case
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Add extra=Forbid to NodeItem
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* feat: set DoclingDocument version as SemanticVersion with default
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* Cleanup
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Simpler literal enforcement
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Fix static document version
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* Rollback changes to allow for semver<3
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
* build: rollback changes to include python-semantic-release as dev dependency
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* feat: set version field as string with pattern and check compatibility
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
* add JSON Pointer validation to refs, fix test data
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
---------
Signed-off-by: Christoph Auer <cau@zurich.ibm.com >
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
Co-authored-by: Peter Staar <taa@zurich.ibm.com >
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com >
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-10-01 09:41:16 +02:00
Panos Vagenas and GitHub
4bde51528d
feat: support heading as chunk metadata ( #36 )
...
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com >
2024-09-30 11:08:09 +02:00
Michele Dolfi and GitHub
a334b9fc72
fix: remove duplicate captions in markdown ( #31 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-09-24 14:33:56 +02:00
Michele Dolfi and GitHub
b843ae6688
feat: add figures in markdown export ( #27 )
...
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com >
2024-09-23 20:12:06 +02:00
Peter W. J. Staar and GitHub
891530f595
feat: add export to doctags for document components ( #25 )
...
* feat: refactoring doc-tokens in new file and add new export function to table
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* reformatted and fixed bugs
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* working on table exporting in document tokens
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated code and tests, need to decide what to commit as test-cases
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* updated the test cases
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored the test-files
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored duplicate code
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored duplicate code (boundingbox to figure and table)
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored duplicate code (making BaseCell the base-class of BaseText)
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* refactored duplicate code (re-use of function in BaseCell)
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
* fixed issues from review (2)
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
---------
Signed-off-by: Peter Staar <taa@zurich.ibm.com >
2024-09-20 14:16:33 +02:00
Cesar Berrospi Ramis
Panos VagenasandGitHub
Michele DolfiandGitHub
Lucas MorinandGitHub
BoyceandGitHub
