Commit Graph
33 Commits
Author SHA1 Message Date
COTHSC 864036d545 feat: add DOCLINGCORE_ALLOWED_PRIVATE_IPS setting to allow private IPs (#608)
* feat: add DOCLINGCORE_ALLOWED_PRIVATE_IPS setting to allow private IP access

Signed-off-by: COTHSC <thierryscully.colomban@gmail.com>

* fix: address review feedback on private-IP allowlist

fix broken response in test_allowed_private_ips (use _closeable_response instead of Response)
document allowed_private_ips
widen _ip_in_allowlist typehint to include ipv6 addresses + add docstring
clarify the malformed entry logmessage to hint at strict CIDR parsing

Signed-off-by: COTHSC <thierryscully.colomban@gmail.com>

* fix: use strict=False with normalization warning for CIDR allowlist entries

Signed-off-by: COTHSC <thierryscully.colomban@gmail.com>

* style: apply ruff-format

Signed-off-by: COTHSC <thierryscully.colomban@gmail.com>

---------

Signed-off-by: COTHSC <thierryscully.colomban@gmail.com>
2026-09-10 14:04:28 +02:00
Cesar Berrospi Ramis 0c46294fac style: modernize type hints with PEP 604 + dependency upgrades (#737)
* style: modernize type hints with PEP 604

Enforce Ruff UP007 and UP045 to align with PEP 604 (union types as X | Y).

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore: upgrade 'uv.lock' dependencies

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-09-04 06:31:04 +02:00
Panos Vagenas aee0674a28 fix(doclang): add deserialize size, depth, and element budgets (#683)
- Cap DocLang XML input bytes before `parseString`
- Bound nesting depth and element count after DOM construction
- Expose limits via settings / `deserialize_str` kwargs and cover with unit tests

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-07-14 15:49:17 +02:00
Peter W. J. Staar f8a36e9a10 feat: split the monolithic document.py into focused modules (#664)
refactored the docling-core for the document.py

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-07-09 14:39:46 +02:00
NintoracandCesar Berrospi Ramis be42fcea2e feat: add fsspec support (#444)
* feat: add universal-pathlib support for document to_*/from_* methods

Enable cloud storage backends (S3, GCS, Azure, etc.) for document
serialization methods via fsspec/universal-pathlib integration.

Signed-off-by: Nintorac <Nintorac@users.noreply.github.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* test(fsspec): remove universal-pathlib check

Remove the import check for universal-pathlib library and the decorators
'@pytest.mark.skipif' since it is unnecessary and adds complexity.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore: remove potentially misleading comment

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* fix(types): convert image URI path to AnyUrl for type safety

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* test(upath): add error handling for remote paths in 'relative_path()'

Catch error in case 'relative_path()' is used with remote paths and 'resolve' fails.
Consolidate the tests in test_upath_support.py module.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* fix: misplaced block, annotations, tests

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor: deduplicate code in '_with_pictures_refs'

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Nintorac <Nintorac@users.noreply.github.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-07-02 16:36:19 +02:00
Christoph Auer 246f131aff fix: Add legacy stubs to unbreak old docling version imports (#649)
Add legacy shims to unbreak old docling version imports

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
2026-06-19 07:41:21 +02:00
Peter W. J. Staar 8951e6f202 chore: removing all legacy / deepsearch code (#644)
* What was removed

  The legacy_doc document model and the rec (Record) types, plus all code/resources/tests that existed only to serve them.

  Deleted packages/modules
  - docling_core/types/legacy_doc/ (whole folder)
  - docling_core/types/rec/ (whole folder)
  - docling_core/utils/legacy.py (legacy conversion functions)
  - docling_core/utils/validate.py (RAW/ANN/OCR CLI)

  Edited production code
  - types/__init__.py — dropped the Record export
  - utils/validators.py — removed validate_schema/validate_raw_schema/validate_ann_schema/validate_ocr_schema and their now-unused imports; kept
  validate_unique_list/validate_datetime
  - utils/generate_docs.py — dropped Record from MODELS

  Removed resources/artifacts
  - resources/schemas/doc/ (legacy RAW/ANN/OCR/DOC), resources/schemas/generated/ccs_document_schema.json, docs/Record.json

  Tests — deleted the 5 legacy/rec-only suites + 2 stale gold files; trimmed test_collection.py, test_base.py, test_doc_base.py to keep their non-legacy parts;
  replaced the _create_hash import in test_code_chunker.py with a local helper. Removed test/data/legacy_doc/ and test/data/rec/.

  Verification

  - No dangling code references remain (only string literals and the vendored _reference/chunker_repo scratch data, intentionally untouched).
  - from docling_core.types import DoclingDocument, Generic works; Record now correctly raises ImportError.
  - Full project suite: 496 passed.
  - generate_docs.py now emits only DoclingDocument.json and Generic.json.

  The full plan is saved at _plans/remove-legacy-doc-and-rec.md.

  Two notes worth flagging:
  - Breaking public API: docling_core.types.Record and the docling_core.utils.legacy conversion functions were public — downstream docling uses
  legacy_to_docling_document/docling_document_to_legacy. The branch name (dev/remove-legacy-document) suggests this is intended, but it'll need a coordinated
  bump downstream.
  - I left _reference/ and _plans/ alone (untracked scratch dirs). Nothing was committed — all changes are staged/working-tree only, per your edit-by-edit
  approval flow.

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* updated the checks.yaml

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed docling_core/types/gen/, docling_core/types/nlp, docling_core/types/legacy_doc, docling_core/types/rec

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed the minimal_document_schema_flat.json

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed docling_core/resources

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* removed the files and updated the imports according to Cesar's suggestion

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fixed the ruff linter

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2026-06-18 17:00:52 +02:00
Christoph Auer d585a13775 feat: Make HTTP downloads respect file-size limits and abort early (#635)
* Early-abort HTTP source fetch when it is too large

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Address review comments

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
2026-06-10 13:38:06 +02:00
Pragnya Khandelwal 701c0ce5d1 fix: reject unsupported URL schemes (#604)
* Reject unsupported URL schemes

Signed-off-by: Pragnya <prag1704@gmail.com>

* Remove PR template

Signed-off-by: Pragnya <prag1704@gmail.com>

* fix: satisfy lint checks for unsupported url scheme

Signed-off-by: Pragnya <prag1704@gmail.com>

---------

Signed-off-by: Pragnya <prag1704@gmail.com>
2026-06-03 12:40:16 +02:00
Panos Vagenas 2087d0f362 fix: refine ImageRef URI handling (#595)
* fix: refine ImageRef URI handling

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* make URI handling configurable via settings

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-04-22 16:13:10 +02:00
Panos Vagenas 473fbacfb9 fix: refine remote filename handling (#591)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2026-04-21 16:37:04 +02:00
Cesar Berrospi Ramis c8f3c01a61 feat: model and serializer for audio tracks (#426)
* refactor: move WebVTT data model from docling

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* fix(webvtt): deal with HTML entities in cue text spans

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(webvtt): support more WebVTT models

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(DoclingDocument): create a new provenance model for media file types

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(webvtt): make WebVTTTimestamp public

Since WebVTTTimestamp is used in DoclingDocument, the class should be public.
Strengthen validation of cue language start tag annotation.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(webvtt): set languages to a list of strings in ProvenanceTrack

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* tests(webvtt): add test for ProvenanceTrack

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(webvtt): make all WebVTT classes public for reuse

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore(webvtt): preserve newlines as WebVTTLineTerminator

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(webvtt): set ProvenanceTrack time fields as float

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore(webvtt): ensure start time offsets are in sequence

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore(webvtt): improve regex to remove note,region,style blocks

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore(webvtt): parse the WebVTT file title

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore(webvtt): rebase to latest changes in idoctags

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* feat(webvtt): add WebVTT serializer

Add a DoclingDocument serializer to WebVTT format.
Improve WebVTT data model.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* fix(webvtt): add 'text/vtt' as extra mimetype

Add 'text/vtt' as extra MIME type to support WebVTT serialization, since it is not
supported by 'mimetypes' with python < 3.11

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(webvtt): roll back DocItem.prov as list of ProvenanceItem

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* tests(webvtt): fix test with STYLE and NOTE blocks

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* style(webvtt): apply X | Y annotation instead of Optional, Union

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(webvtt): simplify TrackProvenance model with tags

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* refactor(webvtt): align class and field names to new 'source' type

Classes and fields that are related to the new source type should aign with their names.
The term 'provenance' will identify the legacy implementation.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* chore(DoclingDocument): drop the validation on field assignment

Drop the validation on field assignment in NodeItem objects.
Add the 'source' argument in the convenient function 'add_text' to create TextItem with track source data.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(webvtt): drop cue span classes, 'lang' and 'c' tags

Drop WebVTT formatting features not covered by Docling across formats.
Only 'u', 'b', 'i', and 'v' are supported and without classes.
Make 'v' tag explicit as 'voice' feature in SourceTrack class.

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-01-27 14:45:03 +01:00
Michele Dolfi 01373c4aab fix: drop python 3.9 and pin built tree-sitter versions (#487)
* drop python 3.9

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* apply linter for python 3.10 changes

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
2026-01-22 19:40:35 +01:00
c73904e68e style: replace black, isort, flake8 and autoflake with ruff (#456)
* Added ruff to dev dependencies

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Added ruff settings to pyproject.toml as in docling

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Cleanup uf pyproject.toml

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Copied settings for ruff pre-commit hooks from docling

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Excluded test/data/** from ruff formatting / linting

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* ruff format

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Added some ignore statements to pyproject.toml such that ruff check raises fewer issues

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* ruff check --fix

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Ignored some more rules

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Fixed the rest of the errors that would only concern 1 - 3 files

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Added another ignore related to df for DataFrame names

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Modified CONTRIBUTING.md such that black / isort are replaced by ruff

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Added UP045 to ignore list such that Optional[...] does not raise

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Moved .flake8 configs to pyproject.toml

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Moved autoflake to be used with ruff

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Moved all .flake8 settings to pyproject.toml to be compatible with ruff (i.e. no separate [tool.flake8] section

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Removed flake8 from .pre-commit hooks

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Applied ruff format (again); formatted some files as the line-length = 120 equals now what was set for the .flake8 settings

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Set max-complexity to 30 (as was originally) in the pyproject.toml as one linting check would fail

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Adding PD901 to ignore list such that pre-commit hooks run fully again

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* Replaced dtype | None syntax by Optional[dtype] in remaining places

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>

* chore: fix 'test' ref in pyproject

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* style: remove typing List, Set, Tuple, Dict

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* style: remove UP015 check from ignore list

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* style: remove UP034 check from ignore list

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* style: normalize dashes in comments and docstrings

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* style: remove PD901 check from ignore list

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* style: remove C403 check from ignore list

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* style: remove C403, C413, C416 check from ignore list

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

* style: remove E203, F811 check from ignore list

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

---------

Signed-off-by: Florian Schwarb <florian.schwarb@gmail.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Co-authored-by: Florian Schwarb <florian.schwarb@gmail.com>
Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
2026-01-13 17:03:10 +01:00
Peter W. J. Staar 928aaed9ab chore: remove unnecesary copyright headers (#440)
* chore: remove unnecesary copyright headers

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* reformatted, still need to fix a test ...

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
2025-11-27 11:17:23 +01:00
Panos Vagenas 1d04154378 feat: add rich table cells (#368)
* feat: add rich table cells

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* propagate cell text resolution, cover row deletions

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add doctags, fix referential integrity, expand tests, reenable mypy

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* bump DoclingDocument version

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* simplify / remove serialize_cell

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* update rich table cell refs in doc indexing

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* update notebook

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* expose new classes in `docling_core.types.doc`

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-08-27 10:07:47 +02:00
Panos Vagenas 14a4fdee87 feat: remodel lists, add MD & HTML ser. params, enable unset marker (#339)
* feat: remodel lists

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* prepare test document separate markers

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add MD/HTML serializer options, expand test data

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* create list groups where not in place

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add auto-increment logic

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* restore UnorderedList as deprecated alias (for backwards compatibility)

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* enable unset marker case

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* make ordered markers able to be "unset" (empty) too

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* rename default marker mode to auto

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-06-27 14:52:25 +02:00
Michele Dolfiandvtempest 3eeb2596c6 feat: Download Google docs and drive files via export url (#335)
* # Google Docs, Files, PDF URLs, Spreadsheets, Presentations: convert to export URL

Signed-off-by: vtempest <1274452+vtempest@users.noreply.github.com>

* standard way of exporting ?format=

Signed-off-by: vtempest <1274452+vtempest@users.noreply.github.com>

---------

Signed-off-by: vtempest <1274452+vtempest@users.noreply.github.com>
Co-authored-by: vtempest <1274452+vtempest@users.noreply.github.com>
2025-06-27 14:48:17 +02:00
Michele Dolfi 4b967ab55c fix: hashlib usage for FIPS (#280)
fix usage of hashlib for FIPS

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
2025-05-02 15:00:02 +02:00
Christoph AuerandPanos Vagenas 786f0c6833 feat: Add ContentLayer attribute to designate items to body or furniture (#148)
* feat: Add ContentLayer attribute to designate items to body or furniture

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* introduce safer data gen mechanism, update chunking test data

Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>

* Do not make test rely on order in yaml

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* chore: format fixes

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* fix: legacy_to_docling_doc must use content_layer

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Add content_layer in iterate_items

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Bump format version, add model_validator for old page_header,page_footer in body

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* fix: Change to before model_validator

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Update tests

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Address review comments

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
2025-02-10 10:58:41 +01:00
Michele Dolfi 268c294cc9 fix: always write with utf8 encoding (#111)
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
2024-12-17 09:37:01 +01:00
Michele Dolfi b31e0a3d05 feat: add legacy to DoclingDocument utility (#108)
* add legacy_to_docling_document

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* fix for python < 3.12

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* add test

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* add docs with not-preserved components

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
2024-12-13 13:28:41 +01:00
Michele DolfiandChristoph Auer 437c498f77 feat: utilities converting document formats (#91)
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Christoph Auer <cau@zurich.ibm.com>
2024-12-09 14:14:01 +01:00
Panos Vagenas 63e6c01863 fix: fix circular import (#87)
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
2024-12-02 16:04:57 +01:00
Michele DolfiandPanos Vagenas 9a74d13fd6 feat: extend source resolution with streams and workdir (#79)
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
Signed-off-by: Michele Dolfi <97102151+dolfim-ibm@users.noreply.github.com>
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
2024-12-02 14:12:55 +01:00
ef49fd3f34 feat: adding HTML export to DoclingDocument, adding export of images in png with links to Markdown & HTML (#69)
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Christoph Auer <cau@zurich.ibm.com>
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com>
2024-11-27 05:27:23 +01:00
Peter W. J. StaarandMichele Dolfi 44941b5591 feat: add headers argument and a custom user-agents for http requests (#53)
* fix: added custom-headers with user-agents for requests

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* updated the logic to ensure we always have headers

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* use lower case headers and change to docling-core agent name

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* change default

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com>
2024-10-24 13:46:54 +02:00
Panos Vagenas 2c88e56f8c fix: fix resolution in case of URL without path (#55)
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
2024-10-24 13:07:59 +02:00
Panos Vagenas e12d6a70c3 fix: fix legacy doc ref (#48)
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
2024-10-18 15:21:01 +02:00
03df97fa73 feat!: Expose DoclingDocument as main type, move old typing to legacy (#41)
* Fix area method of BoundingBox

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* add image placeholder

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* enable picture label

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* refactor captions and markdown

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* add logic to skip repeated caption

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* use DocItemLabel

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* Extend default export labels, add convenience mehtods

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Introduce ListItem API, with marker and enumerated properties

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* add classification and description in PictureData

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* add molecule picture data

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* Fixes for DoclingDocument and aligned methods on legacy doc

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* add advanced picture data content

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* Many markdown export fixes, renaming BaseTableData

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Rename module paths doc->legacy_doc, experimental->doc

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* feat: imageref with pil_image

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* Small fixes

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* docs: remove documentation in markdown to support python 3.13 (#43)

Since json-schema-for-humans dependency does not support python 3.13,
remove the generation of documentation in markdown of main docling types.
Remove 'ds' prefix from documentation scripts.
Update README.
Add python 3.13 in CI/CD workflow checks.

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

* Fix TableCell model validator

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* store list of classes in classification

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* Fixes for DocumentOrigin mimetype validation

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* introduce picturedata as list of annotations

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* feat: adapt hierarchical chunker to v2 DoclingDocument

[skip-ci]

Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>

* feat: add table support in chunker, incl. captions

Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>

* use Field constraints instead of conlist, refactor chunking types

Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>

* revert unnecessary doc module change

Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>

* align test data with upstream changes

Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>

* Update __init__.py on docling_core.types.doc

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Remove DescriptionItem

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
Co-authored-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
2024-10-16 15:53:44 +02:00
Panos Vagenas 752cbc3e89 feat: add file source resolution utility (#22)
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
2024-09-20 06:06:43 +02:00
Cesar Berrospi Ramis 656f56380f fix: fix definition issues in record type (#2)
* fix(test): set static typing compatible to python 3.10

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

* style: enforce some pre-commit hooks on tests

Enforce pre-commit hooks black, isort, autoflake, and mypy on test modules.

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

* fix(rec): fix definition issues in attribute, predicate, subject

Remove duplicate generic types across base and predicate modules.
Create an identifier class for subject names.
Remove unnecessary type variables in attribute model.

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

* docs: refer to Docling data objects

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

---------

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
2024-07-17 17:24:31 +02:00
Cesar Berrospi Ramis 9134ffee93 Initial commit
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
2024-07-15 12:55:20 +02:00