Commit Graph
8 Commits
Author SHA1 Message Date
Panos VagenasandGitHub d8a5256b2c feat: add table annotations (#304)
* feat: add table annotations

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* refactor annotation types

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* expand to HTML

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* introduce annotation serializer

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* Update dummy_doc.yaml

Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
2025-06-05 15:38:46 +02:00
Panos VagenasandGitHub 8415969608 chore: exclude test data from GH Linguist (#309)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-05-28 14:06:22 +02:00
f067c51c48 feat: add annotations in MD & HTML serialization (#295)
* feat: include annotations in MD & HTML serialization

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* (HTML) move annotations into figcaptions

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* add explicit beginning/end markers, fix case of excluded refs

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* improve annotation marking, extend tests

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* wrap captions (#305)

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* revert temp test changes

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com>
2025-05-27 10:49:13 +02:00
Panos VagenasandGitHub d05fe08546 feat: promote serializers to stable API (#276)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-04-30 11:26:39 +02:00
EugeneandGitHub c30ada64d6 chore: Types fixes (#267)
* docs: Recommend installing all groups for mypy to work properly

Signed-off-by: Eugene <fogaprod@gmail.com>

* chore: Fix some incomplete type defs

Signed-off-by: Eugene <fogaprod@gmail.com>

---------

Signed-off-by: Eugene <fogaprod@gmail.com>
2025-04-22 10:29:20 +02:00
763e1364ff feat: Add tiktoken tokenizers support to HybridChunker (#240)
* feat: Add tiktoken tokenizers support to HybridChunker

Signed-off-by: ruizguille <guillermo@codeawake.com>

* separate OpenAI tokenizer

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: ruizguille <guillermo@codeawake.com>
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
Co-authored-by: Panos Vagenas <pva@zurich.ibm.com>
2025-04-18 07:43:20 +02:00
Panos VagenasandGitHub 23036e17fa feat: add serializer provider to chunkers (#239)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-04-09 12:45:44 +02:00
Panos VagenasandGitHub 5e4c0fd51f feat: integrate serialization API into chunkers (#221)
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
2025-04-08 14:04:40 +02:00