mirror of
https://github.com/docling-project/docling-core.git
synced 2026-11-18 20:27:00 +00:00
* fix(webvtt): resolve every HTML character reference in cue text The cue text and annotation validators accepted only the six named references of the WebVTT syntax table and dropped the whole cue on anything else, such as ' or é. The tokenizer (WebVTT 6.4) hands both states to the HTML character reference algorithm, so read with html.unescape and keep a reference that does not resolve as literal text. Serialization still writes only the escapes it needs. A reference that resolves to a line terminator is still rejected, because the cue would be written back across two lines. Refs #919 Signed-off-by: Christopher Krah <48338417+chrikrah@users.noreply.github.com> * chore(webvtt): clarify docstring and remove '_unescape' wrapper Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com> * fix(webvtt): keep exported document text literal The WebVTT serializer embeds item text and voice names into cue markup and parses it back, so any character reference in the document text, such as © or &, was resolved and the exported text changed. Escape both before they reach the cue parser. Share the unescaping of cue text and annotations in one helper and keep only the escapes that are written back. Signed-off-by: Panos Vagenas <pva@zurich.ibm.com> --------- Signed-off-by: Christopher Krah <48338417+chrikrah@users.noreply.github.com> Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com> Signed-off-by: Panos Vagenas <pva@zurich.ibm.com> Co-authored-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com> Co-authored-by: Panos Vagenas <pva@zurich.ibm.com>