- Add 300s timeout to calibredb calls to prevent hanging on large libraries
- Improve JSON parsing with validation for empty/malformed data
- Add case-insensitive format matching for mixed-case extensions
- Validate file existence before adding to conversion queue
- Enhanced convert_ignored_formats handling with robust type checking
- Add comprehensive verbose logging for better debugging
- Protect against malformed file paths and unicode issues
- Graceful handling of edge cases (empty configs, missing attributes)
Implements ability to keep original book formats after conversion to target format.
Users can now select which formats to retain via CWA settings UI.
Features:
- New auto_convert_retained_formats setting with checkbox grid UI
- Automatic conflict prevention (target format always retained)
- Database migration support for backward compatibility
- Enhanced ingest processor with robust format addition logic
Credit to @angelicadvocate for original implementation concept in PR #284.
Fixes edge cases including race conditions, UI state handling, and iteration safety.
- Add null/empty path checks in gdriveutils.py to prevent TypeError on import
- Enhance exception handling in ingest_processor.py to catch TypeError/AttributeError
- Add session guards to all database functions for graceful degradation
- Ensure ingest service continues working when Google Drive is disabled
[bug] Book not ingested on dev image (main branch)
Fixes#621
Fix issue where books in different languages were incorrectly grouped as duplicates
by implementing a comprehensive configurable duplicate detection system.
Key Changes:
Database Schema:
- Add 6 new duplicate detection settings to cwa_schema.sql:
- duplicate_detection_title/author/language (default: enabled)
- duplicate_detection_series/publisher/format (default: disabled)
Frontend UI:
- Add "CWA Duplicate Detection Criteria" section to cwa_settings.html
- Implement checkbox grid for configuring detection criteria
- Include explanatory text and validation warnings
Core Logic Rewrite:
- Replace hardcoded (title, author) matching with configurable criteria
- Support dynamic key generation based on selected metadata fields
- Add comprehensive error handling and edge case coverage
Robustness Improvements:
- Handle missing/null metadata gracefully with fallback values
- Add safety checks for empty collections and corrupt data
- Include CWA database connection error handling
- Performance warnings for large libraries (50k+ books)
Issue Resolution:
- Books in different languages no longer considered duplicates (language included by default)
- Users can now fully customize duplicate detection criteria
- Maintains backward compatibility with existing duplicate manager
- Comprehensive error handling prevents crashes on edge cases
Technical Details:
- Follows established CWA settings patterns for seamless integration
- Boolean settings automatically handled by existing backend logic
- Added datetime import for timestamp sorting fallbacks
- Extensive null/empty validation throughout duplicate detection pipeline
Fixes#614
The converter now reliably detects if a book already exists in a target
format by getting the list of existing book format filenames directly
from the Calibre database rather than making an assumption on the
format filename.
## Major Features Added
### 📧 Auto-Send System
- Automatically emails newly ingested books to users' eReaders
- Configurable delay (1-60 minutes) to allow for processing
- Supports multiple formats (EPUB, MOBI, AZW3, KEPUB, PDF)
- Integrates with existing Calibre-Web email configuration
- Respects user preferences and access controls
### 🏷️ Auto-Metadata Fetch System Enhancements
- Enhanced metadata fetching with multiple provider support
- Added smart metadata application mode with intelligent criteria
- Moved control from user-level to admin-only configuration
- Implemented provider hierarchy with drag-and-drop interface
- Added quality-based metadata replacement logic
## Database Schema Changes
### CWA Settings (scripts/cwa_schema.sql)
- Added auto_metadata_smart_application SMALLINT DEFAULT 0
- Enables intelligent vs direct metadata replacement modes
## User Interface Updates
### Admin Interface (cps/templates/cwa_settings.html)
- Added smart metadata application toggle with detailed tooltip
- Enhanced provider hierarchy management
### User Interface (cps/templates/user_edit.html)
- Removed auto_metadata_fetch controls (now admin-only)
- Cleaned up user profile interface
## Smart Metadata Application Logic
### Direct Replacement Mode (Default)
- Takes metadata from preferred provider exactly as provided
- Complete replacement of existing metadata
- Philosophy: "Just take the metadata as is"
### Smart Application Mode (Optional)
- Intelligent criteria for metadata replacement:
* Titles: Only replace if longer/more descriptive
* Descriptions: Only replace if longer/more detailed
* Publishers: Only replace if current field is empty
* Covers: Only replace if higher resolution
* Authors: Always update for consistency
* Tags/Series: Always add for discoverability
## Technical Implementation
### Metadata Helper (cps/metadata_helper.py)
- Enhanced _apply_metadata_to_book() with smart application logic
- Updated fetch_and_apply_metadata() for admin-only control
- Integrated CWA_DB settings checking for both modes
### Ingest Processor (scripts/ingest_processor.py)
- Removed user-based metadata checking
- Streamlined to use admin settings only
- Improved processing pipeline integration
### Form Processing (cps/cwa_functions.py)
- Auto-detection of boolean settings from schema
- Automatic handling of auto_metadata_smart_application
## Provider System Enhancements
- Google Books, Internet Archive, DNB, ComicVine, Douban support
- Priority-based searching with first-success-wins logic
- Quality criteria evaluation for metadata selection
- Configurable provider hierarchy with drag-and-drop interface
## Documentation
### Wiki Pages Created
- Auto-Send-System.md: Comprehensive user and admin guide
- Auto-Metadata-Fetch-System.md: Detailed configuration and usage
- Enhanced with relevant emojis for improved readability
- Covers troubleshooting, best practices, and technical details
## Integration & Compatibility
- Maintains backward compatibility with existing email settings
- Integrates seamlessly with auto-convert and ingest systems
- Respects existing access controls and user permissions
- No breaking changes to existing functionality
## Testing Notes
- Database schema updates will apply automatically on app startup
- Settings form processing handles new boolean field automatically
- Metadata fetching now controlled entirely by admin settings
- User interface cleaned of deprecated metadata controls
This implementation provides a complete automated book delivery and metadata enhancement system while maintaining the principle of admin-controlled automation and user-friendly operation.
- Instead of writing directly into the library, uploads are saved safely into the ingest folder and a background worker imports them (prevents import of incomplete files)
- Files are first saved with a temporary suffix and then atomically renamed to avoid partial/corrupt reads.
To prevent dupes & handle uploading a new format to an existing book:
- Added a simple “sidecar” instruction file next to the upload so the ingest worker knows to attach that file to the correct book (no duplicate books).
Maintain Google Drive sync:
- After a successful import or add‑format, we trigger the same GDrive sync used after metadata edits (only if GDrive enabled).
Added a little UX polish:
- Invalid new uploads now redirect to the home page; empty upload posts return “400 Bad Request.”
- Polling will also be used when NETWORK_SHARE_MODE is enabled as it has better reliability with shares and using polling over inotify can be activated manually by passing the CWA_WATCH_MODE: "poll" env variable
- If the user reaches max_user_watches limit, the cwa-ingest-service & metadata-change-detector processes will now dynamically fallback to polling
[bug] Synology - hitting inotify.max_user_watches limit
Fixes#311
[Feature Request] Skip actions on read-only data
Fixes#229
- Added connection timeouts everywhere:
SQLAlchemy engines: connect_args={'timeout': 30}
sqlite3 direct connects: timeout=30 across scripts
- Enabled WAL mode on local disks and made it conditional via an environment flag.
-Added small retry/backoff in read-only UI probes to tolerate transient “database is locked.”
Increased Network share support (NFS/SMB):
- Introduced NETWORK_SHARE_MODE to disable WAL and suppress permission-changing operations that can break on network filesystems.
- Suppressed recursive chown when NETWORK_SHARE_MODE is true:
Python:
scripts/auto_library.py: skip chown for /config and /calibre-library during app.db/new library setup
scripts/ingest_processor.py: skip chown of library in set_library_permissions
scripts/convert_library.py: skip chown of convert-library.log; skip library chown
scripts/kindle_epub_fixer.py: skip chown of epub-fixer.log
cps/updater.py: disable os.chown during updates when flag is enabled
Shell:
scripts/setup-cwa.sh: guard chown of /etc/s6-overlay
root/etc/s6-overlay/s6-rc.d/cwa-init/run: guard initial chown of /config and cps/cache; guard requiredDirs loop
Documentation and configuration:
- README: Added “Network shares and SQLite WAL mode” section; documented that NETWORK_SHARE_MODE disables WAL and chown behavior for safety on network shares.
- docker-compose.yml and docker-compose.yml.dev: Exposed NETWORK_SHARE_MODE (default false) in environment.
Fixed a minor YAML indentation issue during compose edits.
Impacts on the Project:
- Fewer “database is locked” errors due to timeouts, gentle retries, and WAL on local disks.
- Safer behavior on NFS/SMB: WAL disabled and chown suppressed when NETWORK_SHARE_MODE=true.
- Clearer guidance for users via README and compose templates.
[bug] chown failed
Fixes#175
Calibre-Web-automated [bug]
Fixes#530
Fixes#481
Problem: When editing book metadata (either in bulk, inline, or through the full-page editor), the function to update the book's directory structure on the filesystem (update_dir_structure) was called multiple times in quick succession or for each book in a selection. This created a race condition, leading to errors and inconsistent folder structures.
Solution: The editing functions in editbooks.py (edit_selected_books, edit_book_param, and do_edit_book) were refactored. The new workflow ensures that all database-level metadata changes are collected and staged first. Only after all changes are ready is the filesystem update function called a single time. This atomic operation prevents race conditions and ensures the directory structure accurately reflects the final state of the metadata.
[bug] Failing to update metadata due to filepath mismatch
Fixes#519
Problem: The cover_enforcer.py script, which processes metadata change logs, failed to correctly parse log entries for books with multiple authors (e.g., "Author One & Author Two"). It would use the entire string as the author's name, leading to incorrect directory paths and "File Not Found" errors.
Solution: The script was modified to correctly handle multi-author strings. It now splits the author field by the " & " delimiter and uses only the first author to construct the directory path, mirroring Calibre-Web's default behavior.
crocodilestick
Joseph Larionov
Bruno Finger
Emmanuel Ferdman
Aymen Djellal
root