Example plugins - API fetcher and check stats, watch list notes hook, update_finalize fix (#4622)

* Example plugins - API fetcher and check stats examples, watch list notes hook, update_finalize fix

New hook watch_list_extra_text(watch) (ported from browser-ui): short plain-text notes under each watch
in the watch list, escaped when rendered.

New example plugins, each `pip install .`-able and importing only from plugin_api:
- fetcher/api_fetcher: register_content_fetcher, plugin_settings_tab, fetcher_status_icon, plugin_static_path
- lifecycle/check_stats: update_handler_alter, update_finalize, watch_list_extra_text, ui_edit_stats_extras,
  get_html_head_extras, plugin_static_path
- processor/wordcounter: also shows "Found: N words" in the watch list via watch_list_extra_text

Fixes found by the examples:
- update_finalize always got update_handler=None after a successful check (the worker deleted the handler
  before the finally block captured it), so update_handler_alter + update_finalize couldn't be paired
- The API rejected plugin fetchers in 'fetch_backend' (hardcoded pattern), now built from installed fetchers
- fetcher_status_icon docstring said it returns HTML, it returns a dict

plugin_api 0.3.0: exports Fetcher and load_plugin_settings.
Tests check each example end to end, including that it's listed under Settings > Info.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Example plugins - Rename the fetcher example to fetcher/example_browser_fetcher

"Example browser fetcher": an extra fetcher that gets pages through its own API. Package
cdio_example_browser_fetcher, fetch method html_example_browser_fetcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Example browser fetcher - Return a screenshot (random noise PNG, standard library only)

Shows the screenshot side of a fetcher: supports_screenshots = True and self.screenshot set to PNG bytes,
saved as last-screenshot.png. The test checks it's served as a valid 240x160 PNG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dgtlmoonandClaude Opus 5.5 authored and GitHub committed 2026-10-29 16:44:03 +01:00
1 parent c0b9320ef3
commit b40193b080
22 files changed
+618 -30

No files matched your search

+53 -2
View File
@@ -5,7 +5,7 @@ active until you install it.
To install one, run `pip install .` from its directory, in the same Python environment that runs
changedetection.io, then restart changedetection.io. Plugins are picked up through the `changedetectionio`
entry point in their `pyproject.toml`. They need plugin API 0.2.0 or newer
entry point in their `pyproject.toml`. They need plugin API 0.3.0 or newer
(`changedetectionio.plugin_api.PLUGIN_API_VERSION`).
Plugins import only from `changedetectionio.plugin_api`, everything else in the package is internal and
@@ -21,6 +21,7 @@ the page and reports a change when the count changes. The `min_word_length` sett
**Hooks**
- `register_processor` - adds `word_counter` as a processor
- `get_api_watch_processor_state` - shows the count as `processor_state_word_counter` in the API
- `watch_list_extra_text` - "Found: 5 words" under the watch in the watch list
**Also shows** `processor.py` (the check itself) and `api.yaml` (documents `processor_config_word_counter`
and `processor_state_word_counter` in the API spec).
@@ -43,10 +44,60 @@ curl "http://localhost:5000/api/v1/watch/YOUR-UUID" -H "x-api-key: YOUR_API_KEY"
---
## Example browser fetcher - `fetcher/example_browser_fetcher`
A new fetch method, "Example browser fetcher", that gets pages through its own API instead of fetching
them directly, for example a remote browser or rendering service.
**Hooks**
- `register_content_fetcher` - adds the `html_example_browser_fetcher` fetch method
- `plugin_settings_tab` - an "Example browser fetcher" tab in Settings for the API URL and key
- `fetcher_status_icon` - the icon next to watches using it in the watch list
- `plugin_static_path` - serves that icon
**Also shows** `fetcher.py`, a `Fetcher` subclass, reading its settings with `load_plugin_settings` and
returning a screenshot (random noise here, a real browser service would send its own).
**Install**
```bash
cd changedetectionio/example_plugins/fetcher/example_browser_fetcher
pip install .
```
**Try it:** in Settings > "Example browser fetcher", set the API URL with `{url}` where the page URL goes,
for example `https://api.example.com/fetch?url={url}`, and your key (sent as `X-Api-Key`). Then pick
"Example browser fetcher" as a watch's fetch method, or `"fetch_backend": "html_example_browser_fetcher"`
in the API.
---
## Check stats - `lifecycle/check_stats`
Times every check. Shows "Last check took 1.23s" under each watch in the watch list, and a small table on
the edit page's Stats tab. Stats are kept in memory, so they start again after a restart.
**Hooks**
- `update_handler_alter` - runs as a check starts, notes the time
- `update_finalize` - runs when the check is over (worked or not), records how long it took
- `watch_list_extra_text` - the note under the watch in the watch list
- `ui_edit_stats_extras` - the table on the edit page's Stats tab
- `get_html_head_extras` - links its CSS on every page
- `plugin_static_path` - serves that CSS
**Install**
```bash
cd changedetectionio/example_plugins/lifecycle/check_stats
pip install .
```
**Try it:** open the watch list after the next checks have run.
---
## Writing your own
Copy an example's directory, then rename the package directory and its `name` / entry point in
`pyproject.toml`. For a processor, also change `PROCESSOR_NAME` and the two schema names in `api.yaml`.
`pyproject.toml`. Once installed, it's listed under Settings > Info with the first line of its docstring. For a processor, also change `PROCESSOR_NAME` and the two schema names in `api.yaml`.
Every hook and its arguments are documented on `ChangeDetectionSpec` in
[`pluggy_interface.py`](../pluggy_interface.py).
@@ -0,0 +1,55 @@
"""
Example plugin - an extra browser fetcher that gets pages through its own API.
Configure it under Settings > "Example browser fetcher", then pick "Example browser fetcher" as a
watch's fetch method. Install it with `pip install .` from the directory above this one, see
example_plugins/README.md. It is not active until installed (or registered).
"""
import os
from changedetectionio.plugin_api import hookimpl
# Names your settings file ({datastore}/cdio_example_browser_fetcher.json) and the settings tab
PLUGIN_ID = 'cdio_example_browser_fetcher'
# Fetcher names must start with 'html_', this is what's stored in a watch's 'fetch_backend'
FETCHER_NAME = 'html_example_browser_fetcher'
# Set at startup by changedetection.io (inject_datastore_into_plugins), the fetcher reads its
# settings with it
datastore = None
@hookimpl
def register_content_fetcher():
from .fetcher import fetcher
return (FETCHER_NAME, fetcher)
@hookimpl
def plugin_settings_tab():
from wtforms import Form, StringField
class SettingsForm(Form):
# These share the settings page form with everything else, so give them unique names
example_browser_fetcher_url = StringField(
'API URL',
description='Use {url} where the page URL goes, for example https://api.example.com/fetch?url={url}')
example_browser_fetcher_key = StringField(
'API key', description='Sent to the API in the X-Api-Key header')
return {'plugin_id': PLUGIN_ID, 'tab_label': 'Example browser fetcher', 'form_class': SettingsForm}
@hookimpl
def fetcher_status_icon(fetcher_name):
"""The icon next to watches using this fetcher in the watch list."""
if fetcher_name != FETCHER_NAME:
return None
return {'filename': 'cdio_example_browser_fetcher.svg', 'alt': 'Example browser fetcher', 'title': 'Fetched with the example browser fetcher',
'group': 'plugin', 'style': 'height: 1em;'}
@hookimpl
def plugin_static_path():
"""Files in here are served at /static/plugin/<filename>."""
return os.path.join(os.path.dirname(__file__), 'static')
@@ -0,0 +1,75 @@
import asyncio
from urllib.parse import quote
from changedetectionio.plugin_api import Fetcher, load_plugin_settings
def noise_png(width=240, height=160):
"""A PNG of random noise, standing in for a real page screenshot. Standard library only."""
import os
import struct
import zlib
def chunk(kind, data):
return struct.pack('>I', len(data)) + kind + data + struct.pack('>I', zlib.crc32(kind + data) & 0xffffffff)
# Each row: filter type 0, then width RGB pixels
pixels = b''.join(b'\x00' + os.urandom(width * 3) for _ in range(height))
return (b'\x89PNG\r\n\x1a\n'
+ chunk(b'IHDR', struct.pack('>IIBBBBB', width, height, 8, 2, 0, 0, 0)) # 8 bit RGB
+ chunk(b'IDAT', zlib.compress(pixels))
+ chunk(b'IEND', b''))
class fetcher(Fetcher):
fetcher_description = 'Example browser fetcher'
# Tells changedetection.io this fetcher returns a screenshot (see _fetch), so it's shown and saved
supports_screenshots = True
def __init__(self, proxy_override=None, custom_browser_connection_url=None, **kwargs):
super().__init__(**kwargs)
async def run(self, url=None, timeout=None, request_headers=None, **kwargs):
# requests is blocking, keep it off the event loop
await asyncio.get_event_loop().run_in_executor(None, lambda: self._fetch(url, timeout, request_headers))
def _fetch(self, url, timeout, request_headers):
import requests
from . import PLUGIN_ID, datastore
settings = load_plugin_settings(datastore.datastore_path, PLUGIN_ID) if datastore else {}
api_url = (settings.get('example_browser_fetcher_url') or '').strip()
if '{url}' not in api_url:
raise Exception("Example browser fetcher isn't set up, add an API URL with {url} in Settings")
headers = dict(request_headers or {})
if settings.get('example_browser_fetcher_key'):
headers['X-Api-Key'] = settings['example_browser_fetcher_key']
r = requests.get(api_url.replace('{url}', quote(url, safe='')), headers=headers, timeout=timeout or 60)
if not r.ok:
raise Exception(f"Example browser fetcher API returned HTTP {r.status_code}")
# What a fetcher must set for the processor
self.status_code = r.status_code
self.headers = dict(r.headers)
self.content = r.text
self.raw_content = r.content
# A real browser service would return its screenshot of the page, saved as last-screenshot.png
# (so return PNG bytes). This example just makes some noise.
self.screenshot = noise_png()
async def quit(self, watch=None):
pass
def get_error(self):
return self.error
def get_last_status_code(self):
return self.status_code
def screenshot_step(self, step_n):
return None
def is_ready(self):
return True
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M18 10h-1.26A8 8 0 1 0 9 20h9a5 5 0 0 0 0-10z"/></svg>

After

Width:  |  Height:  |  Size: 222 B

@@ -0,0 +1,23 @@
[build-system]
requires = ["setuptools>=77"] # 77+ for the SPDX license string
build-backend = "setuptools.build_meta"
[project]
name = "cdio-example-browser-fetcher"
version = "0.1.0"
description = "Example changedetection.io plugin: an extra browser fetcher that gets pages through its own API"
license = "Apache-2.0"
requires-python = ">=3.11"
# No dependency on changedetection.io itself: install this into the same environment that runs it
# (it needs plugin API 0.3.0 or newer, changedetectionio.plugin_api.PLUGIN_API_VERSION)
dependencies = []
# This is what makes changedetection.io load the plugin at startup
[project.entry-points.changedetectionio]
cdio_example_browser_fetcher = "cdio_example_browser_fetcher"
[tool.setuptools]
packages = ["cdio_example_browser_fetcher"]
[tool.setuptools.package-data]
cdio_example_browser_fetcher = ["static/*"]
@@ -0,0 +1,84 @@
"""
Example plugin - times every check and shows the result in the watch list and on the edit page.
Install it with `pip install .` from the directory above this one, see example_plugins/README.md.
It is not active until installed (or registered), nothing scans example_plugins/ for plugins.
The stats are kept in memory only, so they start empty again after a restart.
"""
import os
import time
from markupsafe import escape
from changedetectionio.plugin_api import hookimpl
# Per watch uuid: {'duration': seconds, 'ok': bool, 'error': str or None, 'finished': unix time, 'count': int}
STATS = {}
# Where the handler keeps its start time between the two check hooks below
_STARTED_ATTR = '_cdio_example_check_stats_started'
@hookimpl
def update_handler_alter(update_handler, watch, datastore):
"""Called once a check is about to start, before the page is fetched."""
setattr(update_handler, _STARTED_ATTR, time.monotonic())
# None = keep the handler as it is, you could also return a wrapped one
return None
@hookimpl
def update_finalize(update_handler, watch, datastore, processing_exception):
"""Called once the check is over, whether it worked or not."""
started = getattr(update_handler, _STARTED_ATTR, None) if update_handler else None
if started is None or not watch:
return
previous = STATS.get(watch.get('uuid'), {})
STATS[watch.get('uuid')] = {
'duration': time.monotonic() - started,
'ok': processing_exception is None,
'error': str(processing_exception) if processing_exception else None,
'finished': time.time(),
'count': previous.get('count', 0) + 1,
}
@hookimpl
def watch_list_extra_text(watch):
"""A short note under the watch in the watch list. Called for every row, so in memory only."""
stats = STATS.get(watch.get('uuid'))
if not stats:
return None
if stats['ok']:
return f"Last check took {stats['duration']:.2f}s"
return f"Last check failed after {stats['duration']:.2f}s"
@hookimpl
def ui_edit_stats_extras(watch):
"""A small table on the edit page's Stats tab. This is raw HTML, so escape anything you include."""
stats = STATS.get(watch.get('uuid'))
if not stats:
return '<p class="cdio-example-check-stats">No checks timed since the last restart.</p>'
rows = [
('Checks timed since restart', stats['count']),
('Last check took', f"{stats['duration']:.2f}s"),
('Last check', 'OK' if stats['ok'] else f"Failed: {stats['error']}"),
]
body = ''.join(f'<tr><td>{escape(label)}</td><td>{escape(value)}</td></tr>' for label, value in rows)
return f'<table class="pure-table cdio-example-check-stats"><tbody>{body}</tbody></table>'
@hookimpl
def plugin_static_path():
"""Files in here are served at /static/plugin/<filename>."""
return os.path.join(os.path.dirname(__file__), 'static')
@hookimpl
def get_html_head_extras():
"""Added inside <head> on every page."""
from flask import url_for
css = url_for('static_content', group='plugin', filename='cdio_example_check_stats.css')
return f'<link rel="stylesheet" href="{css}">'
@@ -0,0 +1,4 @@
/* Served by the plugin's plugin_static_path(), linked from get_html_head_extras() */
.cdio-example-check-stats td:first-child {
font-weight: bold;
}
@@ -0,0 +1,23 @@
[build-system]
requires = ["setuptools>=77"] # 77+ for the SPDX license string
build-backend = "setuptools.build_meta"
[project]
name = "cdio-example-check-stats"
version = "0.1.0"
description = "Example changedetection.io plugin: times every check, shows it in the watch list and edit page"
license = "Apache-2.0"
requires-python = ">=3.11"
# No dependency on changedetection.io itself: install this into the same environment that runs it
# (it needs plugin API 0.3.0 or newer, changedetectionio.plugin_api.PLUGIN_API_VERSION)
dependencies = []
# This is what makes changedetection.io load the plugin at startup
[project.entry-points.changedetectionio]
cdio_example_check_stats = "cdio_example_check_stats"
[tool.setuptools]
packages = ["cdio_example_check_stats"]
[tool.setuptools.package-data]
cdio_example_check_stats = ["static/*"]
@@ -25,6 +25,15 @@ def register_processor():
}
@hookimpl
def watch_list_extra_text(watch):
"""A note under the watch in the watch list. Called for every row, so in memory only."""
if watch.get('processor') != PROCESSOR_NAME:
return None
word_count = (watch.get(f'processor_state_{PROCESSOR_NAME}') or {}).get('word_count')
return f"Found: {word_count} words" if word_count is not None else None
@hookimpl
def get_api_watch_processor_state(watch, processor_name, context):
"""processor_state_word_counter in GET /api/v1/watch/<uuid> and GET /api/v1/watch, see api.yaml"""