Add support for extensions

discord_rss_bot/extensions/README.md

@@ -0,0 +1,157 @@
+# Extension System
+
+Extensions are Python plugins that extract data from RSS/Atom entries and/or mutate the Discord webhook payload before messages are sent.
+
+**Two hooks per entry:**
+
+1. **`process_entry()`** → return `{variable_name: string_value}` dict → available as `{{variable_name}}` in templates.
+2. **`modify_webhook()`** → mutate the `DiscordWebhook` (files, embeds, author, etc.) after template replacement.
+
+---
+
+## TL;DR — Make an Extension
+
+Drop a `.py` file in your `extensions/` dir (or `$EXTENSIONS_DIR`). Subclass `FeedExtension`:
+
+```python
+"""
+Word count example — provides {{word_count}} per entry.
+"""
+from __future__ import annotations
+import logging
+from typing import ClassVar, TYPE_CHECKING
+from discord_rss_bot.extensions.base import FeedExtension
+
+if TYPE_CHECKING:
+    from reader import Entry, Reader
+
+logger = logging.getLogger(__name__)
+
+
+class WordCountExtension(FeedExtension):
+    name = "word_count"
+    description = "Counts words in each entry's content."
+    provides_variables: ClassVar[list[str]] = ["word_count"]
+
+    def process_entry(self, entry: Entry, reader: Reader) -> dict[str, str]:
+        text = " ".join(
+            c.value for c in (entry.content or []) if hasattr(c, "value") and c.value
+        ) or (entry.summary or "")
+        return {"word_count": str(len(text.split())) if text else "0"}
+```
+
+Then restart the bot, enable it in the web UI's Extensions page, and use `{{word_count}}` in templates.
+
+---
+
+## API Reference
+
+### Class Attributes
+
+| Attribute                  | Type                  | Required | Purpose                                                   |
+| -------------------------- | --------------------- | -------- | --------------------------------------------------------- |
+| `name`                     | `ClassVar[str]`       | **Yes**  | Unique slug for logs, registry, enable/disable            |
+| `description`              | `ClassVar[str]`       | No       | Shown in web UI                                           |
+| `provides_variables`       | `ClassVar[list[str]]` | No       | Template variables this extension produces                |
+| `auto_enable_url_patterns` | `ClassVar[list[str]]` | No       | Regex patterns; matching feeds auto-enable this extension |
+
+### `process_entry(self, entry: Entry, reader: Reader) -> dict[str, str]` (required)
+
+Called for every entry. Return `{var: value}` pairs. Missing variables from `provides_variables` get filled with `""`.
+
+- `entry.content` — list of `Content` objects (`.value`, `.type`)
+- `entry.summary` — plain-text string
+- `entry.feed.url`, `entry.link`, `entry.title`, `entry.id`, etc.
+
+### `modify_webhook(self, webhook: DiscordWebhook, entry: Entry, reader: Reader) -> DiscordWebhook` (optional)
+
+Called after template replacement. Return modified `webhook` or it unchanged.
+
+```python
+# Content
+webhook.content = "..."  # or None to clear
+
+# Author
+webhook.username = "Bot Name"
+webhook.avatar_url = "https://..."
+
+# Embeds
+embed = DiscordEmbed()
+embed.set_title("...")
+embed.set_description("...")
+embed.set_url("https://...")
+embed.set_color(0x00FF00)  # or hex str "00FF00"
+embed.set_author(name="...", url="...", icon_url="...")
+embed.set_thumbnail(url="...")
+embed.set_image(url="...")
+embed.set_footer(text="...", icon_url="...")
+embed.add_embed_field(name="Key", value="Value", inline=True)
+embed.set_timestamp(timestamp="1234567890")
+webhook.add_embed(embed)
+webhook.remove_embeds()
+
+# Files
+webhook.add_file(file=bytes_, filename="img.png")
+```
+
+### Utility Class Methods
+
+- `matches_feed_url(cls, feed_url) → bool` — checks `feed_url` against `auto_enable_url_patterns`
+- `get_enabled_variables(cls, registry, enabled_names) → list[str]` — sorted union of `provides_variables` for enabled extensions
+
+---
+
+## Discovery
+
+Extensions are discovered from two sources (external overrides built-in):
+
+1. **Built-in** — `discord_rss_bot/extensions/*.py` (this dir)
+2. **External** — `EXTENSIONS_DIR` env var, default `extensions/` relative to CWD
+
+Every `.py` (except `__init__`) is imported. Concrete `FeedExtension` subclasses with a non-empty `name` go into a global `registry: dict[str, type[FeedExtension]]`. Duplicate names log a warning; last one wins.
+
+| Function                           | Description                                      |
+| ---------------------------------- | ------------------------------------------------ |
+| `discover_plugins(*, force=False)` | Scan dirs, populate registry, return it          |
+| `get_registry()`                   | Return current registry (discover on first call) |
+| `registry_clear()`                 | Clear registry (for tests)                       |
+
+---
+
+## Auto-Enable
+
+Set `auto_enable_url_patterns` on your class:
+
+```python
+auto_enable_url_patterns: ClassVar[list[str]] = [
+    r"store\.steampowered\.com",
+    r"youtube\.com/feeds/videos\.xml",
+]
+```
+
+Uses `re.search()` (matches anywhere in URL). Only applies when the feed's extensions tag was never explicitly saved by the user. Runs on feed creation and lazily on first process for old feeds.
+
+---
+
+## Per-Feed Storage
+
+Enabled extensions stored as a JSON list under reader tag key `"extensions"`.
+
+| Function                                                        | Description                  |
+| --------------------------------------------------------------- | ---------------------------- |
+| `get_enabled_extensions_for_feed(reader, feed_url) → list[str]` | Get enabled names for a feed |
+| `set_enabled_extensions_for_feed(reader, feed_url, names)`      | Persist enabled names        |
+
+Web UI: `GET /extensions?feed_url=...` renders checkboxes; `POST /extensions` saves.
+
+---
+
+## Built-in Extensions
+
+| Extension                                      | `name`               | Variables                                                                                                   | Auto-enable                                    |
+| ---------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------- | ---------------------------------------------- |
+| [youtube.py](youtube.py)                       | `youtube`            | `youtube_video_id`, `youtube_embed_url`                                                                     | `youtube.com/feeds/videos.xml`                 |
+| [steam.py](steam.py)                           | `steam`              | `steam_thumbnail_url`, `steam_app_id`                                                                       | `store.steampowered.com`, `steamcommunity.com` |
+| [hoyolab.py](hoyolab.py)                       | `hoyolab`            | `hoyolab_subject`, `hoyolab_description`, `hoyolab_image`, `hoyolab_author`                                 | `feeds.c3kay.de`                               |
+| [jwplayer_thumbnail.py](jwplayer_thumbnail.py) | `jwplayer_thumbnail` | `jwplayer_thumbnail`, `jwplayer_file`                                                                       | `hentaigasm.com`, `hgasm[0-9]*.com`            |
+| [wordpress.py](wordpress.py)                   | `wordpress`          | `wp_content`, `wp_content_raw`, `wp_excerpt`, `wp_excerpt_raw`, `wp_jwplayer_thumbnail`, `wp_jwplayer_file` | _(none)_                                       |

discord_rss_bot/extensions/__init__.py

@@ -0,0 +1,32 @@
+"""Feed extension system: discover, configure, and run per-entry content extensions.
+
+Extensions are Python classes that inherit from ``FeedExtension``,
+auto-discovered from the ``EXTENSIONS_DIR`` directory (default: ``extensions/``).
+
+Typical usage::
+
+    from discord_rss_bot.extensions import run_extensions
+
+    extra_vars: dict[str, str] = run_extensions(entry, reader)
+    # extra_vars == {"jwplayer_thumbnail": "https://...", ...}
+"""
+
+from __future__ import annotations
+
+from discord_rss_bot.extensions.base import FeedExtension
+from discord_rss_bot.extensions.discovery import discover_plugins
+from discord_rss_bot.extensions.discovery import get_registry
+from discord_rss_bot.extensions.discovery import registry_clear
+from discord_rss_bot.extensions.runner import auto_enable_extensions_for_feed
+from discord_rss_bot.extensions.runner import run_extensions
+from discord_rss_bot.extensions.runner import run_modify_webhook
+
+__all__ = [
+    "FeedExtension",
+    "auto_enable_extensions_for_feed",
+    "discover_plugins",
+    "get_registry",
+    "registry_clear",
+    "run_extensions",
+    "run_modify_webhook",
+]

discord_rss_bot/extensions/base.py

@@ -0,0 +1,127 @@
+"""Abstract base class for feed extensions.
+
+Every extension in the ``EXTENSIONS_DIR`` must inherit from
+``FeedExtension`` and implement ``process_entry()``.
+"""
+
+from __future__ import annotations
+
+import re
+from abc import ABC
+from abc import abstractmethod
+from typing import TYPE_CHECKING
+from typing import ClassVar
+
+if TYPE_CHECKING:
+    from reader import Entry
+    from reader import Reader
+
+    from discord_rss_bot.webhook import DiscordWebhook
+
+
+class FeedExtension(ABC):
+    """Base class for per-feed content extensions.
+
+    Subclasses are auto-discovered from the ``EXTENSIONS_DIR`` directory.
+    Each instance processes one entry and returns template variables
+    that become available as ``{{variable_name}}`` in Discord messages
+    and embeds.
+
+    Class attributes set by subclasses:
+
+    ``name``:
+        Unique identifier (used in logs and per-feed enable/disable config).
+        Should be a short slug, e.g. ``"jwplayer_thumbnail"``.
+
+    ``description``:
+        Human-readable description shown in the web UI (optional).
+    """
+
+    name: ClassVar[str] = ""
+    description: ClassVar[str] = ""
+
+    #: List of template variable names this extension provides.
+    #: Used by the web UI to show users what ``{{variable}}`` tags
+    #: are available when an extension is enabled.
+    provides_variables: ClassVar[list[str]] = []
+
+    #: List of regex patterns.  When a feed URL matches any pattern,
+    #: this extension is automatically enabled for that feed.
+    auto_enable_url_patterns: ClassVar[list[str]] = []
+
+    @abstractmethod
+    def process_entry(self, entry: Entry, reader: Reader) -> dict[str, str]:
+        """Return template variable pairs extracted from *entry*.
+
+        Args:
+            entry: The feed entry to process.
+            reader: The reader instance (useful for loading feed tags).
+
+        Returns:
+            A dict of ``{variable_name: value}`` pairs.  Return an empty
+            dict if this entry does not need any extra variables.
+        """
+
+    def modify_webhook(
+        self,
+        webhook: DiscordWebhook,
+        _entry: Entry,
+        _reader: Reader,
+    ) -> DiscordWebhook:
+        """Modify the Discord webhook before it is sent.
+
+        Override this method when your extension needs to change the
+        webhook payload itself (e.g. to add files, replace embeds, set
+        author information, etc.).  By default it returns the webhook
+        unchanged.
+
+        Args:
+            webhook: The fully built webhook payload.
+            _entry: The feed entry being processed.
+            _reader: The reader instance.
+
+        Returns:
+            The (possibly modified) ``DiscordWebhook``.  Return the
+            original *webhook* unchanged if no modifications are needed.
+        """
+        return webhook
+
+    @classmethod
+    def get_enabled_variables(cls, registry: dict[str, type[FeedExtension]], enabled_names: list[str]) -> list[str]:
+        """Return the union of ``provides_variables`` for all enabled extensions.
+
+        Args:
+            registry: The extension registry ``{name: class}``.
+            enabled_names: List of enabled extension names for a feed.
+
+        Returns:
+            Sorted list of unique variable names.
+        """
+        seen: set[str] = set()
+        result: list[str] = []
+        for name in enabled_names:
+            ext_cls = registry.get(name)
+            if ext_cls is None:
+                continue
+            for var_name in ext_cls.provides_variables:
+                if var_name not in seen:
+                    seen.add(var_name)
+                    result.append(var_name)
+        result.sort()
+        return result
+
+    @classmethod
+    def matches_feed_url(cls, feed_url: str) -> bool:
+        """Return ``True`` if *feed_url* matches any of the auto-enable patterns.
+
+        Args:
+            feed_url: The feed URL to test.
+
+        Returns:
+            ``True`` if a match is found, ``False`` otherwise (including when
+            no patterns are defined).
+        """
+        if not cls.auto_enable_url_patterns:
+            return False
+
+        return any(re.search(p, feed_url) for p in cls.auto_enable_url_patterns)

discord_rss_bot/extensions/discovery.py

@@ -0,0 +1,191 @@
+"""Plugin discovery — import ``.py`` files from ``EXTENSIONS_DIR``.
+
+All ``FeedExtension`` subclasses found in those files are collected into
+a global registry (``dict[str, type[FeedExtension]]``).
+"""
+
+from __future__ import annotations
+
+import importlib.util
+import logging
+import os
+import sys
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    import types
+
+from discord_rss_bot.extensions.base import FeedExtension
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+#: Environment variable that points to the extensions directory.
+ENV_EXTENSIONS_DIR: str = "EXTENSIONS_DIR"
+
+#: Fallback if the env var is not set.
+DEFAULT_EXTENSIONS_DIR: str = "extensions"
+
+_registry: dict[str, type[FeedExtension]] = {}
+_discovered: bool = False
+
+
+def _get_extensions_dir() -> Path:
+    """Return the path to the extensions directory.
+
+    Respects the ``EXTENSIONS_DIR`` environment variable.  Falls back to
+    ``extensions/`` relative to the current working directory.
+    """
+    raw: str = os.getenv(ENV_EXTENSIONS_DIR, "").strip()
+    if raw:
+        return Path(raw).resolve()
+    return Path.cwd() / DEFAULT_EXTENSIONS_DIR
+
+
+def _try_import_module(module_name: str, filepath: Path) -> types.ModuleType | None:
+    """Create a module spec and execute it, returning the module or ``None``.
+
+    Args:
+        module_name: The name to give the imported module.
+        filepath: Absolute path to a ``.py`` file.
+
+    Returns:
+        The imported module, or ``None`` if the spec could not be created.
+    """
+    spec = importlib.util.spec_from_file_location(module_name, filepath)
+    if spec is None or spec.loader is None:
+        logger.warning("Could not create module spec for %s", filepath)
+        return None
+
+    module = importlib.util.module_from_spec(spec)
+    # Make the parent package importable so relative imports in plugins work.
+    sys.modules[module_name] = module
+    spec.loader.exec_module(module)
+    return module
+
+
+def _import_module_from_path(filepath: Path) -> types.ModuleType | None:
+    """Import a single ``.py`` file as a module and return the module object.
+
+    Args:
+        filepath: Absolute path to a ``.py`` file.
+
+    Returns:
+        The imported module, or ``None`` on failure.
+    """
+    module_name: str = f"_ext_plugin_{filepath.stem}"
+
+    # Avoid re-importing if already loaded.
+    if module_name in sys.modules:
+        return sys.modules[module_name]
+
+    try:
+        return _try_import_module(module_name, filepath)
+    except Exception:
+        logger.exception("Failed to import extension plugin from %s", filepath)
+        return None
+
+
+def _collect_extensions_from_module(module: types.ModuleType) -> list[type[FeedExtension]]:
+    """Return ``FeedExtension`` subclasses defined in *module*.
+
+    Only concrete (non-ABC) subclasses with a non-empty ``name`` are
+    returned.
+
+    Args:
+        module: An imported module object.
+
+    Returns:
+        List of extension classes found in the module.
+    """
+    extensions: list[type[FeedExtension]] = []
+    for attr_name in dir(module):
+        obj = getattr(module, attr_name)
+        if (
+            isinstance(obj, type)
+            and issubclass(obj, FeedExtension)
+            and obj is not FeedExtension
+            and not getattr(obj, "__abstractmethods__", None)
+        ):
+            name: str = getattr(obj, "name", "") or attr_name
+            extensions.append(obj)
+            logger.debug("Discovered extension %r (class %s)", name, attr_name)
+    return extensions
+
+
+def _scan_directory(ext_dir: Path) -> None:
+    """Scan a single directory and register any ``FeedExtension`` subclasses found."""
+    if not ext_dir.is_dir():
+        return
+
+    py_files: list[Path] = sorted(p for p in ext_dir.iterdir() if p.suffix == ".py" and p.stem != "__init__")
+    for filepath in py_files:
+        module = _import_module_from_path(filepath)
+        if module is None:
+            continue
+        classes = _collect_extensions_from_module(module)
+        for cls in classes:
+            cls_name: str = cls.name or cls.__name__
+            if cls_name in _registry:
+                logger.warning("Duplicate extension name %r — overwriting with %s", cls_name, filepath)
+            _registry[cls_name] = cls
+
+
+def discover_plugins(*, force: bool = False) -> dict[str, type[FeedExtension]]:
+    """Scan built-in and external extension directories and register.
+
+    All ``FeedExtension`` subclasses found in those directories are
+    collected into a global registry.
+
+    Order of discovery:
+    1. Built-in extensions shipped with the bot (``discord_rss_bot/extensions/``)
+    2. External user-provided plugins from ``EXTENSIONS_DIR``
+
+    External plugins can override built-in ones by using the same ``name``.
+    This is called automatically on first import.  Call with
+    ``force=True`` to re-scan (useful in tests).
+
+    Args:
+        force: If ``True``, re-scan even if already discovered.
+
+    Returns:
+        The global registry ``{name: class}``.
+    """
+    global _discovered  # ruff:ignore[global-statement]
+    if _discovered and not force:
+        return _registry
+
+    # 1. Scan the built-in package directory first.
+    built_in_dir: Path = Path(__file__).resolve().parent
+    logger.debug("Scanning built-in extensions directory: %s", built_in_dir)
+    _scan_directory(built_in_dir)
+
+    # 2. Scan the external user-provided extensions directory.
+    ext_dir: Path = _get_extensions_dir()
+    if ext_dir != built_in_dir:
+        logger.info("Scanning external extensions directory: %s", ext_dir)
+        _scan_directory(ext_dir)
+    else:
+        logger.debug("External extensions directory is the same as built-in — skipping duplicate scan")
+
+    _discovered = True
+    logger.info("Discovered %d extension plugin(s): %s", len(_registry), list(_registry.keys()))
+    return _registry
+
+
+def get_registry() -> dict[str, type[FeedExtension]]:
+    """Return the current extension registry, discovering plugins if needed.
+
+    Returns:
+        The global registry ``{name: class}``.
+    """
+    if not _discovered:
+        return discover_plugins()
+    return _registry
+
+
+def registry_clear() -> None:
+    """Clear the registry (useful for testing)."""
+    global _discovered, _registry  # ruff:ignore[global-statement]
+    _registry = {}
+    _discovered = False

discord_rss_bot/extensions/hoyolab.py

@@ -0,0 +1,331 @@
+"""Hoyolab feed extension: fetches full post data from the Hoyolab API.
+
+Detects feeds from ``feeds.c3kay.de`` and replaces the Discord embed
+with richer content fetched directly from Hoyolab (Genshin Impact,
+Honkai Starrail, Honkai Impact 3rd, Zenless Zone Zero).
+
+Usage:
+    1. Enable ``hoyolab`` on the feed's Extensions page.
+    2. The extension automatically decorates the embed with the post
+       title, author avatar, game color, images, and more.
+"""
+
+from __future__ import annotations
+
+import contextlib
+import json
+import logging
+import re
+from typing import TYPE_CHECKING
+from typing import ClassVar
+from typing import cast
+
+import requests
+
+from discord_rss_bot.extensions.base import FeedExtension
+from discord_rss_bot.webhook import DiscordEmbed
+from discord_rss_bot.webhook import DiscordWebhook
+
+if TYPE_CHECKING:
+    from reader import Entry
+    from reader import Reader
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+type JsonValue = bool | int | float | str | list[JsonValue] | dict[str, JsonValue] | None
+type JsonObject = dict[str, JsonValue]
+
+
+def is_c3kay_feed_url(feed_url: str) -> bool:
+    """Return whether a feed URL is from feeds.c3kay.de."""
+    return "feeds.c3kay.de" in (feed_url or "")
+
+
+def extract_post_id(url: str) -> str | None:
+    """Extract the Hoyolab post ID from a URL.
+
+    Args:
+        url: The URL to inspect (e.g. ``https://www.hoyolab.com/article/38588239``).
+
+    Returns:
+        The numeric post ID if found, otherwise ``None``.
+    """
+    if not url:
+        return None
+    try:
+        match = re.search(r"/article/(\d+)", url)
+        if match:
+            return match.group(1)
+    except (ValueError, AttributeError, TypeError) as exc:
+        logger.warning("Error extracting post ID from URL %s: %s", url, exc)
+    return None
+
+
+def _try_fetch_post(post_id: str) -> JsonObject | None:
+    """Attempt to fetch a Hoyolab post, returning the payload or ``None``.
+
+    Does not catch exceptions — the caller handles them.
+
+    Returns:
+        The post payload dict, or ``None`` on failure.
+    """
+    url: str = f"https://bbs-api-os.hoyolab.com/community/post/wapi/getPostFull?post_id={post_id}"
+    response: requests.Response = requests.get(url, timeout=10)
+    if response.status_code == http_ok:
+        data = cast("JsonObject", response.json())
+        data_payload: JsonObject = cast("JsonObject", data.get("data", {}))
+        post_payload: JsonObject = cast("JsonObject", data_payload.get("post", {}))
+        if data.get("retcode") == 0 and post_payload:
+            return post_payload
+    logger.warning("Failed to fetch Hoyolab post %s: %s", post_id, response.text)
+    return None
+
+
+def fetch_post(post_id: str) -> JsonObject | None:
+    """Fetch post data from the Hoyolab API.
+
+    Args:
+        post_id: The numeric post ID.
+
+    Returns:
+        The post payload dict, or ``None`` on failure.
+    """
+    if not post_id:
+        return None
+    try:
+        return _try_fetch_post(post_id)
+    except (requests.RequestException, ValueError):
+        logger.exception("Error fetching Hoyolab post %s", post_id)
+    return None
+
+
+http_ok: int = 200
+
+
+def _as_json_object(value: JsonValue) -> JsonObject:
+    return cast("JsonObject", value) if isinstance(value, dict) else {}
+
+
+class HoyolabExtension(FeedExtension):
+    """Fetches full post data from the Hoyolab API for c3kay.de feeds.
+
+    Replaces the Discord embed with richer content (title, author avatar,
+    game color, images, video, and more).
+    """
+
+    name = "hoyolab"
+    description = (
+        "Detects c3kay.de / Hoyolab feeds and replaces the Discord embed "
+        "with full post data from the Hoyolab API "
+        "(Genshin Impact, Honkai Starrail, Honkai Impact 3rd, ZZZ)."
+    )
+    provides_variables: ClassVar[list[str]] = [
+        "hoyolab_subject",
+        "hoyolab_description",
+        "hoyolab_image",
+        "hoyolab_author",
+    ]
+    auto_enable_url_patterns: ClassVar[list[str]] = [
+        r"feeds\.c3kay\.de",
+    ]
+
+    def process_entry(self, entry: Entry, reader: Reader) -> dict[str, str]:  # ruff:ignore[unused-method-argument]
+        """Provide Hoyolab post data as template variables.
+
+        Args:
+            entry: The feed entry to process.
+            reader: The reader instance.
+
+        Returns:
+            Dict with ``hoyolab_subject``, ``hoyolab_description``,
+            ``hoyolab_image`` and ``hoyolab_author`` if the data was
+            fetched successfully, otherwise empty dict.
+        """
+        if not is_c3kay_feed_url(entry.feed.url):
+            return {}
+
+        post_id: str | None = extract_post_id(str(entry.link or ""))
+        if not post_id:
+            return {}
+
+        post_data: JsonObject | None = fetch_post(post_id)
+        if not post_data:
+            return {}
+
+        post: JsonObject = _as_json_object(post_data.get("post"))
+        subject: str = str(post.get("subject", ""))
+        description: str = str(post.get("desc", ""))
+
+        image_url: str = ""
+        image_list = post_data.get("image_list", [])
+        if isinstance(image_list, list) and image_list:
+            first_image = image_list[0]
+            if isinstance(first_image, dict):
+                image_url = str(first_image.get("url", ""))
+
+        author: str = ""
+        user: JsonObject = _as_json_object(post_data.get("user"))
+        if user:
+            author = str(user.get("nickname", ""))
+
+        return {
+            "hoyolab_subject": subject,
+            "hoyolab_description": description,
+            "hoyolab_image": image_url,
+            "hoyolab_author": author,
+        }
+
+    def modify_webhook(
+        self,
+        webhook: DiscordWebhook,
+        entry: Entry,
+        reader: Reader,  # ruff:ignore[unused-method-argument]
+    ) -> DiscordWebhook:
+        """Replace the embed with richer Hoyolab content when applicable.
+
+        Args:
+            webhook: The fully built webhook payload.
+            entry: The feed entry being processed.
+            reader: The reader instance.
+
+        Returns:
+            The modified webhook with Hoyolab data.
+        """
+        if not is_c3kay_feed_url(entry.feed.url):
+            return webhook
+
+        # Don't interfere with non-embed delivery modes.
+        if not webhook.json.get("embeds"):
+            return webhook
+
+        post_id: str | None = extract_post_id(str(entry.link or ""))
+        if not post_id:
+            return webhook
+
+        post_data: JsonObject | None = fetch_post(post_id)
+        if not post_data:
+            return webhook
+
+        discord_embed = self._build_embed_from_post(post_data, entry.link or entry.feed.url)
+
+        self._attach_video_if_present(webhook, entry, post_data)
+        self._apply_author_from_post(webhook, post_data)
+        self._apply_structured_content(webhook, post_data)
+
+        webhook.remove_embeds()
+        webhook.add_embed(discord_embed)
+        return webhook
+
+    def _build_embed_from_post(self, post_data: JsonObject, entry_link: str) -> DiscordEmbed:
+        """Build a ``DiscordEmbed`` populated with post data.
+
+        Args:
+            post_data: Raw Hoyolab post payload.
+            entry_link: Fallback link URL.
+
+        Returns:
+            A populated embed.
+        """
+        post: JsonObject = _as_json_object(post_data.get("post"))
+        subject: str = str(post.get("subject", ""))
+        content_raw: str = str(post.get("content", "{}"))
+
+        content_data: JsonObject = {}
+        with contextlib.suppress(json.JSONDecodeError, ValueError):
+            loaded = cast("JsonValue", json.loads(content_raw))
+            content_data = _as_json_object(loaded)
+
+        description: str = str(content_data.get("describe", ""))
+        if not description:
+            description = str(post.get("desc", ""))
+
+        discord_embed = DiscordEmbed()
+        discord_embed.set_title(subject)
+        discord_embed.set_url(entry_link)
+
+        # Image list
+        image_list_value: JsonValue = post_data.get("image_list", [])
+        image_list: list[JsonObject] = (
+            [cast("JsonObject", item) for item in image_list_value if isinstance(item, dict)]
+            if isinstance(image_list_value, list)
+            else []
+        )
+        if image_list:
+            discord_embed.set_image(url=str(image_list[0].get("url", "")))
+
+        # Game colour
+        game: JsonObject = _as_json_object(post_data.get("game"))
+        if game and game.get("color"):
+            discord_embed.set_color(str(game.get("color", "")).removeprefix("#"))
+
+        # Footer
+        classification: JsonObject = _as_json_object(post_data.get("classification"))
+        if classification and classification.get("name"):
+            discord_embed.set_footer(text=str(classification.get("name", "")))
+
+        # Event dates
+        event_start: str = str(post.get("event_start_date", ""))
+        if event_start and event_start != "0":
+            discord_embed.add_embed_field(name="Start", value=f"<t:{event_start}:R>")
+        event_end: str = str(post.get("event_end_date", ""))
+        if event_end and event_end != "0":
+            discord_embed.add_embed_field(name="End", value=f"<t:{event_end}:R>")
+
+        created_at: str = str(post.get("created_at", ""))
+        if created_at and created_at != "0":
+            discord_embed.set_timestamp(timestamp=created_at)
+
+        if description:
+            discord_embed.set_description(description)
+
+        return discord_embed
+
+    def _attach_video_if_present(self, webhook: DiscordWebhook, entry: Entry, post_data: JsonObject) -> None:
+        """Download and attach a Hoyolab video to the webhook if present."""
+        video: JsonObject = _as_json_object(post_data.get("video"))
+        if not video or not video.get("url"):
+            return
+        video_url: str = str(video.get("url", ""))
+        with contextlib.suppress(requests.RequestException):
+            video_response: requests.Response = requests.get(video_url, stream=True, timeout=10)
+            if video_response.ok:
+                webhook.add_file(file=video_response.content, filename=f"{entry.id}.mp4")
+
+    def _apply_author_from_post(self, webhook: DiscordWebhook, post_data: JsonObject) -> None:
+        """Set webhook author (username and avatar) from post user data."""
+        user: JsonObject = _as_json_object(post_data.get("user"))
+        author_name: str = str(user.get("nickname", ""))
+        avatar_url: str = str(user.get("avatar_url", ""))
+        if author_name:
+            webhook.avatar_url = avatar_url
+            webhook.username = author_name
+
+    def _apply_structured_content(self, webhook: DiscordWebhook, post_data: JsonObject) -> None:
+        """Parse structured content for YouTube embeds and add them to the webhook."""
+        post: JsonObject = _as_json_object(post_data.get("post"))
+        structured_content: str = str(post.get("structured_content", ""))
+        if not structured_content:
+            return
+        with contextlib.suppress(json.JSONDecodeError, ValueError):
+            loaded_structured = cast("JsonValue", json.loads(structured_content))
+            structured_items: list[JsonObject] = (
+                [cast("JsonObject", item) for item in loaded_structured if isinstance(item, dict)]
+                if isinstance(loaded_structured, list)
+                else []
+            )
+            for item in structured_items:
+                self._apply_structured_item(webhook, item)
+
+    def _apply_structured_item(self, webhook: DiscordWebhook, item: JsonObject) -> None:
+        """Apply a single structured content item (e.g. YouTube embed)."""
+        insert: JsonObject = _as_json_object(item.get("insert"))
+        if not insert:
+            return
+        video_url_str: str = str(insert.get("video", ""))
+        if not video_url_str:
+            return
+        video_id_match = re.search(r"embed/([a-zA-Z0-9_-]+)", video_url_str)
+        if not video_id_match:
+            return
+        webhook.content = f"https://www.youtube.com/watch?v={video_id_match.group(1)}"
+        webhook.remove_embeds()

discord_rss_bot/extensions/jwplayer_thumbnail.py

@@ -0,0 +1,285 @@
+"""Extension: extract JWPlayer thumbnail/video URLs from WordPress sites.
+
+`JWPlayer <https://www.jwplayer.com/>`_ is a commercial video player
+used by many sites to serve self-hosted or CDN-hosted video. The player
+is embedded via a ``<script>`` block with a ``jwplayer()`` setup call
+that contains ``image:`` (thumbnail) and ``file:`` (video URL) properties.
+
+Feedparser strips ``<script>`` tags, so these URLs are invisible in the
+parsed feed content. This extension recovers them by either scanning the
+raw feed content directly or, for WordPress sites, fetching post HTML
+from the REST API in a single batched request (cached by slug).
+
+Created in response to `issue #432
+<https://github.com/TheLovinator1/discord-rss-bot/issues/432>`_.
+
+Currently auto-enabled for sites matching ``hentaigasm.com`` and
+``hgasm[0-9]*.com``.
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+from typing import TYPE_CHECKING
+from typing import ClassVar
+from urllib.parse import quote
+from urllib.parse import urlparse
+
+import httpx2
+
+from discord_rss_bot.extensions.base import FeedExtension
+
+if TYPE_CHECKING:
+    from reader import Entry
+    from reader import Reader
+
+    from discord_rss_bot.feeds import JsonValue
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+_IMAGE_PATTERN: re.Pattern[str] = re.compile(
+    r'image:\s*["\']([^"\']+)["\']',
+    re.IGNORECASE,
+)
+
+_FILE_PATTERN: re.Pattern[str] = re.compile(
+    r'file:\s*["\']([^"\']+\.\w+)["\']',
+    re.IGNORECASE,
+)
+
+#: Shared HTTP client — pooled connections, no TLS handshake per call.
+_HTTP_CLIENT: httpx2.Client | None = None
+
+#: Cache of ``{slug: post_data}`` — populated by the first entry
+#: that needs it, then reused for all subsequent entries from the same site.
+#: Each post_data dict contains ``content``, ``excerpt``, and ``title`` keys.
+_SLUG_CACHE: dict[str, dict[str, dict[str, str]]] = {}
+
+_HTTP_OK: int = 200
+
+
+def _get_http() -> httpx2.Client:
+    """Return the shared ``httpx2.Client``, creating it on first call.
+
+    Returns:
+        The shared ``httpx2.Client`` instance.
+    """
+    global _HTTP_CLIENT  # ruff:ignore[global-statement]
+    if _HTTP_CLIENT is None:
+        _HTTP_CLIENT = httpx2.Client(timeout=10)
+    return _HTTP_CLIENT
+
+
+def _extract_slug(url: str) -> str | None:
+    """Extract the last non-numeric path segment as a potential post slug.
+
+    Args:
+        url: The URL to extract the slug from.
+
+    Returns:
+        The extracted slug, or ``None`` if no slug could be extracted.
+    """
+    if not url:
+        return None
+    try:
+        path: str = urlparse(url).path.rstrip("/")
+        segments: list[str] = [s for s in path.split("/") if s]
+        if not segments or segments[-1].isdigit():
+            return None
+        return segments[-1]
+    except ValueError:
+        return None
+
+
+def _build_slug_cache(base_url: str) -> dict[str, dict[str, str]]:
+    """Fetch all recent posts from the WordPress API and build the slug cache.
+
+    Fetches ``content``, ``excerpt``, and ``title`` for each post so that
+    callers can access full post data without additional API requests.
+
+    Args:
+        base_url: The site base URL (scheme + netloc).
+
+    Returns:
+        A dict mapping slugs to ``{content: ..., excerpt: ..., title: ...}``.
+    """
+    client: httpx2.Client = _get_http()
+    fields: str = "slug,content,excerpt,title"
+    api_url: str = f"{base_url}/wp-json/wp/v2/posts?per_page=100&orderby=date&order=desc&_fields={fields}"
+    resp = client.get(api_url)
+    if resp.status_code != _HTTP_OK:
+        logger.warning("WordPress API returned %s for batch slug lookup", resp.status_code)
+        return {}
+
+    data = resp.json()
+    if not isinstance(data, list):
+        return {}
+
+    fresh: dict[str, dict[str, str]] = {}
+    for post in data:
+        if not isinstance(post, dict):
+            continue
+        ps = post.get("slug")
+        if not isinstance(ps, str):
+            continue
+        content_rendered: str = _extract_rendered(post.get("content"))
+        excerpt_rendered: str = _extract_rendered(post.get("excerpt"))
+        title_rendered: str = _extract_rendered(post.get("title"))
+        fresh[ps] = {
+            "content": content_rendered,
+            "excerpt": excerpt_rendered,
+            "title": title_rendered,
+        }
+
+    logger.info("Batch-fetched %d posts from %s", len(fresh), base_url)
+    return fresh
+
+
+def _extract_rendered(obj: JsonValue) -> str:
+    """Extract the ``rendered`` string from a WordPress API object.
+
+    Args:
+        obj: A WordPress API object (typically a dict with a ``rendered`` key).
+
+    Returns:
+        The rendered string, or ``""`` if the object is not a dict or
+        has no ``rendered`` key.
+    """
+    if isinstance(obj, dict):
+        rendered = obj.get("rendered")
+        if isinstance(rendered, str):
+            return rendered
+    return ""
+
+
+def _get_rendered_for_slug(slug: str, base_url: str) -> str | None:
+    """Return the content.rendered HTML for *slug*, fetching on first miss.
+
+    The first call for a given *base_url* triggers a **single** batch
+    request (``?per_page=100``) that caches all recent posts. Every
+    subsequent call for the same site is a dict lookup — zero network.
+
+    Args:
+        slug: The post slug to look up.
+        base_url: The site base URL (scheme + netloc).
+
+    Returns:
+        The rendered content HTML for the post, or ``None`` if not found.
+    """
+    post_data: dict[str, str] | None = _get_post_data_for_slug(slug, base_url)
+    if post_data is None:
+        return None
+    return post_data.get("content") or None
+
+
+def _get_post_data_for_slug(slug: str, base_url: str) -> dict[str, str] | None:
+    """Return the full post data dict for *slug*, fetching on first miss.
+
+    The returned dict contains ``content``, ``excerpt``, and ``title`` keys.
+
+    The first call for a given *base_url* triggers a **single** batch
+    request (``?per_page=100``) that caches all recent posts. Every
+    subsequent call for the same site is a dict lookup — zero network.
+
+    Args:
+        slug: The post slug to look up.
+        base_url: The site base URL (scheme + netloc).
+
+    Returns:
+        A dict with ``content``, ``excerpt``, ``title`` keys, or ``None``.
+    """
+    site_cache: dict[str, dict[str, str]] | None = _SLUG_CACHE.get(base_url)
+    if site_cache is not None:
+        return site_cache.get(slug)
+
+    try:
+        fresh: dict[str, dict[str, str]] = _build_slug_cache(base_url)
+    except Exception:
+        logger.exception("Failed to batch-fetch WordPress posts from %s", base_url)
+        fresh = {}
+
+    _SLUG_CACHE[base_url] = fresh
+    return fresh.get(slug)
+
+
+class JWPlayerThumbnailExtension(FeedExtension):
+    """Extract the ``image`` URL from a JWPlayer ``setup()`` call.
+
+    Uses a single batch WordPress API request per site to fetch all
+    recent post content, then looks up entries by slug — zero API
+    calls per entry.
+    """
+
+    name = "jwplayer_thumbnail"
+    description = (
+        "Extracts JWPlayer thumbnail and video URLs from entry content "
+        "or the WordPress REST API (batched). "
+        "Available as {{jwplayer_thumbnail}} and {{jwplayer_file}}."
+    )
+    provides_variables: ClassVar[list[str]] = ["jwplayer_thumbnail", "jwplayer_file"]
+    auto_enable_url_patterns: ClassVar[list[str]] = [
+        r"hentaigasm\.com",
+        r"hgasm[0-9]*\.com",
+    ]
+
+    def process_entry(self, entry: Entry, _reader: Reader) -> dict[str, str]:
+        """Extract JWPlayer URLs from feed content or WordPress API.
+
+        Args:
+            entry: The feed entry to process.
+
+        Returns:
+            Dict with ``jwplayer_thumbnail`` and ``jwplayer_file``
+            if found, otherwise empty dict.
+        """
+        result: dict[str, str] = {}
+
+        # Phase 1: search feed content (fast).
+        sources: list[str] = []
+        if entry.content:
+            sources.extend(item.value for item in entry.content if hasattr(item, "value") and item.value)
+        entry_summary: str | None = getattr(entry, "summary", None)
+        if entry_summary:
+            sources.append(entry_summary)
+
+        for source in sources:
+            self._search(source, result)
+            if "jwplayer_thumbnail" in result and "jwplayer_file" in result:
+                return result
+
+        # Phase 2: batch WordPress API lookup (one request per site).
+        if not result:
+            entry_link: str | None = getattr(entry, "link", None)
+            if entry_link:
+                slug: str | None = _extract_slug(entry_link)
+                if slug:
+                    base_url: str | None = None
+                    try:
+                        parsed = urlparse(entry.feed.url)
+                        base_url = f"{parsed.scheme}://{parsed.netloc}"
+                    except ValueError:
+                        return result
+
+                    html: str | None = _get_rendered_for_slug(slug, base_url)
+                    if html:
+                        self._search(html, result)
+
+        return result
+
+    @staticmethod
+    def _search(html: str, result: dict[str, str]) -> None:
+        """Search a single HTML source for JWPlayer patterns.
+
+        Args:
+            html: The HTML content to search.
+            result: The result dict to populate.
+        """
+        if "jwplayer_thumbnail" not in result:
+            match = _IMAGE_PATTERN.search(html)
+            if match:
+                result["jwplayer_thumbnail"] = quote(match.group(1).strip(), safe=":/")
+        if "jwplayer_file" not in result:
+            match = _FILE_PATTERN.search(html)
+            if match:
+                result["jwplayer_file"] = quote(match.group(1).strip(), safe=":/")

discord_rss_bot/extensions/runner.py

@@ -0,0 +1,239 @@
+"""Functions for running extensions against entries.
+
+This module is separated from ``__init__.py`` to keep the package's
+public API as pure re-exports (Ruff rule: non-empty-init-module).
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+from typing import TYPE_CHECKING
+
+from discord_rss_bot.extensions.discovery import discover_plugins
+from discord_rss_bot.extensions.discovery import get_registry
+from discord_rss_bot.extensions.storage import EXTENSIONS_TAG
+from discord_rss_bot.extensions.storage import get_enabled_extensions_for_feed
+from discord_rss_bot.extensions.storage import set_enabled_extensions_for_feed
+
+if TYPE_CHECKING:
+    from reader import Entry
+    from reader import Reader
+
+    from discord_rss_bot.extensions.base import FeedExtension
+    from discord_rss_bot.webhook import DiscordWebhook
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+# Auto-discover plugins on first import so the registry is ready.
+discover_plugins()
+
+
+def auto_enable_extensions_for_feed(reader: Reader, feed_url: str) -> list[str]:
+    """Enable extensions whose URL patterns match *feed_url*, if not already enabled.
+
+    Auto-enable only runs when the extensions tag has **never** been explicitly
+    set for this feed.  Once the user visits the Extensions page and saves
+    (even with an empty list), auto-enable is skipped — the user's choice
+    is respected.
+
+    This is called automatically when a feed is created, and lazily on first
+    processing for existing feeds.
+
+    Args:
+        reader: The reader instance.
+        feed_url: The feed URL to match against.
+
+    Returns:
+        The updated list of enabled extensions for this feed.
+    """
+    # If the user has ever saved the Extensions page, respect their choice.
+    if reader.get_tag(feed_url, EXTENSIONS_TAG, None) is not None:
+        return get_enabled_extensions_for_feed(reader, feed_url)
+
+    registry: dict[str, type[FeedExtension]] = get_registry()
+    already_enabled: list[str] = get_enabled_extensions_for_feed(reader, feed_url)
+
+    to_add: list[str] = []
+    for name, cls in registry.items():
+        if name in already_enabled:
+            continue
+        if not getattr(cls, "auto_enable_url_patterns", None):
+            continue
+        for pattern in cls.auto_enable_url_patterns:
+            if re.search(pattern, feed_url):
+                logger.info(
+                    "Auto-enabling extension %r for feed %s (matched pattern %r)",
+                    name,
+                    feed_url,
+                    pattern,
+                )
+                to_add.append(name)
+                break
+
+    if not to_add:
+        return already_enabled
+
+    updated: list[str] = already_enabled + [n for n in to_add if n not in already_enabled]
+    set_enabled_extensions_for_feed(reader, feed_url, updated)
+    return updated
+
+
+def _get_enabled_instances(entry: Entry, reader: Reader) -> list[FeedExtension]:
+    """Return enabled extension instances for the given entry.
+
+    Auto-enables extensions whose URL patterns match the feed URL
+    (lazy initialisation for feeds that were created before the
+    extension system existed).
+
+    Args:
+        entry: The feed entry to process.
+        reader: The reader instance.
+
+    Returns:
+        List of ``FeedExtension`` instances enabled for this entry's feed.
+        Empty list if no extensions are enabled.
+    """
+    feed_url: str = entry.feed.url
+    enabled: list[str] = auto_enable_extensions_for_feed(reader, feed_url)
+    if not enabled:
+        return []
+
+    registry: dict[str, type[FeedExtension]] = get_registry()
+    instances: list[FeedExtension] = []
+
+    for name in enabled:
+        cls = registry.get(name)
+        if cls is None:
+            logger.warning(
+                "Extension %r is enabled for feed %r but was not found in the registry",
+                name,
+                feed_url,
+            )
+            continue
+        instances.append(cls())
+
+    return instances
+
+
+def _process_extension_instance(
+    instance: FeedExtension,
+    entry: Entry,
+    reader: Reader,
+    results: dict[str, str],
+) -> bool:
+    """Process a single extension and update *results* in place.
+
+    The ``try/except`` wrapping belongs in the caller so that this
+    helper stays focused on happy-path logic.
+
+    Args:
+        instance: The extension instance to process.
+        entry: The feed entry to process.
+        reader: The reader instance.
+        results: Accumulator dict to update with variable values.
+
+    Returns:
+        ``True`` if the extension result was processed successfully,
+        ``False`` if the result was not a ``dict`` (and was skipped).
+    """
+    extra: dict[str, str] = instance.process_entry(entry, reader)
+    if not isinstance(extra, dict):
+        logger.warning(
+            "Extension %r returned %s, expected dict — skipping",
+            instance.name,
+            type(extra).__name__,
+        )
+        return False
+    for var_name, var_value in extra.items():
+        if not isinstance(var_value, str):
+            logger.warning(
+                "Extension %r returned non-string value for %r (%s) — coercing",
+                instance.name,
+                var_name,
+                type(var_value).__name__,
+            )
+            results[var_name] = str(var_value)
+        else:
+            results[var_name] = var_value
+    return True
+
+
+def run_extensions(entry: Entry, reader: Reader) -> dict[str, str]:
+    """Run all enabled extensions for the given entry.
+
+    For every enabled extension, all of its ``provides_variables`` are
+    guaranteed to appear in the result dict.  Variables that the
+    extension did not produce (no match in the entry content) are set
+    to an empty string so that ``{{variable}}`` tags are always
+    replaced rather than left as literal text.
+
+    If an extension raises, its error is logged and processing
+    continues with the next one.
+
+    Args:
+        entry: The feed entry to process.
+        reader: The reader instance (used to load per-feed config).
+
+    Returns:
+        Flat dict of ``{variable_name: value}`` pairs.  Always returns a
+        dict (possibly empty if no extensions are enabled).
+    """
+    results: dict[str, str] = {}
+
+    for instance in _get_enabled_instances(entry, reader):
+        # Seed with empty strings so every declared variable is at
+        # least present (prevents literal ``{{var}}`` in output).
+        for var_name in getattr(type(instance), "provides_variables", []):
+            results.setdefault(var_name, "")
+
+        try:
+            _process_extension_instance(instance, entry, reader, results)
+        except Exception:
+            logger.exception(
+                "Extension %r failed while processing entry %s",
+                instance.name,
+                entry.id,
+            )
+
+    return results
+
+
+def run_modify_webhook(
+    webhook: DiscordWebhook,
+    entry: Entry,
+    reader: Reader,
+) -> DiscordWebhook:
+    """Let enabled extensions modify the Discord webhook before sending.
+
+    Each enabled extension's ``modify_webhook()`` is called in order.
+    If an extension raises, its error is logged and the webhook is
+    passed to the next extension unchanged.
+
+    Args:
+        webhook: The fully built webhook payload.
+        entry: The feed entry being processed.
+        reader: The reader instance.
+
+    Returns:
+        The (possibly modified) webhook.
+    """
+    current: DiscordWebhook = webhook
+
+    for instance in _get_enabled_instances(entry, reader):
+        try:
+            current = instance.modify_webhook(current, entry, reader)
+            if current is None:
+                logger.warning(
+                    "Extension %r returned None from modify_webhook — skipping",
+                    instance.name,
+                )
+                current = webhook
+        except Exception:
+            logger.exception(
+                "Extension %r modify_webhook failed for entry %s",
+                instance.name,
+                entry.id,
+            )
+
+    return current

discord_rss_bot/extensions/steam.py

@@ -0,0 +1,276 @@
+"""Steam feed extension: extracts app metadata and provides thumbnail URLs.
+
+Detects feeds from ``store.steampowered.com`` and ``steamcommunity.com``,
+extracts the application ID, and exposes the game's capsule image URL
+as ``{{steam_thumbnail_url}}`` and the app ID as ``{{steam_app_id}}``.
+
+Usage:
+    1. Enable ``steam`` on the feed's Extensions page.
+    2. In embed settings, set Thumbnail URL to ``{{steam_thumbnail_url}}``.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import logging
+from pathlib import Path
+from typing import TYPE_CHECKING
+from typing import ClassVar
+from urllib.parse import parse_qs
+from urllib.parse import urlparse
+
+from reader import ReaderError
+
+from discord_rss_bot.extensions.base import FeedExtension
+from discord_rss_bot.settings import data_dir
+from discord_rss_bot.webhook import DiscordWebhook
+from discord_rss_bot.webhook import WebhookFile
+
+if TYPE_CHECKING:
+    from reader import Entry
+    from reader import Feed
+    from reader import Reader
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+_STEAM_NETLOCS: frozenset[str] = frozenset({"store.steampowered.com", "steamcommunity.com"})
+
+
+def is_steam_url(url: str) -> bool:
+    """Return whether *url* belongs to Steam."""
+    if not url:
+        return False
+    try:
+        parsed = urlparse(url)
+    except ValueError:
+        return False
+    return parsed.netloc.lower().removeprefix("www.") in _STEAM_NETLOCS
+
+
+def _extract_store_app_id(segments: list[str]) -> str | None:
+    """Extract app ID from ``store.steampowered.com`` path segments.
+
+    Returns:
+        The numeric app ID, or ``None`` if no match.
+    """
+    for prefix in (
+        ("feeds", "news", "app"),
+        ("news", "app"),
+        ("app",),
+    ):
+        if len(segments) > len(prefix) and tuple(segments[: len(prefix)]) == prefix:
+            candidate: str = segments[len(prefix)]
+            if candidate.isdigit():
+                return candidate
+    return None
+
+
+def _extract_community_app_id(segments: list[str]) -> str | None:
+    """Extract app ID from ``steamcommunity.com`` path segments.
+
+    Returns:
+        The numeric app ID, or ``None`` if no match.
+    """
+    if len(segments) > 1 and segments[0] in frozenset({"games", "app"}) and segments[1].isdigit():
+        return segments[1]
+    return None
+
+
+def _extract_app_id_from_query(query: str) -> str | None:
+    """Extract app ID from query parameters as a fallback.
+
+    Args:
+        query: The URL query string.
+
+    Returns:
+        The numeric app ID, or ``None`` if no match.
+    """
+    for key in ("appid", "app_id", "appids"):
+        for value in parse_qs(query).get(key, []):
+            if value.strip().isdigit():
+                return value.strip()
+    return None
+
+
+def extract_app_id(url: str) -> str | None:
+    """Extract a Steam application ID from a URL.
+
+    Args:
+        url: The URL to inspect.
+
+    Returns:
+        The numeric app ID if found, otherwise ``None``.
+    """
+    if not url:
+        return None
+    try:
+        parsed = urlparse(url)
+    except ValueError:
+        return None
+
+    netloc: str = parsed.netloc.lower().removeprefix("www.")
+    segments: list[str] = [s for s in parsed.path.split("/") if s]
+
+    if netloc == "store.steampowered.com":
+        return _extract_store_app_id(segments) or _extract_app_id_from_query(parsed.query)
+
+    if netloc == "steamcommunity.com":
+        return _extract_community_app_id(segments) or _extract_app_id_from_query(parsed.query)
+
+    # Query parameter fallback for any other netloc
+    return _extract_app_id_from_query(parsed.query)
+
+
+def _try_read_icon_file(app_id: str) -> WebhookFile | None:
+    """Read a Steam game icon from disk, returning a ``WebhookFile`` or ``None``.
+
+    Does not catch exceptions — the caller handles them.
+
+    Returns:
+        A ``WebhookFile`` with the icon content, or ``None`` if no icon
+        was found on disk.
+    """
+    # Check the old project-relative icons/ directory first,
+    # then fall back to the data directory.
+    old_icon: Path = Path(__file__).resolve().parent.parent.parent / "icons" / f"{app_id}.png"
+    icon_path = old_icon if old_icon.is_file() else Path(data_dir) / "steam_icons" / f"{app_id}.png"
+    if not icon_path.is_file():
+        return None
+    icon_bytes: bytes = icon_path.read_bytes()
+    if not icon_bytes:
+        return None
+    content_hash: str = hashlib.sha256(icon_bytes).hexdigest()[:12]
+    return WebhookFile(
+        filename=f"steam-app-{app_id}-{content_hash}.png",
+        content=icon_bytes,
+    )
+
+
+def get_icon_file_for_app(app_id: str) -> WebhookFile | None:
+    """Return a local Steam game icon file if one exists on disk."""
+    try:
+        return _try_read_icon_file(app_id)
+    except Exception:
+        logger.exception("Failed to read local Steam icon for app %s", app_id)
+        return None
+
+
+def _steam_thumbnail_enabled(reader: Reader, feed: Entry | Feed) -> bool:
+    """Return ``True`` if the "use Steam game icon" toggle is on for *feed*.
+
+    Reads the embed tag directly from the reader to avoid importing
+    ``custom_message`` (which would create a circular dependency).
+    Defaults to ``True`` when the tag is missing or unparseable.
+    """
+    try:
+        embed_tag_raw = reader.get_tag(feed, "embed", None)
+    except ReaderError:
+        return True
+
+    raw_val = True
+    if isinstance(embed_tag_raw, str) and embed_tag_raw.strip():
+        try:
+            embed_data = json.loads(embed_tag_raw)
+        except (json.JSONDecodeError, ValueError):
+            return True
+        if isinstance(embed_data, dict):
+            raw_val = embed_data.get("show_steam_game_icon_in_thumbnail", True)
+    elif isinstance(embed_tag_raw, dict):
+        raw_val = embed_tag_raw.get("show_steam_game_icon_in_thumbnail", True)
+
+    return raw_val  # pyright: ignore[reportReturnType]
+
+
+class SteamExtension(FeedExtension):
+    """Provide ``{{steam_thumbnail_url}}`` and ``{{steam_app_id}}``.
+
+    Extracts the app ID from Steam store and community feeds and makes
+    the game's capsule image URL and app ID available as template variables.
+    """
+
+    name = "steam"
+    description = (
+        "Extracts the Steam app ID from feed URLs and exposes the game's "
+        "capsule image as {{steam_thumbnail_url}}. "
+        "Available as {{steam_app_id}}."
+    )
+    provides_variables: ClassVar[list[str]] = ["steam_thumbnail_url", "steam_app_id"]
+    auto_enable_url_patterns: ClassVar[list[str]] = [
+        r"store\.steampowered\.com",
+        r"steamcommunity\.com",
+    ]
+
+    def process_entry(self, entry: Entry, reader: Reader) -> dict[str, str]:  # ruff:ignore[unused-method-argument]
+        """Extract Steam app metadata from the entry.
+
+        Args:
+            entry: The feed entry to process.
+            reader: The reader instance.
+
+        Returns:
+            Dict with ``steam_thumbnail_url`` and ``steam_app_id`` if
+            this is a Steam feed, otherwise empty dict.
+        """
+        feed_url: str = entry.feed.url or ""
+        if not is_steam_url(feed_url):
+            return {}
+
+        app_id: str | None = extract_app_id(feed_url) or extract_app_id(str(entry.link or ""))
+        if not app_id:
+            return {}
+
+        return {
+            "steam_app_id": app_id,
+            "steam_thumbnail_url": f"https://cdn.cloudflare.steamstatic.com/steam/apps/{app_id}/capsule_sm_120.jpg",
+        }
+
+    def modify_webhook(
+        self,
+        webhook: DiscordWebhook,
+        entry: Entry,
+        reader: Reader,
+    ) -> DiscordWebhook:
+        """Set the embed thumbnail to the Steam game's capsule image.
+
+        Also attaches a locally cached icon file when available.
+        Only applies when ``show_steam_game_icon_in_thumbnail`` is enabled
+        in the feed's embed settings.
+
+        Args:
+            webhook: The fully built webhook payload.
+            entry: The feed entry being processed.
+            reader: The reader instance.
+
+        Returns:
+            The (possibly modified) webhook.
+        """
+        feed_url: str = entry.feed.url or ""
+        if not is_steam_url(feed_url):
+            return webhook
+
+        if not _steam_thumbnail_enabled(reader, entry.feed):
+            return webhook
+
+        app_id: str | None = extract_app_id(feed_url) or extract_app_id(str(entry.link or ""))
+        if not app_id:
+            return webhook
+
+        # Only modify embed thumbnail when there IS an embed (not in text mode).
+        embeds = webhook.json.get("embeds", [])
+        if not isinstance(embeds, list) or not embeds:
+            return webhook
+
+        # Use a local icon file if available (better quality), otherwise CDN.
+        icon_file: WebhookFile | None = get_icon_file_for_app(app_id)
+        if icon_file:
+            webhook.add_file(file=icon_file.content, filename=icon_file.filename)
+            thumbnail_url: str = f"attachment://{icon_file.filename}"
+        else:
+            thumbnail_url = f"https://cdn.cloudflare.steamstatic.com/steam/apps/{app_id}/capsule_sm_120.jpg"
+
+        embed = embeds[0]
+        if isinstance(embed, dict):
+            embed["thumbnail"] = {"url": thumbnail_url}
+
+        return webhook

discord_rss_bot/extensions/storage.py

@@ -0,0 +1,68 @@
+"""Per-feed storage of enabled extension lists.
+
+Enabled extensions for a feed are stored as a JSON list in the reader
+tag ``"extensions"``.  An empty list or missing tag means no extensions
+are active for that feed.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+from typing import TYPE_CHECKING
+from typing import cast
+
+if TYPE_CHECKING:
+    from reader import Reader
+    from reader.types import JSONType
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+EXTENSIONS_TAG: str = "extensions"
+
+
+def get_enabled_extensions_for_feed(reader: Reader, feed_url: str) -> list[str]:
+    """Return the list of enabled extension names for the given feed.
+
+    Args:
+        reader: The reader instance.
+        feed_url: The feed URL.
+
+    Returns:
+        List of extension names enabled for this feed.  Never ``None``.
+    """
+    raw = reader.get_tag(feed_url, EXTENSIONS_TAG, None)
+    if raw is None:
+        return []
+
+    if isinstance(raw, list):
+        return [str(name) for name in raw]
+
+    if isinstance(raw, str):
+        if not raw.strip():
+            return []
+        try:
+            parsed = json.loads(raw)
+            if isinstance(parsed, list):
+                return [str(name) for name in parsed]
+        except (json.JSONDecodeError, ValueError):
+            logger.warning("Failed to parse extensions tag for feed %r: %r", feed_url, raw)
+            return []
+
+    logger.warning("Unexpected type for extensions tag on feed %r: %s (%s)", feed_url, type(raw).__name__, raw)
+    return []
+
+
+def set_enabled_extensions_for_feed(
+    reader: Reader,
+    feed_url: str,
+    extension_names: list[str],
+) -> None:
+    """Persist a list of enabled extension names for a feed.
+
+    Args:
+        reader: The reader instance.
+        feed_url: The feed URL.
+        extension_names: Extension names to enable (empty = disable all).
+    """
+    reader.set_tag(feed_url, EXTENSIONS_TAG, cast("JSONType", extension_names))

discord_rss_bot/extensions/wordpress.py

@@ -0,0 +1,116 @@
+"""WordPress REST API extension: fetch post content with scripts intact.
+
+Shares the batch cache with ``jwplayer_thumbnail`` so only one API call
+is made per site regardless of which extension is enabled.
+
+Provides ``{{wp_content}}`` (formatted) / ``{{wp_content_raw}}`` (raw HTML),
+``{{wp_excerpt}}`` (formatted) / ``{{wp_excerpt_raw}}`` (raw HTML),
+``{{wp_jwplayer_thumbnail}}`` and ``{{wp_jwplayer_file}}``.
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+from typing import TYPE_CHECKING
+from typing import ClassVar
+from urllib.parse import quote
+from urllib.parse import urlparse
+
+from discord_rss_bot.extensions.base import FeedExtension
+from discord_rss_bot.extensions.jwplayer_thumbnail import _extract_slug
+from discord_rss_bot.extensions.jwplayer_thumbnail import _get_post_data_for_slug
+from discord_rss_bot.html_format import format_entry_html_for_discord
+
+if TYPE_CHECKING:
+    from reader import Entry
+    from reader import Reader
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+_IMAGE_PATTERN: re.Pattern[str] = re.compile(
+    r'image:\s*["\']([^"\']+)["\']',
+    re.IGNORECASE,
+)
+
+_FILE_PATTERN: re.Pattern[str] = re.compile(
+    r'file:\s*["\']([^"\']+\.\w+)["\']',
+    re.IGNORECASE,
+)
+
+
+class WordPressExtension(FeedExtension):
+    """Fetch post content from the WordPress REST API.
+
+    Retrieves full post HTML so that ``<script>`` blocks survive.
+    Shares the batch cache with ``jwplayer_thumbnail``.
+    """
+
+    name = "wordpress"
+    description = (
+        "Fetches post HTML from the WordPress REST API (batched). "
+        "Extracts JWPlayer URLs. "
+        "Available as {{wp_content}} (formatted) / {{wp_content_raw}} (raw HTML), "
+        "{{wp_excerpt}} (formatted) / {{wp_excerpt_raw}} (raw HTML), "
+        "{{wp_jwplayer_thumbnail}} and {{wp_jwplayer_file}}."
+    )
+    provides_variables: ClassVar[list[str]] = [
+        "wp_content",
+        "wp_content_raw",
+        "wp_excerpt",
+        "wp_excerpt_raw",
+        "wp_jwplayer_file",
+        "wp_jwplayer_thumbnail",
+    ]
+
+    def process_entry(self, entry: Entry, _reader: Reader) -> dict[str, str]:
+        """Return WordPress post data as template variables.
+
+        Uses the shared batch cache (single API call per site, not per entry).
+        Exposes the full post content and excerpt, plus any JWPlayer URLs
+        found within the content.
+
+        Args:
+            entry: The feed entry to process.
+
+        Returns:
+            Dict with ``wp_content`` (formatted) / ``wp_content_raw`` (raw), ``wp_excerpt``
+            (formatted) / ``wp_excerpt_raw`` (raw), ``wp_jwplayer_thumbnail``
+            and ``wp_jwplayer_file`` if found, otherwise empty dict.
+        """
+        entry_link: str = entry.link or ""
+        slug: str | None = _extract_slug(entry_link)
+        if not slug:
+            return {}
+
+        try:
+            parsed = urlparse(entry.feed.url)
+            base_url: str = f"{parsed.scheme}://{parsed.netloc}"
+        except ValueError:
+            return {}
+
+        post_data: dict[str, str] | None = _get_post_data_for_slug(slug, base_url)
+        if not post_data:
+            return {}
+
+        result: dict[str, str] = {}
+
+        # Expose full content and excerpt from the WordPress REST API.
+        content_html: str = post_data.get("content", "")
+        if content_html:
+            result["wp_content"] = format_entry_html_for_discord(content_html)
+            result["wp_content_raw"] = content_html
+        excerpt_html: str = post_data.get("excerpt", "")
+        if excerpt_html:
+            result["wp_excerpt"] = format_entry_html_for_discord(excerpt_html)
+            result["wp_excerpt_raw"] = excerpt_html
+
+        # Extract JWPlayer URLs from the full content HTML.
+        image_match = _IMAGE_PATTERN.search(content_html)
+        if image_match:
+            result["wp_jwplayer_thumbnail"] = quote(image_match.group(1).strip(), safe=":/")
+        file_match = _FILE_PATTERN.search(content_html)
+        if file_match:
+            result["wp_jwplayer_file"] = quote(file_match.group(1).strip(), safe=":/")
+
+        return result

discord_rss_bot/extensions/youtube.py

@@ -0,0 +1,110 @@
+"""YouTube feed extension: extracts video IDs from YouTube video URLs.
+
+Detects feeds from ``youtube.com/feeds/videos.xml`` and exposes the
+video ID as ``{{youtube_video_id}}`` and the full embeddable URL as
+``{{youtube_embed_url}}``.
+
+Usage:
+    1. Enable ``youtube`` on the feed's Extensions page.
+    2. Use ``{{youtube_video_id}}`` or ``{{youtube_embed_url}}`` in
+       your message template or embed.
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+from typing import TYPE_CHECKING
+from typing import ClassVar
+from urllib.parse import parse_qs
+from urllib.parse import urlparse
+
+from discord_rss_bot.extensions.base import FeedExtension
+
+if TYPE_CHECKING:
+    from reader import Entry
+    from reader import Reader
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+# Various YouTube URL formats for video ID extraction.
+_VIDEO_ID_PATTERNS: tuple[re.Pattern[str], ...] = (
+    re.compile(r"(?:v|vi|video)=([\w-]+)"),
+    re.compile(r"(?:youtu\.be|youtube\.com/embed)/([\w-]+)"),
+    re.compile(r"^([\w-]{11})$"),
+)
+
+
+def extract_youtube_video_id(url: str) -> str | None:
+    """Extract a YouTube video ID from a URL.
+
+    Args:
+        url: The YouTube video URL.
+
+    Returns:
+        The video ID if found, otherwise ``None``.
+    """
+    if not url:
+        return None
+    try:
+        parsed = urlparse(url)
+    except ValueError:
+        return None
+
+    # Try query parameter first
+    for value in parse_qs(parsed.query).get("v", []):
+        if value.strip():
+            return value.strip()
+
+    # Try path-based patterns
+    for pattern in _VIDEO_ID_PATTERNS:
+        match = pattern.search(url)
+        if match:
+            return match.group(1)
+
+    return None
+
+
+def is_youtube_feed_url(feed_url: str) -> bool:
+    """Return whether a feed URL is a YouTube video feed."""
+    return "youtube.com/feeds/videos.xml" in (feed_url or "")
+
+
+class YouTubeExtension(FeedExtension):
+    """Extract YouTube video IDs from entry links.
+
+    Makes ``{{youtube_video_id}}`` and ``{{youtube_embed_url}}``
+    available for YouTube channel feeds.
+    """
+
+    name = "youtube"
+    description = (
+        "Extracts the YouTube video ID from entry links. Available as {{youtube_video_id}} and {{youtube_embed_url}}."
+    )
+    provides_variables: ClassVar[list[str]] = ["youtube_video_id", "youtube_embed_url"]
+    auto_enable_url_patterns: ClassVar[list[str]] = [
+        r"youtube\.com/feeds/videos\.xml",
+    ]
+
+    def process_entry(self, entry: Entry, _reader: Reader) -> dict[str, str]:
+        """Extract the video ID from the entry link.
+
+        Args:
+            entry: The feed entry to process.
+
+        Returns:
+            Dict with ``youtube_video_id`` and ``youtube_embed_url`` if
+            this is a YouTube feed, otherwise empty dict.
+        """
+        if not is_youtube_feed_url(entry.feed.url):
+            return {}
+
+        entry_link: str = entry.link or ""
+        video_id: str | None = extract_youtube_video_id(entry_link)
+        if not video_id:
+            return {}
+
+        return {
+            "youtube_video_id": video_id,
+            "youtube_embed_url": f"https://www.youtube.com/watch?v={video_id}",
+        }