diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..94f24c5 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,94 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this repo is + +Umbrella of plugins for [maubot](https://github.com/maubot/maubot), the +Matrix bot framework. The maubot daemon itself is provisioned by +`~/dev/deploy/server-deploy/modules/services/maubot.nix` on the castle +hosts (currently cfaun). **This repo is the canonical place for +plugin code; do not edit plugin code from a server-deploy session.** + +Layout: one subdir per plugin, each containing `maubot.yaml` + the +Python sources. Built `.mbp` files are gitignored and live next to +their source dir after `zip -j ../.mbp ...`. + +## Where to find context + +- **Per-plugin docs:** `/README.md` covers commands, schema, + quirks, etc. +- **Repo-wide build/upload flow:** root `README.md`. +- **Community-organizer protocol spec:** `docs/community-organizer-spec.md` + defines the vocabulary, event shapes (NIP-52 + NIP-72), lifecycle + states, and tag conventions shared across the `tracker` plugin, the + `inky-impression` renderer, and any future surface (webapp form, CLI, + etc.). **Read this before changing any verb behavior, tag shape, or + event structure** — it's the contract other implementations (and + other communities) build against. Don't redesign these in plugin code; + update the spec first. +- **Maubot patterns and footguns:** `~/dev/CLAUDE.md` under + "Maubot plugin development" — covers `database_type` semantics, + `@command.new` vs `@command.passive`, multi-line caveats, etc. + Read that before adding new plugins or non-trivial command + handlers; there are several silent-data-loss footguns. +- **Daemon configuration / nginx / sops secrets:** in + `~/dev/deploy/server-deploy/modules/services/maubot.nix` and the + host's `sops.nix` / `secrets.yaml`. Don't edit those from here — + cross-repo coordination point. + +## Iteration loop + +Standard cycle when modifying an existing plugin: + +```sh +cd / +# edit +$EDITOR journal.py +# bump version in maubot.yaml so the UI surfaces the new build +$EDITOR maubot.yaml +# zip +rm -f ../.mbp +zip -j ../.mbp maubot.yaml *.py +# upload via maubot UI → Plugins → click existing → upload +# then click into the instance and hit SAVE (toggling Enabled +# alone doesn't persist — easy facepalm) +``` + +For brand-new plugins, also create the bot's Matrix account first +(via Continuwuity registration token from the admin room — `!admin +token issue --once`, then register through Element), add it as a +Client in the maubot UI, then create an Instance binding the plugin +to that client. Existing example: `@journalbot:ariege.io` for +`journal/`. + +### `maubot.yaml` conventions for new plugins + +- **`id`: use the `dev.aiolabs.` namespace.** Maubot keys plugins + by this string in its DB and on disk (`/var/lib/maubot/plugins/ + dev.aiolabs.-v.mbp`), so it must be globally unique across + every maubot ecosystem — reverse-DNS is the convention (cf. + `xyz.maubot.reminder`). Reserving `dev.aiolabs.*` for our plugins + keeps ids predictable and rename-safe. Changing the id later is a + fork, not a rename: every existing instance gets orphaned. +- **`database_type:` if you need storage → `asyncpg`** (or `sqlalchemy` + for legacy code). That field names the API style, NOT the storage + backend. `sqlite` or `postgres` there fails at instance start with + `RuntimeError: Unrecognized database type ...` — the storage backend + is chosen at the daemon level via `plugin_databases.{sqlite,postgres}` + in the maubot config and is independent of what the plugin declares. + +## Commits + +Direct commits to `main` are the working convention while this stays +single-maintainer. Conventional-commits style (`feat():`, +`fix():`, `docs:`). Tag at deploy-ready boundaries if/when +the repo ever needs publishable releases — for now, the maubot UI +reads versions from the uploaded `.mbp`, not from git tags. + +## Pyright noise + +`maubot` / `mautrix` imports are unresolved in pyright + `.subcommand` +"unknown attribute" warnings on decorator-extended functions are +expected — the SDK is heavily dynamic and pyright can't introspect +the decorators. Ignore these; they don't reflect runtime behavior. diff --git a/README.md b/README.md new file mode 100644 index 0000000..551177b --- /dev/null +++ b/README.md @@ -0,0 +1,71 @@ +# maubot-plugins + +Umbrella for [maubot](https://github.com/maubot/maubot) plugins used by +the aiolabs / Château du Faune Matrix stack. The maubot daemon itself +is provisioned via `server-deploy/modules/services/maubot.nix` on the +castle hosts; the actual plugin code lives here. + +## Plugins + +| Plugin | Purpose | +|---|---| +| [`journal/`](./journal/) | Farm-journal bot. `!journal ` records what you did, scoped per-user/room/timestamp. `!journal show [@user]` and `!journal today` query back. | +| [`tracker/`](./tracker/) | Community-organizer bot. `!add` / `!task` / `!sidequest` / `!remind` / `!done` / `!list` / `!setup`. Implements the [Community Organizer spec](./docs/community-organizer-spec.md) — per-room shortcuts, 5-level priority, rules-based inbox classifier. | +| [`wiki/`](./wiki/) | Docs-lookup bot. `!ask ` / `!doc ` / `!wiki [refresh\|status]`. Points at any Quartz-rendered docs site (default: `docs.ariege.io`), full-text searches the corpus, replies with snippets + links. Internal-network deployment posture — works during WAN outages. | + +## Community Organizer protocol + +`docs/community-organizer-spec.md` defines the protocol the plugins in +this repo (and companion renderers like +[`inky-impression`](https://git.atitlan.io/aiolabs/inky-impression)) use +to coordinate community life — tasks, journals, reminders, shopping +lists — over Matrix capture + Nostr storage. Designed to be adopted by +other communities; reuses NIP-52 + NIP-72 instead of inventing new +event kinds. Read it before changing verb behavior or event shapes in +any plugin. + +## Building a plugin + +A `.mbp` is just a zip containing `maubot.yaml` + the plugin's Python +modules at the root. No special tooling needed: + +```sh +cd / +zip -j ../.mbp maubot.yaml *.py +``` + +(`-j` strips the directory prefix so files land at the zip root.) + +## Uploading / iterating + +1. Open the maubot UI (e.g. `https://maubot./_matrix/maubot/`). +2. **Plugins → +** (first time) or click the existing plugin → upload + the new `.mbp`. Maubot keys plugins by `id`; uploading a new + `version` of the same `id` replaces the old one. +3. **Hit Save** on the affected instance after upload — toggling + Enabled without Save will revert. Easy facepalm. + +Bump `version:` in `maubot.yaml` for every meaningful change so the +maubot UI surfaces it cleanly and old `.mbp` files in +`/var/lib/maubot/plugins/` aren't ambiguous. + +## Bot account convention + +Each plugin attaches to a Matrix client (a regular Matrix user account +controlled by maubot). For the journal bot: `@journalbot:ariege.io`. +Bot accounts are created the same way as any user — issue a +registration token from the Continuwuity admin room +(`!admin token issue --once`) and register through Element, then add +the client in the maubot UI. + +Invite the bot to whichever rooms it should serve via `/invite +@:` — maubot's autojoin handles new invites that arrive +after the client's sync loop is up. + +## Patterns + gotchas + +Maubot-specific patterns (command decorators, multi-line caveats, +`database_type` in `maubot.yaml`, etc.) live in `~/dev/CLAUDE.md` +under "Maubot plugin development". Read that before writing a new +plugin — there are several footguns that look fine but silently lose +data. diff --git a/docs/community-organizer-spec.md b/docs/community-organizer-spec.md new file mode 100644 index 0000000..4174e8d --- /dev/null +++ b/docs/community-organizer-spec.md @@ -0,0 +1,917 @@ +# Community Organizer Spec + +**Status:** Draft 0.1 +**Authors:** aiolabs / Château du Faune (Ariège, France) +**License:** CC0 — adopt, fork, remix freely + +A protocol for coordinating the day-to-day operations of a physical +community — tasks, reminders, shopping lists, journals, infrastructure +backlog — over chat (capture) and an open event store (canonical +record), so the same data can drive multiple display surfaces (eink +panels in shared spaces, web dashboards, mobile, etc.). + +This document is the contract. Implementations are interchangeable: a +Matrix maubot plugin, a Discord bot, a CLI, a web form, an SMS gateway +— all can produce the same events and consume from the same store. + +--- + +## 0. Why a spec, not just an app + +Every community organizing itself eventually invents some flavor of +"the list of stuff we need to do, the log of what we did, the things +we need to buy". Tools rot, maintainers move on, the data gets stuck +in proprietary stores. Communities re-implement the same wheel. + +This spec puts the *data* on open protocols (Nostr, NIP-52, NIP-72) so +that: + +- Multiple input surfaces can write to the same record +- Multiple output surfaces (displays, dashboards, mobile) can consume + without coordination +- A community's organizational memory survives any single tool +- Other communities can adopt the same shape and benefit from + interoperable implementations +- Self-hosting is achievable end-to-end — no SaaS lock-in + +The spec deliberately reuses existing standards (RFC 5545 VTODO, +NIP-52, NIP-72, ActivityStreams vocabulary) rather than inventing new +ones. If we need something a NIP doesn't cover, we lean on tag +conventions before proposing a new kind. + +--- + +## 1. Prior art + +A community-organizer-via-chat-bot isn't new territory — there's a +lot to pull from rather than reinvent. Each item below names what +we're borrowing and why it maps to our context. + +1. **GTD (Getting Things Done, Allen, 2001)** — the + capture/clarify/organize/reflect/engage workflow. The observation + that *"clean the wound on the dog should be a task, not a journal + entry"* is literally the GTD "clarify" step. Capture is + frictionless and unbucketed; clarification happens on a cadence. + This is the conceptual backbone for an `!add`-style capture surface + that later sorts into typed buckets (rules → LLM → human override). + +2. **Bullet Journal (Carroll)** — daily log + signifiers (•, ○, ☐) + + "migration" (moving incomplete items to the next page). Maps + cleanly to a `!journal` daily-log + a weekly review that migrates + open items into tasks. The persistence-across-time framing ("new + person arriving can reference") is exactly Bullet Journal's + migration archive. + +3. **iCalendar VTODO (RFC 5545)** — battle-tested TODO data model: + `STATUS` (NEEDS-ACTION / IN-PROCESS / COMPLETED / CANCELLED), + `PRIORITY`, `DUE`, `COMPLETED`, `CATEGORIES`. NIP-52 is loosely + modeled on iCalendar; aligning our task shape to VTODO gives us + interop with the entire CalDAV ecosystem essentially for free. + +4. **ActivityStreams 2.0 / ActivityPub** — + Actor-Verb-Object-Context vocabulary. The semantic primitive is + "actor did verb on object in context", which is what every + `!command` is. Established vocab. + +5. **MagicMirror² + Home Assistant Lovelace** — proven *modular + display* pattern: data sources feed a render graph, plugins/cards + subscribe. The lesson for output surfaces: don't bake a layout, + expose a scene/card plugin API. + +6. **CalDAV/CardDAV** — the architectural lesson: *standardized data + on a shared store enables a plurality of input/output clients*. + The store and schema are the contract; clients are + interchangeable. + +7. **Linear's Project / Cycle / Issue triad + Loomio's + proposal/decision flow** — modern open-source patterns for triage + and decision-making at the community level. Linear's status + lifecycle (Triage → Backlog → Todo → In Progress → Done → + Canceled) is borrowable. + +8. **Logseq / Obsidian daily-notes** — capture appended to a daily + log, structured later via block refs/tags. Validates the "log + first, structure second" posture and shows how tags become the + organizational graph. + +9. **maubot/reminder, maubot/rss** — Matrix-side prior art for + command shape, recurring schedules, room-scoped subscriptions. + Worth reading before writing handlers. + +10. **NIP-52 + NIP-72 (Nostr)** — Nostr-native primitives for + calendar events and community scoping. Reusing these instead of + inventing new kinds is what makes the protocol portable to anyone + with a Nostr relay. + +The generalization insight: *the bot is one reference implementation +of a protocol-level contract*. The contract (event shapes + community +model + vocabulary) is what other groups adopt; their implementations +can be different bots, different UIs, different displays. + +--- + +## 2. Roles + +| Role | Description | +|---|---| +| **Member** | A human in the community. Identified by chat handle (e.g. MXID for Matrix). May or may not have their own Nostr identity. | +| **Bot** | An automated process holding one Nostr keypair, capturing input from a chat surface and writing to the relay. The bot acts as a *trusted publisher* on behalf of members in a single trust boundary (one chat room = one community). | +| **Community** | A trust boundary. Defined by a NIP-72 community definition event (kind 34550). Maps 1:1 to a chat room. Members of the room can record, edit, query within the community. | +| **Relay** | A Nostr relay (or set of relays) holding the canonical event log. May be public, community-private (auth-gated), or both. | +| **Renderer** | Any output surface that subscribes to the relay and displays events: eink panel, web dashboard, mobile push, voice assistant, etc. | + +--- + +## 3. Vocabulary + +### 3.1 Universal verbs + +Every conformant implementation MUST recognize these verbs. Verbs are +the *surface form* — the underlying event is the same regardless of +whether the user typed `!task #buy milk` or `!buy milk` (via a +configured shortcut) or `!add we need milk` (LLM-classified into +`task #buy`). + +| Verb | Purpose | Underlying kind | +|---|---|---| +| `!add ` | Frictionless inbox capture; classifier sorts later | 31922 with `["t", "unclassified"]` initially; tag mutated on classification | +| `!task [#tag…]` | Record an actionable task | 31922 (date-based) or 31923 (timed) with `["t", "task"]` + user tags | +| `!sidequest [#tag…]` | Record an optional / passion-project item, distinct from the community's core tasks | 31922 with `["t", "sidequest"]` + user tags | +| `!journal ` | Past-tense log entry; append-only; never "done" | 31922 with `["t", "journal"]` | +| `!remind ` | Time-bound prompt that fires a chat ping at `due_at` | 31923 with `["t", "remind"]` + `start` tag | +| `!done ` | Close a task or sidequest | 31925 status event with `accepted` / equivalent | +| `!list [type] [#tag…]` | Query recent items in current community | read-only | +| `!setup` | Per-community configuration (verb shortcuts, publish posture, etc.) | writes to community config (see §6) | + +**Note on `!sidequest`:** Sidequests are intentionally separate from +tasks — they're the passion-project / experimental / "if-you-have-time" +items a community wants to track without cluttering the core task list. +Same lifecycle as tasks (open → done) but renderers SHOULD surface them +in a distinct view and exclude them from default `!list task` results. +Lets communities keep day-to-day operations clean while still capturing +the "wouldn't it be cool to…" backlog. + +### 3.2 Lifecycle states + +Aligned with RFC 5545 VTODO `STATUS` values: + +| State | VTODO equivalent | Notes | +|---|---|---| +| `unclassified` | NEEDS-ACTION | Inbox; awaiting classification | +| `open` | NEEDS-ACTION | Classified, not started | +| `in_progress` | IN-PROCESS | Someone's actively working it | +| `done` | COMPLETED | Closed via `!done` | +| `canceled` | CANCELLED | Closed without completion | + +Journal entries do NOT participate in this lifecycle — they're +append-only. `!done` on a journal entry is undefined behavior; +implementations SHOULD reject it. + +Sidequests use the same lifecycle as tasks. + +### 3.3 Tag conventions + +NIP-52 events use `["t", ""]` tags for free-form categorization. +This spec uses: + +- A **kind tag** (exactly one): `task`, `sidequest`, `journal`, + `remind`, `unclassified` — identifies the entry's role in the + lifecycle. +- **Domain tags** (zero or more): `buy`, `steward`, `kitchen`, + `animals`, `harvest`, etc. — community-specific categories. +- A **priority tag** (zero or one): see §3.3.1. +- A **classification-source tag** (exactly one): `src:explicit`, + `src:shortcut`, `src:rules`, `src:llm`, `src:manual`. Records how + the entry was bucketed; useful for audit and for tuning the + classifier. + +#### 3.3.1 Priority scale + +Five colloquial levels for everyday capture, RFC 5545 VTODO +`PRIORITY` compatible (VTODO is `1..9` with `1` highest): + +| Tag | Label | Meaning | VTODO PRIORITY | +|---|---|---|---| +| `priority:1` | **urgent** | Drop everything; time-critical | 1 | +| `priority:2` | **crucial** | Critical importance; do this week | 3 | +| `priority:3` | **important** | Matters; do when possible | 5 | +| `priority:4` | **future** | Backlog; someday/maybe | 7 | +| `priority:5` | **frequent/ongoing** | Recurring obligation that's never permanently "done" | 9 | + +Renderers MAY surface labels instead of numbers; the on-the-wire tag +is `priority:N` for sort stability. + +**Note on `priority:5` (frequent/ongoing):** This level is a v1 +shorthand for items that recur on a cadence (daily chicken feeding, +weekly market run, monthly equipment check). A proper recurrence +model (RFC 5545 `RRULE` equivalent) is in §11 open questions. +Until then, `!done` on a `priority:5` item is permitted but treated +as "done for this cycle" — the user re-captures next time. Renderers +SHOULD surface these in a distinct "recurring" view so they don't +mask urgent work. + +Example complete tag set for a `!buy door handles` typed via the +`buy` shortcut configured for the room: + +``` +["t", "task"] +["t", "buy"] +["t", "src:shortcut"] +``` + +### 3.4 Per-community shortcuts + +Communities declare their own surface verbs that expand to a +universal verb + tags. Example (Ariège): + +```yaml +verbs: + buy: { kind: task, tags: [buy] } + steward: { kind: task, tags: [steward, priority:4] } + kitchen: { kind: task, tags: [kitchen] } + urgent: { kind: task, tags: [priority:1] } # shortcut for a priority level + chores: { kind: task, tags: [priority:5] } # recurring/ongoing items +``` + +Priority shortcuts (`!urgent fix the leak`) expand to +`!task #priority:1 fix the leak`. Same per-community config +mechanism — no special-casing of priority verbs. + +A bakery co-op's config might declare `harvest`, `market`, `brewery` +instead. **The spec mandates the universal verbs; per-community +shortcuts are configuration, not protocol.** This means a renderer +written for one community works for another without code changes — +it just filters on different tags. + +--- + +## 4. Event shapes + +All events follow NIP-52 (calendar events) and are scoped to a +community via NIP-72 `a`-tag. + +### 4.1 Task / journal / unclassified entry (kind 31922) + +Date-based event (no specific time). Used for items where the +capture moment matters more than a scheduled time. + +```json +{ + "kind": 31922, + "pubkey": "", + "created_at": 1716559200, + "tags": [ + ["d", ""], + ["title", ""], + ["t", "task"], + ["t", "buy"], + ["t", "src:shortcut"], + ["a", "34550::"], + ["client", "", ""], + ["author", ""] + ], + "content": "door handles from Laura", + "id": "", + "sig": "" +} +``` + +Required tags: +- `d` — replaceable-event identifier; unique within `(kind, pubkey)`. + Recommend `:` or a ULID. +- `title` — short summary for renderers; ≤80 chars. +- exactly one kind tag from §3.3 +- `a` — NIP-72 community reference (see §5). +- `author` — original chat handle for attribution. The bot signs as + itself, but human accountability needs to be visible. + +Optional tags: +- Domain tags, priority tags, classification-source tag (§3.3) +- `client` — bot identifier + version +- `e` — references to related events (e.g. a journal entry that + spawned a follow-up task) + +### 4.2 Timed entry (kind 31923) + +Used for `!remind` and any task with a specific time. Adds: + +```json +"tags": [ + ... + ["start", ""], + ["end", ""] // optional +] +``` + +If only `start` is present, the entry is a point-in-time prompt. + +### 4.3 Status / done (kind 31925) + +NIP-52 RSVP event. Used by `!done` to close a task: + +```json +{ + "kind": 31925, + "tags": [ + ["d", ""], + ["a", "31922::"], + ["status", "accepted"], + ["a", "34550::"] + ], + "content": "" +} +``` + +`status` values per NIP-52: `accepted` (we map this to `done`), +`tentative` (we map to `in_progress`), `declined` (we map to +`canceled`). + +### 4.4 Deletions (kind 5) + +Standard NIP-09 deletion event. Used to retract an entry recorded in +error. Renderers MUST honor deletions. + +--- + +## 5. Community model (NIP-72) + +Every community is identified by a **kind 34550 community definition +event** published by the community's founder (or the bot on their +behalf at `!setup` time). The community is referenced by an `a`-tag: + +``` +a = "34550::" +``` + +All entries scoped to a community MUST include this `a`-tag. +Renderers filter on it to surface the right community's data. + +### 5.1 Community definition event + +```json +{ + "kind": 34550, + "pubkey": "", + "tags": [ + ["d", ""], + ["name", "Château du Faune — Animals"], + ["description", "Daily chores and infra for alpacas, hens, ducks, LGDs"], + ["image", ""], + ["moderator", "", ""], + ["relay", "", "author"], + ["relay", "", "requests"] + ], + "content": "" +} +``` + +### 5.2 Chat room ↔ community mapping + +A conformant bot SHOULD maintain a 1:1 mapping from chat room → NIP-72 +community. The room IS the trust boundary — anyone in the room can +record/edit/close in the community. + +Lazy creation is permitted: the first `!setup` (or first capture +event) in a room MAY create the community definition. + +### 5.3 Publish posture + +Each community declares its publish posture (configurable via +`!setup`): + +| Posture | Behavior | +|---|---| +| `matrix-only` | Bot records to local cache; does NOT publish to any relay. Use for sensitive rooms. | +| `community-relay` | Bot publishes to the community's own relay (typically authenticated, member-only). | +| `public-relays` | Bot publishes to all configured public relays. Use for outward-facing rooms (events, marketplace). | + +The posture is a hint to the publisher — the spec does not mandate +relay authentication; that's the relay operator's choice. + +--- + +## 6. Inbox and classification + +The `!add` verb supports frictionless capture for low-literacy users +who shouldn't have to remember verb taxonomy. Entries land +immediately as `kind: unclassified` and progress through: + +``` +captured (unclassified) → rules classifier → [classified] OR [inbox] + ↓ + LLM classifier (optional) → [classified] + ↓ + manual review → [classified] +``` + +**Classification is never a gatekeeper.** The entry is always +recorded immediately. Classification mutates tags asynchronously and +is reversible by: +- a chat reaction on the bot's ack message (UI-defined) +- the `!sort` or `!reclassify` verb (implementation-defined) +- direct edit by a moderator + +The classification source MUST be recorded in the +`src:` tag. This preserves audit trail and +enables classifier tuning. + +### 6.1 Conformance levels for classifiers + +- **Level 0** — no classifier. Entries captured via `!add` stay + `unclassified` forever until human-sorted. Acceptable for small + communities. +- **Level 1** — rules-based. Keyword/regex matchers cover the + obvious cases (e.g. "buy/purchase/order" → `task #buy`; past-tense + verbs → `journal`). Transparent, deterministic, no external + dependency. +- **Level 2** — LLM fallback. Anything the rules don't catch is + classified by a small local model (or external API). Recommended: + local-first to preserve the self-hostable property. + +--- + +## 7. Permission model + +### 7.1 v1 (trust-the-bot) + +- The bot owns one Nostr keypair. +- All events for the community are signed by the bot. +- Human attribution is carried in the `author` tag (originating chat + handle). +- Trust derives from chat-room membership: if you're in the room, + you can record/edit/close in the community. +- Relay operators MAY auth-gate (require NIP-42) to prevent + community-scoped writes from non-members. Out of band coordination. + +### 7.2 Per-user signing (v2; design-for-it now) + +Once external-identity binding is in place (see §7.3), the bot SHOULD +sign events as the originating user, not as itself. The chat-room → +community mapping stays the same; only the signer changes. + +Implementations MUST design their signing layer as an abstraction so +per-user signing plugs in without refactor. Recommended interface +(adopted from the LNbits reference implementation, see §12): + +| Signer | Holds | Where it signs | Lifecycle role | +|---|---|---|---| +| `BotSigner` | One server-side keypair shared across all events | Server-side, in-process | The v1 fallback and the unbound-user fallback | +| `RemoteBunkerSigner` | A NIP-46 connection token + bunker URL + scoped perms (per-user) | RPC over relay (kind 24133) to a sidecar bunker | **Steady state** — what every bound user ends up on | +| `ClientSideOnlySigner` | Nothing — sentinel meaning "user signs in their own client" | Not the bot; events for this user come in via subscription from elsewhere | Sovereignty escape (NIP-41) — user took ownership | +| `LocalSigner` | A user's encrypted-at-rest keypair on the bot's host | Server-side, in-process | **Transitional** — migration helper only; never the long-term home for a key | + +**The steady-state architecture is the operator-IdP-with-sidecar-bunker +pattern.** The bot (or any other client of the identity provider) holds +zero nsec material; a separate bunker process on the same host (e.g. +`nak bunker`) holds every user's target key and signs on RPC. The bot +authenticates to the bunker with a per-user NIP-46 connection token +issued at binding time (§7.3), scoped to only the kinds the bot needs +to sign. Compromise of the bot leaks the scoped tokens; revoking them +at the bunker side is one RPC and doesn't disturb the target key or +any other client device the user has authorized. + +`LocalSigner` is included in the abstraction for two narrow reasons: +(a) it's how identity providers migrate existing plaintext-nsec rows +into the bunker safely, and (b) it lets adopters who don't (yet) run a +bunker fall back to encrypt-at-rest in-process signing. Once a bunker +is available, the migration drains every `LocalSigner` row into it and +flips them to `RemoteBunkerSigner`. New accounts MUST NOT default to +`LocalSigner`. + +For each captured message the bot: + +1. Looks up the originating chat handle (e.g. MXID) in its binding + table (§7.3) — yielding a connection token + bunker URL + pubkey + (or `None` if unbound). +2. If bound and the signer is server-callable (`RemoteBunkerSigner` + or, transitionally, `LocalSigner`), signs as the user. Drop the + `author` tag — the pubkey is the attribution. +3. If unbound — or bound to `ClientSideOnlySigner` for a verb the user + must sign themselves — falls back to `BotSigner` with the `author` + tag (the v1 §7.1 behavior). + +This degrades gracefully: a community can run v1 indefinitely with +just `BotSigner`; users opt into per-user signing individually as they +bind their identity. + +**NIP-46 is the only remote signer protocol this spec recognizes.** +It's what the Nostr ecosystem has converged on for client-without-nsec +flows, works without browser extensions on iOS, and natively supports +the per-device scoped tokens that make multi-client identity safe. +NIP-26 delegation tokens are explicitly NOT used (the Nostr ecosystem +has deprecated NIP-26 and NIP-46 covers the same ground better). + +### 7.3 External-identity binding (v2) + +For per-user signing to mean anything, the bot needs a verified +mapping from `chat handle → identity provider → signer`. The binding +mechanism is implementation-defined but MUST be: + +- **Opt-in** per user. No silent association. +- **Verifiable** — the binding proves the user controls both identities + (e.g. magic-link round-trip via the IdP's authenticated session, + signature challenge, NIP-39 external identity proof). +- **Revocable** — user can unbind. Bot drops back to `BotSigner` + fallback. Crucially: revocation MUST also revoke whatever per-device + signing capability the binding established (see below). + +#### Binding artifact (the important part) + +A binding is not just a row in a mapping table — it's a **per-device +NIP-46 connection token** scoped to exactly the kinds the bot needs to +sign on behalf of the user. The bot stores: + +- The user's pubkey (for display, addressing, ECDH) +- The bunker URL (transport: which relay channel the bunker listens on) +- The connection token (the credential to authenticate to the bunker) +- The granted permission scope (e.g. `sign_event:31922,31923,31925` + for a community-organizer bot — no kind-1 notes, no DMs, no profile + edits) + +Per-device-scoped tokens are the security property that makes +multi-client identity safe. If the bot is compromised, the attacker +can publish NIP-52 events as the user — annoying, but they can't +publish kind-1 notes, change the user's profile, send DMs, or do +anything else the user's other devices can. Revocation of the bot's +token at the bunker side is one RPC and doesn't disturb the target +key or any other authorized device. + +#### Identity-provider options + +The spec does not mandate a specific identity provider. Any system +that can answer "for chat handle X, issue me a scoped connection +token to sign on their behalf" works: + +- A **Lightning/Nostr account system** (LNbits, Alby Hub, etc.) — the + reference implementation. These already hold user pubkeys and have + an authenticated session surface that can mediate the binding flow. + Per the operator-IdP pattern (§12), the IdP itself doesn't hold the + nsec — a sidecar bunker does — and the IdP brokers token issuance + to that bunker. +- A **standalone web app** with its own auth and a `bind chat handle` + flow that issues NIP-46 connection tokens. +- An **on-Nostr profile claim** — user publishes a kind-0 / kind-10002 + / NIP-39 attestation linking the chat handle to a pubkey, with a + pre-issued bunker URL in their profile. Bot reads relay and + cryptographically verifies the claim. + +Adopters without any of the above can stay on v1 (`BotSigner` only) +— per-user signing is enhancement, not requirement. + +--- + +## 8. Worked examples + +### 8.1 Capture: explicit verb + tag + +User types in Matrix room `#animals:matrix.example`: + +``` +!task #buy goat dewormer from the co-op +``` + +Bot publishes: + +```json +{ + "kind": 31922, + "pubkey": "", + "created_at": 1716559200, + "tags": [ + ["d", "cdf-animals:01HXXJ8K7VP"], + ["title", "goat dewormer from the co-op"], + ["t", "task"], + ["t", "buy"], + ["t", "src:explicit"], + ["a", "34550::cdf-animals"], + ["author", "@alice:matrix.example"] + ], + "content": "goat dewormer from the co-op" +} +``` + +Bot replies: `✅ Task recorded (#buy). React with ❌ to delete.` + +### 8.2 Capture: community shortcut + +Same room has `!buy` configured as a shortcut. User types: + +``` +!buy door handles from Laura +``` + +Bot expands to `!task #buy door handles from Laura` and publishes +the same shape as §8.1 but with `["t", "src:shortcut"]`. + +### 8.2a Capture: priority + sidequest + +``` +!sidequest #priority:3 build a chicken-tractor prototype from the old pallets +``` + +Bot publishes: + +```json +"tags": [ + ["d", "cdf-animals:01HXXJ9PK2Q"], + ["title", "build a chicken-tractor prototype from the old pallets"], + ["t", "sidequest"], + ["t", "priority:3"], + ["t", "src:explicit"], + ["a", "34550::cdf-animals"], + ["author", "@alice:matrix.example"] +] +``` + +Bot replies: `🎯 Sidequest recorded (priority: important). React with +❌ to delete.` + +### 8.2b Capture: priority shortcut + frequent/ongoing + +Room has `urgent` and `chores` configured per §3.4 example. + +``` +!urgent the alpaca fence is down by the south paddock +``` + +Expands to `!task #priority:1 the alpaca fence is down…` — the bot +SHOULD also notify the room (urgency-level escalation hook). + +``` +!chores brush the goats +``` + +Expands to `!task #priority:5 brush the goats`. Renderers surface +this in the "recurring" view, not the urgent-action queue. + +### 8.3 Capture: freeform with rules classification + +User types: + +``` +!add need to roll out flypaper for the stable +``` + +Bot writes immediately as `unclassified`, classifier sees "need to" +and "flypaper" (matches `buy` rule), updates tags within 100ms: + +```json +"tags": [ + ["t", "task"], + ["t", "buy"], + ["t", "src:rules"], + ... +] +``` + +Bot edits its ack: `📥 Logged as a #buy task. React with 🔄 to +re-classify.` + +### 8.4 Capture: freeform falling through to LLM (Level 2) + +User types: + +``` +!add the duck babies straw needs scraping clean tomorrow +``` + +Rules don't match cleanly (no buy keyword, future-tense not past). +Bot writes as `unclassified`; LLM classifier (running async on a +local Ollama model) returns `task #animals` within 2–5s. Bot updates +tags, edits ack with the result. + +### 8.5 Close + +User reacts to the bot's original ack message with ✅, OR types: + +``` +!done 17 +``` + +Bot publishes a kind 31925 event referencing the task's `a`-tag with +`status: accepted`. + +### 8.6 Journal entry (past-tense) + +User types: + +``` +!journal +- opened, watered, and fed the hens, alpacas, and chicks +- put insulators on the rebars so they're ready for the fence +- mentally plotted potential hidden-in-plain-sight stable locations +``` + +Bot publishes a single kind 31922 with `["t", "journal"]`, +`content` containing the full multi-line body verbatim. + +### 8.7 Reminder + +User types: + +``` +!remind tomorrow 9am check sapphi's wound +``` + +Bot publishes kind 31923 with `start = `, +`["t", "remind"]`. At fire time, the bot posts a chat message +pinging the originating user. + +### 8.8 Query + +User types: + +``` +!list task #buy +``` + +Bot returns the most recent N open tasks in the current community +tagged `buy`. Implementation-defined formatting. + +### 8.9 Inbound: external publisher + +A community member uses the webapp (or any other client) to create +a NIP-52 event with the right community `a`-tag. The bot, which +subscribes to events tagged with its communities, mirrors it into +its local cache. The next `!list` includes the externally-created +entry. The eink renderer (also subscribed) shows it. + +--- + +## 9. Conformance + +An implementation is **conformant** if: + +1. It produces events matching §4 for all universal verbs in §3.1. +2. It scopes events with NIP-72 `a`-tags per §5. +3. It honors NIP-09 deletions (§4.4). +4. It records classification-source tags (§3.3). +5. It supports at least classifier Level 0 (§6.1). +6. It documents its surface vocabulary (universal verbs + any + per-community shortcuts) in a form usable by community members. + +Bots SHOULD additionally: + +- Subscribe to events tagged with their communities (not just + publish) — bidirectional sync makes the spec actually portable. +- Carry the `author` tag for human attribution (§4.1). +- Maintain a local cache for read latency and offline tolerance. + +Renderers SHOULD: + +- Filter by `a`-tag, not by `pubkey` (so events from any bot OR any + external publisher targeting the community are surfaced). +- Apply NIP-09 deletions before display. +- Surface a stale-data indicator when relay is unreachable. + +--- + +## 10. Extension points + +This spec is intentionally minimal. Communities and implementations +can extend without breaking conformance via: + +- **New domain tags** — add whatever `t` tags make sense locally. + Renderers ignore tags they don't recognize. +- **New shortcut verbs** — add via per-community config; never + modify universal verbs. +- **New classification sources** — `src:` is open. +- **Richer content formats** — `content` is freeform text in v1; + Markdown or other formats are permitted (signal via a `format` + tag) but renderers fall back to plain text. + +If you find yourself wanting a new event kind or a new universal +verb, please open an issue against this spec — protocol-level +additions need broader review. + +--- + +## 11. Open questions + +Not yet decided in this draft: + +- **Recurring reminders.** RFC 5545 has `RRULE`; NIP-52 doesn't + formalize recurrence yet. Need a tag convention or wait for an + upstream NIP. +- **Cross-community references.** A task in `#kitchen` that depends + on a buy entry in `#purchasing`. Needs convention for `e`-tags + across community boundaries. +- **Assignment.** Today `author` records who captured; we don't + formalize "who's assigned". Could use `p`-tag with a role + qualifier. +- **Migration / weekly review.** Bullet-journal-style migration of + stale open items into a new period. Needs a verb (`!migrate`?) + and a state transition spec. +- **External signer story.** Architecture is concrete in §7.2 / §7.3 + / §12 (operator-IdP with sidecar `nsecbunkerd`, per-device scoped + connection tokens). Phase 1 of [aiolabs/lnbits#9](https://git.atitlan.io/aiolabs/lnbits/issues/9) + has shipped the signer abstraction; phase 2 ([#18](https://git.atitlan.io/aiolabs/lnbits/issues/18)) + is the actual bunker integration. Pending design: ergonomics of the + chat-side binding flow (DM the bot? web callback? both?), and how + to handle `ClientSideOnlySigner` users whose events can't be + bot-published at all (the bot subscribes and mirrors instead). + +Contributions welcome on any of these. + +--- + +## 12. Reference implementation + +The aiolabs reference implementation lives at +`git.atitlan.io/aiolabs/maubot-plugins/` (the `tracker/` plugin) and +serves the Château du Faune community at Ariège, France. It targets +Matrix as the chat surface and uses NIP-52 + NIP-72 over a +self-hosted Nostr relay. + +A companion eink renderer at +`git.atitlan.io/aiolabs/inky-impression/` consumes the same events +to drive a 13.3" panel in the community's shared foyer. + +Both serve as worked examples of the spec; neither is mandatory. + +### Alternate runtimes + +Adopters who want a richer agent layer than a focused Matrix bot can +satisfy this spec on top of a general-purpose agent runtime such as +[ZeroClaw](https://github.com/zeroclaw-labs/zeroclaw) (Rust, Apache-2.0 +/ MIT, ships Matrix + Nostr channels and Ollama provider out of the +box). The spec is runtime-agnostic — what matters is that captured +input produces conformant events per §4 and respects the community +scoping in §5. A ZeroClaw-based implementation would carry the +`["client", "zeroclaw", ""]` tag (per §4.1) instead of +`["client", "maubot-tracker", "..."]`; renderers ignore the difference +since they filter by community `a`-tag. + +### Reference identity provider — operator-IdP pattern + +The aiolabs reference implementation runs the **operator-IdP-with- +sidecar-bunker** pattern that NIP-46 was designed for. Three processes +on the same host: + +``` + ┌─────────────────────────────┐ + │ nsecbunkerd (sidecar) │ Holds every user's target key + │ │ + the operator master. + │ - admin: M_lnbits │ Per-device scoped connection + │ - targets: X_alice, … │ tokens. + │ - per-client tokens │ Speaks NIP-46 over kind-24133; + │ - scoped perms │ admin RPC over kind-24134. + └────────────┬────────────────┘ + │ kind-24133 / 24134 over internal relay + ▼ + ┌─────────────────────────────┐ + │ LNbits (identity broker) │ Holds zero nsec material. + │ │ Stores per-user (pubkey, + │ - account → pubkey │ signer_type, signer_config) + │ - account → bunker URL │ where signer_config is the + │ - admin RPC client │ bunker URL + connection token + │ (scoped agent key) │ + scoped perms. + └────────────┬────────────────┘ + │ HTTPS (token issuance, account lookup) + ▼ + ┌─────────────────────────────┐ + │ Bot / tracker plugin / │ Holds per-user connection + │ other clients │ tokens scoped to its needs. + │ │ Speaks NIP-46 directly to + │ - NIP-46 client per user │ the bunker. Compromise → + │ for sign_event RPCs │ revoke just that token. + └─────────────────────────────┘ +``` + +This is the architecture from [aiolabs/lnbits#9](https://git.atitlan.io/aiolabs/lnbits/issues/9) +(the IdP framing and signer abstraction) and +[aiolabs/lnbits#18](https://git.atitlan.io/aiolabs/lnbits/issues/18) +(the concrete bunker integration using +[`nsecbunkerd`](https://github.com/kind-0/nsecbunkerd)). PR #17 has +shipped phase 1 (the abstraction + transitional `LocalSigner` + the +classify migration); phase 2 (#18) is the actual bunker integration. + +#### Why a sidecar bunker, not "encrypt at rest in the IdP" + +The IdP attack surface (web requests, dependency tree, plugin system, +admin endpoints, the database) is large. Putting nsec material on the +other side of an RPC boundary — held by a much smaller process whose +only job is signing — bounds compromise to "we leak signed events +during the window the bunker is unavailable for revocation" rather +than "the entire user identity pool is exfiltrated". The bunker also +gets its own audit log, independent of the IdP's. + +The pattern is reusable beyond LNbits — any operator who wants to +provide identity-as-a-service to their users can run this same shape +with any NIP-46-compliant bunker (`nsecbunkerd`, Pablo's reference +implementation, or any future alternative). The Nostriga 2024 NIP-46 +panel describes the same architecture. + +#### Adopters without an IdP + +Communities without an existing identity provider can run v1 +(`BotSigner` only) indefinitely. Per-user signing is enhancement, +not requirement — the community organizer use case works fine with +the bot signing as itself and human attribution carried in the +`author` tag. + +--- + +## Changelog + +- **0.1** (2026-05-24) — initial draft. diff --git a/journal/README.md b/journal/README.md new file mode 100644 index 0000000..e7b84de --- /dev/null +++ b/journal/README.md @@ -0,0 +1,82 @@ +# journal + +Daily-journal Matrix bot. Each room member can record what they did, +and anyone in the room can query the log. + +## Commands + +``` +!journal record an entry (multi-line OK) +!journal show [@user:domain] last 10 entries, optionally filtered by user +!journal today all entries from today (UTC) +``` + +Multi-line works either inline or after a newline: + +``` +!journal Did three things today: +- planted garlic +- mucked out the goat pen +- finished the irrigation patch +``` + +``` +!journal +- planted garlic +- mucked out the goat pen +``` + +Both record the full body verbatim. + +## Storage + +One SQLite database per maubot instance, at +`/var/lib/maubot/plugin-dbs/journal.db` on the host. Schema (managed +by `mautrix.util.async_db.UpgradeTable`): + +```sql +CREATE TABLE entries ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + user TEXT NOT NULL, -- @sender:domain + room TEXT NOT NULL, -- !roomid:domain + ts BIGINT NOT NULL, -- ms since epoch (from evt.timestamp) + text TEXT NOT NULL -- raw entry body +); +CREATE INDEX entries_user_ts ON entries (user, ts DESC); +CREATE INDEX entries_ts ON entries (ts DESC); +``` + +Wipe data via the maubot UI's per-instance **Database** tab: + +```sql +DELETE FROM entries; +DELETE FROM sqlite_sequence WHERE name = 'entries'; +``` + +(The second line resets the auto-increment counter; skip it if you'd +rather keep IDs monotonic across resets.) + +## Known quirks + +- **Edited messages don't re-trigger the bot.** Matrix sends edits as + a separate `m.replace` event that bots don't react to. If you typed + `!journal` then edited the message to add content, the bot saw only + the empty `!journal` and won't record. Send a fresh message instead + of editing. +- **`!journal show ` runs the show query with that text + as the user filter.** If it doesn't match any MXID, you get + "No entries." Use a fully-qualified MXID like `@pat:ariege.io`. +- **Subcommand detection only looks at the first line.** Anything + starting with `show ` or `today` on the first line dispatches to + the query handlers; anything else (including prose that happens to + contain "show" mid-text) records as an entry. + +## Architecture note + +This plugin uses `@command.passive` with a regex matcher rather than +`@command.new`. The reason — and why other plugins should consider the +same pattern for prose-input commands — is documented in `~/dev/CLAUDE.md` +under "Multi-line freeform parent commands". Short version: +`@command.new` silently drops `!journal\n` because maubot's +parser only treats space as the command/args delimiter, leading to +invisible data loss when users naturally hit Enter after the command. diff --git a/journal/journal.py b/journal/journal.py index fdb3a3e..2a78fe3 100644 --- a/journal/journal.py +++ b/journal/journal.py @@ -1,5 +1,5 @@ +import re from datetime import datetime, timezone -from typing import Optional from maubot import MessageEvent, Plugin from maubot.handlers import command @@ -25,6 +25,22 @@ async def upgrade_v1(conn: Connection) -> None: await conn.execute("CREATE INDEX entries_ts ON entries (ts DESC)") +# Match `!journal` followed by any whitespace (space, tab, OR newline) +# and capture everything after. Maubot's @command.new parser only treats +# *space* as the command/args delimiter, so `!journal\n` gets +# parsed as a command name of "journal\n" and matches nothing, +# silently dropping multi-line entries. A passive regex matcher with +# DOTALL bypasses the parser quirk and catches every form. +_JOURNAL_RE = re.compile(r"^!journal(?:[ \t\r\n]+(.*))?$", re.DOTALL) + +_USAGE = ( + "Usage:\n" + "- `!journal ` — record an entry (multi-line OK)\n" + "- `!journal show [@user]` — last 10 entries, optionally filtered\n" + "- `!journal today` — all entries from today (UTC)" +) + + def _fmt(rows) -> str: if not rows: return "No entries." @@ -40,49 +56,55 @@ class JournalBot(Plugin): def get_db_upgrade_table(cls) -> UpgradeTable: return upgrade_table - @command.new( - "journal", - help="Farm journal — record what you did today", - require_subcommand=False, - arg_fallthrough=False, - ) - @command.argument("text", pass_raw=True, required=False) - async def journal(self, evt: MessageEvent, text: str = "") -> None: - if not text: - await evt.reply( - "Usage:\n" - "- `!journal ` — record an entry\n" - "- `!journal show [@user]` — last 10 entries (optionally filtered by user)\n" - "- `!journal today` — all entries from today" - ) + @command.passive(regex=_JOURNAL_RE) + async def journal(self, evt: MessageEvent, match) -> None: + rest = (match[1] or "").strip() + + if not rest: + await evt.reply(_USAGE) return + # Subcommand detection on the first whitespace-delimited token of + # the first line — only catches `show`/`today` if they appear + # alone on the first line (with optional arg). Anything else + # (including pasted multi-line lists) is recorded as-is. + first_line, _, _ = rest.partition("\n") + first_token, _, after = first_line.partition(" ") + + if first_token == "show": + user = after.strip() or None + if user: + rows = await self.database.fetch( + "SELECT user, ts, text FROM entries" + " WHERE user = $1 ORDER BY ts DESC LIMIT 10", + user, + ) + else: + rows = await self.database.fetch( + "SELECT user, ts, text FROM entries ORDER BY ts DESC LIMIT 10", + ) + await evt.reply(_fmt(rows)) + return + + if first_token == "today": + midnight = datetime.now(timezone.utc).replace( + hour=0, minute=0, second=0, microsecond=0 + ) + cutoff_ms = int(midnight.timestamp() * 1000) + rows = await self.database.fetch( + "SELECT user, ts, text FROM entries" + " WHERE ts >= $1 ORDER BY ts ASC", + cutoff_ms, + ) + await evt.reply(_fmt(rows)) + return + + # Default: record the full rest (multi-line preserved) await self.database.execute( "INSERT INTO entries (user, room, ts, text) VALUES ($1, $2, $3, $4)", - evt.sender, evt.room_id, evt.timestamp, text, + evt.sender, + evt.room_id, + evt.timestamp, + rest, ) await evt.reply(f"📓 Logged for {evt.sender}.") - - @journal.subcommand("show", help="Show recent entries, optionally filtered by user") - @command.argument("user", required=False) - async def show(self, evt: MessageEvent, user: Optional[str] = None) -> None: - if user: - rows = await self.database.fetch( - "SELECT user, ts, text FROM entries WHERE user = $1 ORDER BY ts DESC LIMIT 10", - user, - ) - else: - rows = await self.database.fetch( - "SELECT user, ts, text FROM entries ORDER BY ts DESC LIMIT 10", - ) - await evt.reply(_fmt(rows)) - - @journal.subcommand("today", help="All entries from today (UTC) across users") - async def today(self, evt: MessageEvent) -> None: - midnight = datetime.now(timezone.utc).replace(hour=0, minute=0, second=0, microsecond=0) - cutoff_ms = int(midnight.timestamp() * 1000) - rows = await self.database.fetch( - "SELECT user, ts, text FROM entries WHERE ts >= $1 ORDER BY ts ASC", - cutoff_ms, - ) - await evt.reply(_fmt(rows)) diff --git a/journal/maubot.yaml b/journal/maubot.yaml index 9352d83..74e2ae3 100644 --- a/journal/maubot.yaml +++ b/journal/maubot.yaml @@ -1,6 +1,6 @@ maubot: 0.1.0 id: dev.aiolabs.journal -version: 0.1.3 +version: 0.2.0 license: AGPL-3.0-or-later modules: - journal diff --git a/tracker/README.md b/tracker/README.md new file mode 100644 index 0000000..0afa4ea --- /dev/null +++ b/tracker/README.md @@ -0,0 +1,176 @@ +# tracker + +Community-organizer bot for Matrix. Implements the +[Community Organizer spec](../docs/community-organizer-spec.md) over +maubot — capture/list/close tasks, sidequests, reminders, and freeform +inbox entries scoped per room. + +Sibling to [`journal/`](../journal/) (which owns `!journal`); the two +work side-by-side on the same maubot instance if you want both. + +## Quickstart + +``` +!add door handles from Laura # freeform, classifier sorts it +!task fix the south fence #urgent # explicit task +!sidequest #priority:3 build a chicken-tractor +!remind in 30m check the eggs # reminder; fires in chat at due time +!list # open items in this room +!list sidequest # only sidequests +!done 17 # close item 17 +!setup # show room shortcuts +!setup add buy task #buy # define a !buy shortcut for this room +``` + +Multi-line input works either inline or after a newline: + +``` +!task +- fix the alpaca fence +- patch the duck pond liner +- replace the gate latch +``` + +## Commands + +| Command | Purpose | +|---|---| +| `!add ` | Freeform capture. Rules classifier sorts into a kind (task/journal/remind/etc.) or leaves it in the inbox as `unclassified` for later sorting. | +| `!task [#tag…]` | Direct task capture. Tags become NIP-52 `t` tags. | +| `!sidequest [#tag…]` | Optional / passion-project item. Same lifecycle as a task but listed separately so the day-to-day list stays clean. | +| `!remind in (s\|m\|h\|d) ` | Schedule a chat-side reminder. Fires by posting in the room at the due time. | +| `!done ` | Close a task or sidequest by id. | +| `!list [task\|sidequest\|remind\|inbox\|all]` | List open items in this room (last 20). Default is everything except journals. | +| `!setup` | Show current room shortcuts. | +| `!setup add [#tag…]` | Add a per-room shortcut (e.g. `!setup add buy task #buy`). | +| `!setup remove ` | Drop a shortcut. | + +Universal verbs are hardcoded; per-room shortcuts are stored in the +plugin DB and apply only to the room they're defined in. A bakery +co-op can define `!harvest` / `!market` in their rooms; Ariège can +define `!buy` / `!steward` / `!chores` in theirs. Same plugin code. + +## Priority + +Items can carry a priority tag (`#priority:1` through `#priority:5`) +per the spec's 5-level scale: urgent / crucial / important / future / +frequent-ongoing. The bot surfaces the label in `!list` output; +renderers (eink panel, etc.) decide how to use it for sorting and +visual emphasis. + +The most ergonomic capture is via room shortcuts: + +``` +!setup add urgent task #priority:1 +!setup add chores task #priority:5 +``` + +Then `!urgent the alpaca fence is down` records a priority-1 task. + +## Classification + +`!add` runs through a tiny rules engine (`classify.py`): + +| Pattern | Bucket | +|---|---| +| `remind me to…` / `don't forget…` / `remember to…` | `remind` (no auto-time; you'll need to re-capture with `!remind in …` to set a fire-time) | +| `buy` / `purchase` / `order` / `pick up` / `grab` / `get more` | `task #buy` | +| past-tense verbs (`did`, `finished`, `cleaned`, `watered`, `fed`, …) | `journal` | +| `need to` / `should` / `have to` / `must` / `todo` | `task` | +| anything else | `unclassified` (sits in `!list inbox` until sorted) | + +This is intentionally **conservative** — better to leave something +unclassified than to mis-bucket it. The Phase 2b plan is an LLM +fallback on a dedicated inference node that handles the long tail. + +## Storage + +One SQLite DB per maubot instance, schema managed by +`mautrix.util.async_db.UpgradeTable`. Two tables: + +```sql +-- The items log +CREATE TABLE items ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + kind TEXT NOT NULL, -- task | sidequest | journal | remind | unclassified + status TEXT NOT NULL, -- open | done | canceled + tags TEXT NOT NULL DEFAULT '', -- comma-separated; e.g. "buy,priority:2" + room TEXT NOT NULL, + user TEXT NOT NULL, -- originating MXID + ts BIGINT NOT NULL, -- captured at (ms since epoch) + due_at BIGINT, -- for reminders/timed tasks + body TEXT NOT NULL, + classification_source TEXT, -- explicit | shortcut | rules | llm | manual + nostr_event_id TEXT -- populated in Phase 2 (Nostr bridge) +); + +-- Per-room shortcut config + (future) publish posture +CREATE TABLE community_config ( + room TEXT PRIMARY KEY, + config_json TEXT NOT NULL +); +``` + +Wipe via the maubot UI's per-instance **Database** tab: + +```sql +DELETE FROM items; +DELETE FROM community_config; +DELETE FROM sqlite_sequence WHERE name = 'items'; +``` + +## Reminders survive restarts + +Pending reminders are reloaded from the DB when the bot starts — +nothing in flight when maubot restarts, but anything scheduled for +the future re-arms. Reminders whose due time elapsed while the bot +was down stay open (they aren't auto-fired late); query with +`!list remind` to find them. + +## Known quirks + +- **Edited messages don't re-trigger the bot.** Matrix sends edits + as a separate `m.replace` event that maubot doesn't pass to + handlers. Type a fresh message instead of editing. +- **Time parsing is minimal in v1.** Only `in (s|m|h|d)` works. + No "tomorrow 9am", no calendar dates. Use Phase 2b's LLM layer or + upstream `dateparser` for richer parsing later. +- **`!done` accepts only the id.** Closing by partial text match + (e.g. `!done flypaper`) isn't implemented in v1. +- **Phase 1 is Matrix-local.** Nothing publishes to Nostr yet — the + `nostr_event_id` column exists but is always NULL until Phase 2a + lands the bridge. +- **Universal verbs win over shortcuts.** You can't define a shortcut + named `task`, `add`, etc. — those are protocol-level. + +## Build + iterate + +```sh +cd ~/dev/maubot-plugins/tracker +# bump version in maubot.yaml so the UI surfaces the new build +zip -j ../tracker.mbp maubot.yaml *.py +``` + +Upload via maubot UI → Plugins → click existing → upload new `.mbp`. +**Hit Save on the affected instance** after upload — toggling +Enabled alone doesn't persist (easy facepalm). Create a fresh bot +client (e.g. `@trackerbot:ariege.io`) or attach to an existing one; +invite to the rooms you want it active in. + +See `~/dev/CLAUDE.md` "Maubot plugin development" for the multi-line +command footgun and other plugin-wide gotchas. + +## Architecture notes + +This plugin uses `@command.passive` with a single dispatch regex +(`^!(\w+)…`) rather than per-verb `@command.new` decorators. Two +reasons: + +1. **Multi-line freeform input.** Per `~/dev/CLAUDE.md`, `@command.new` + silently drops `!task\n` because maubot's parser only + treats space as the command/args delimiter. Passive regex with + `re.DOTALL` catches every form. +2. **Per-room shortcuts.** A single dispatch handler can route + universal verbs to fixed handlers AND check the room's shortcut + table for unknown verbs, all in one place. Cleaner than declaring + N decorators and N matchers. diff --git a/tracker/classify.py b/tracker/classify.py new file mode 100644 index 0000000..a7394ef --- /dev/null +++ b/tracker/classify.py @@ -0,0 +1,48 @@ +"""Rules-based classifier for !add freeform capture. + +Level 1 of the spec's classifier conformance (§6.1). Maps obvious +shapes to a kind+tags. Anything ambiguous returns (None, []) and the +caller records the entry as `unclassified` for later sorting. + +LLM fallback is Phase 2b and lives separately. +""" + +import re + +_BUY_RE = re.compile( + r"\b(buy|purchase|order|pick\s*up|grab|get\s+more)\b", + re.IGNORECASE, +) +_REMIND_PREFIX_RE = re.compile( + r"\b(remind\s+me\s+to|don'?t\s+forget|remember\s+to)\b", + re.IGNORECASE, +) +_PAST_TENSE_RE = re.compile( + r"\b(" + r"did|finished|completed|cleaned|watered|fed|opened|closed|" + r"planted|harvested|fixed|moved|painted|swept|mucked|brushed|" + r"installed|repaired|cooked|prepared" + r")\b", + re.IGNORECASE, +) +_TASK_INTENT_RE = re.compile( + r"\b(need\s+to|should|have\s+to|must|todo|to\s+do)\b", + re.IGNORECASE, +) + + +def classify(body: str) -> tuple[str | None, list[str]]: + """Return (kind, tags). (None, []) means unclassified.""" + if _REMIND_PREFIX_RE.search(body): + return ("remind", []) + + if _BUY_RE.search(body): + return ("task", ["buy"]) + + if _PAST_TENSE_RE.search(body): + return ("journal", []) + + if _TASK_INTENT_RE.search(body): + return ("task", []) + + return (None, []) diff --git a/tracker/maubot.yaml b/tracker/maubot.yaml new file mode 100644 index 0000000..452d522 --- /dev/null +++ b/tracker/maubot.yaml @@ -0,0 +1,9 @@ +maubot: 0.1.0 +id: dev.aiolabs.tracker +version: 0.1.0 +license: AGPL-3.0-or-later +modules: + - tracker +main_class: TrackerBot +database: true +database_type: asyncpg diff --git a/tracker/scheduler.py b/tracker/scheduler.py new file mode 100644 index 0000000..84b27a1 --- /dev/null +++ b/tracker/scheduler.py @@ -0,0 +1,67 @@ +"""Reminder scheduler for !remind items. + +Holds an asyncio.Task per pending reminder. On bot start, replays +open reminders from the DB whose due_at is still in the future. On +stop, cancels all in-flight timers. + +`!done` and explicit deletion are not yet plumbed into the scheduler +— a fired reminder marks itself `done` in the DB so duplicate +delivery on bot restart is avoided. +""" + +import asyncio +import time +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from tracker import TrackerBot + + +class RemindScheduler: + def __init__(self, bot: "TrackerBot") -> None: + self.bot = bot + self._tasks: dict[int, asyncio.Task] = {} + self._closed = False + + async def start(self) -> None: + now_ms = int(time.time() * 1000) + rows = await self.bot.database.fetch( + "SELECT id, room, body, due_at FROM items" + " WHERE kind = 'remind' AND status = 'open'" + " AND due_at IS NOT NULL AND due_at > $1", + now_ms, + ) + for row in rows: + self.schedule(row["id"], row["due_at"], row["room"], row["body"]) + + def schedule(self, item_id: int, due_at_ms: int, room: str, body: str) -> None: + if self._closed: + return + if item_id in self._tasks: + self._tasks[item_id].cancel() + now_ms = int(time.time() * 1000) + delay_s = max(0.0, (due_at_ms - now_ms) / 1000.0) + task = asyncio.create_task(self._fire(item_id, delay_s, room, body)) + self._tasks[item_id] = task + + async def _fire(self, item_id: int, delay_s: float, room: str, body: str) -> None: + try: + await asyncio.sleep(delay_s) + if self._closed: + return + await self.bot.client.send_text(room, f"⏰ Reminder: {body}") + await self.bot.database.execute( + "UPDATE items SET status = 'done' WHERE id = $1", item_id + ) + except asyncio.CancelledError: + raise + except Exception: + self.bot.log.exception("reminder fire failed for item %d", item_id) + finally: + self._tasks.pop(item_id, None) + + async def stop(self) -> None: + self._closed = True + for task in self._tasks.values(): + task.cancel() + self._tasks.clear() diff --git a/tracker/tracker.py b/tracker/tracker.py new file mode 100644 index 0000000..3f39b0e --- /dev/null +++ b/tracker/tracker.py @@ -0,0 +1,392 @@ +import json +import re +import time +from datetime import datetime, timezone + +from maubot import MessageEvent, Plugin +from maubot.handlers import command +from mautrix.util.async_db import Connection, UpgradeTable + +from classify import classify +from scheduler import RemindScheduler + +upgrade_table = UpgradeTable() + + +@upgrade_table.register(description="Initial schema") +async def upgrade_v1(conn: Connection) -> None: + await conn.execute( + """ + CREATE TABLE items ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + kind TEXT NOT NULL, + status TEXT NOT NULL, + tags TEXT NOT NULL DEFAULT '', + room TEXT NOT NULL, + user TEXT NOT NULL, + ts BIGINT NOT NULL, + due_at BIGINT, + body TEXT NOT NULL, + classification_source TEXT, + nostr_event_id TEXT + ) + """ + ) + await conn.execute( + "CREATE INDEX items_room_kind_status_ts" + " ON items (room, kind, status, ts DESC)" + ) + await conn.execute( + "CREATE INDEX items_due_at ON items (due_at) WHERE due_at IS NOT NULL" + ) + await conn.execute( + """ + CREATE TABLE community_config ( + room TEXT PRIMARY KEY, + config_json TEXT NOT NULL + ) + """ + ) + + +_CMD_RE = re.compile(r"^!(\w+)(?:[ \t\r\n]+(.*))?$", re.DOTALL) +_TAG_RE = re.compile(r"#(\S+)") +_RELATIVE_TIME_RE = re.compile(r"^in\s+(\d+)\s*([smhd])\b", re.IGNORECASE) + +UNIVERSAL_VERBS = {"add", "task", "sidequest", "remind", "done", "list", "setup"} + +DEFAULT_CONFIG = { + "verbs": {}, + "publish": "matrix-only", +} + +PRIORITY_LABELS = { + "1": "urgent", + "2": "crucial", + "3": "important", + "4": "future", + "5": "frequent/ongoing", +} + +_USAGE = ( + "**Tracker commands:**\n" + "- `!add ` — freeform inbox capture (auto-classified)\n" + "- `!task [#tag…]` — record a task\n" + "- `!sidequest [#tag…]` — record an optional / passion-project item\n" + "- `!remind in (s|m|h|d) ` — fire a reminder later\n" + "- `!done ` — close a task or sidequest\n" + "- `!list [task|sidequest|remind|inbox|all]` — list recent items in this room\n" + "- `!setup` — show or configure room shortcuts\n" + "\nSee `docs/community-organizer-spec.md` for the protocol shape." +) + + +def _parse_tags(body: str) -> tuple[list[str], str]: + """Pull #tag tokens out of body; return (tags, cleaned_body).""" + tags = _TAG_RE.findall(body) + cleaned = _TAG_RE.sub("", body).strip() + cleaned = re.sub(r"\s+", " ", cleaned) + return tags, cleaned + + +def _parse_relative_time(s: str) -> tuple[int | None, str]: + """Returns (due_at_ms, remainder). (None, s) if unparseable.""" + m = _RELATIVE_TIME_RE.match(s) + if not m: + return None, s + n = int(m.group(1)) + unit = m.group(2).lower() + mult = {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit] + due_at_s = time.time() + n * mult + rest = s[m.end():].strip() + return int(due_at_s * 1000), rest + + +def _format_age(ts_ms: int) -> str: + delta = time.time() - ts_ms / 1000 + if delta < 60: + return "just now" + if delta < 3600: + return f"{int(delta / 60)}m ago" + if delta < 86400: + return f"{int(delta / 3600)}h ago" + return f"{int(delta / 86400)}d ago" + + +def _priority_label(tags: list[str]) -> str | None: + for t in tags: + if t.startswith("priority:"): + return PRIORITY_LABELS.get(t.split(":", 1)[1]) + return None + + +class TrackerBot(Plugin): + scheduler: RemindScheduler + + @classmethod + def get_db_upgrade_table(cls) -> UpgradeTable: + return upgrade_table + + async def start(self) -> None: + await super().start() + self.scheduler = RemindScheduler(self) + await self.scheduler.start() + + async def stop(self) -> None: + if hasattr(self, "scheduler"): + await self.scheduler.stop() + await super().stop() + + async def _get_config(self, room: str) -> dict: + row = await self.database.fetchrow( + "SELECT config_json FROM community_config WHERE room = $1", room + ) + if row: + return json.loads(row["config_json"]) + return json.loads(json.dumps(DEFAULT_CONFIG)) + + async def _set_config(self, room: str, cfg: dict) -> None: + await self.database.execute( + "INSERT INTO community_config (room, config_json) VALUES ($1, $2)" + " ON CONFLICT (room) DO UPDATE SET config_json = excluded.config_json", + room, + json.dumps(cfg), + ) + + async def _record( + self, + evt: MessageEvent, + kind: str, + body: str, + tags: list[str], + status: str = "open", + due_at: int | None = None, + source: str = "explicit", + ) -> int: + tags_str = ",".join(tags) + item_id = await self.database.fetchval( + "INSERT INTO items" + " (kind, status, tags, room, user, ts, due_at, body, classification_source)" + " VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) RETURNING id", + kind, + status, + tags_str, + evt.room_id, + evt.sender, + evt.timestamp, + due_at, + body, + source, + ) + return item_id + + @command.passive(regex=_CMD_RE) + async def dispatch(self, evt: MessageEvent, match) -> None: + verb = match[1].lower() + body = (match[2] or "").strip() + + if verb in UNIVERSAL_VERBS: + handler = getattr(self, f"_handle_{verb}") + await handler(evt, body) + return + + cfg = await self._get_config(evt.room_id) + shortcut = cfg.get("verbs", {}).get(verb) + if shortcut: + await self._handle_shortcut(evt, body, shortcut, verb) + return + + async def _handle_add(self, evt: MessageEvent, body: str) -> None: + if not body: + await evt.reply(_USAGE) + return + explicit_tags, cleaned = _parse_tags(body) + kind, auto_tags = classify(cleaned) + if kind is None: + await self._record(evt, "unclassified", cleaned, explicit_tags, "open", None, "rules") + await evt.reply( + "📥 Logged to inbox (unclassified).\n" + "Reply with `!sort ` on this thread to bucket, or use `!list inbox`." + ) + return + tags = list(dict.fromkeys(auto_tags + explicit_tags)) + await self._record(evt, kind, cleaned, tags, "open", None, "rules") + tag_display = (" #" + " #".join(tags)) if tags else "" + await evt.reply(f"📥 Logged as `{kind}`{tag_display}. (auto-classified)") + + async def _handle_task(self, evt: MessageEvent, body: str) -> None: + await self._task_like(evt, body, "task", [], "explicit", "✅ Task") + + async def _handle_sidequest(self, evt: MessageEvent, body: str) -> None: + await self._task_like(evt, body, "sidequest", [], "explicit", "🎯 Sidequest") + + async def _handle_shortcut( + self, evt: MessageEvent, body: str, shortcut: dict, verb: str + ) -> None: + kind = shortcut.get("kind", "task") + base_tags = list(shortcut.get("tags", [])) + emoji = "🎯" if kind == "sidequest" else "✅" + await self._task_like(evt, body, kind, base_tags, "shortcut", f"{emoji} {kind.capitalize()} (#{verb})") + + async def _task_like( + self, + evt: MessageEvent, + body: str, + kind: str, + base_tags: list[str], + source: str, + label: str, + ) -> None: + if not body: + await evt.reply(f"Usage: `!{kind} [#tag…]`") + return + explicit_tags, cleaned = _parse_tags(body) + tags = list(dict.fromkeys(base_tags + explicit_tags)) + item_id = await self._record(evt, kind, cleaned, tags, "open", None, source) + prio = _priority_label(tags) + prio_display = f" *(priority: {prio})*" if prio else "" + tag_display = (" #" + " #".join(t for t in tags if not t.startswith("priority:"))) if tags else "" + await evt.reply(f"{label}{tag_display}{prio_display} — id `{item_id}`") + + async def _handle_remind(self, evt: MessageEvent, body: str) -> None: + if not body: + await evt.reply( + "Usage: `!remind in (s|m|h|d) `\n" + "Example: `!remind in 30m check the eggs`" + ) + return + due_at_ms, rest = _parse_relative_time(body) + if due_at_ms is None or not rest: + await evt.reply( + "Couldn't parse the time. Use `in (s|m|h|d) ` " + "(e.g. `in 2h water the chickens`)." + ) + return + explicit_tags, cleaned = _parse_tags(rest) + item_id = await self._record( + evt, "remind", cleaned, explicit_tags, "open", due_at_ms, "explicit" + ) + self.scheduler.schedule(item_id, due_at_ms, evt.room_id, cleaned) + when = datetime.fromtimestamp(due_at_ms / 1000, tz=timezone.utc).strftime("%Y-%m-%d %H:%M UTC") + await evt.reply(f"⏰ Reminder set for {when} — id `{item_id}`") + + async def _handle_done(self, evt: MessageEvent, body: str) -> None: + if not body: + await evt.reply("Usage: `!done ` — close a task or sidequest.") + return + try: + item_id = int(body.strip().split()[0]) + except (ValueError, IndexError): + await evt.reply("That doesn't look like an id. Try `!list` to find one.") + return + row = await self.database.fetchrow( + "SELECT id, kind, body, status FROM items" + " WHERE id = $1 AND room = $2", + item_id, + evt.room_id, + ) + if not row: + await evt.reply(f"No item `{item_id}` in this room.") + return + if row["kind"] == "journal": + await evt.reply("Journal entries don't get closed — they're append-only.") + return + if row["status"] == "done": + await evt.reply(f"`{item_id}` is already done.") + return + await self.database.execute( + "UPDATE items SET status = 'done' WHERE id = $1", item_id + ) + await evt.reply(f"✅ Closed `{item_id}`: {row['body']}") + + async def _handle_list(self, evt: MessageEvent, body: str) -> None: + body = body.strip() + filter_clause = "kind != 'journal' AND status = 'open'" + title = "Open items" + if body in ("task", "tasks"): + filter_clause = "kind = 'task' AND status = 'open'" + title = "Open tasks" + elif body in ("sidequest", "sidequests"): + filter_clause = "kind = 'sidequest' AND status = 'open'" + title = "Open sidequests" + elif body in ("remind", "reminders"): + filter_clause = "kind = 'remind' AND status = 'open'" + title = "Pending reminders" + elif body == "inbox": + filter_clause = "kind = 'unclassified' AND status = 'open'" + title = "Inbox (unclassified)" + elif body == "all": + filter_clause = "status = 'open'" + title = "Everything open" + rows = await self.database.fetch( + f"SELECT id, kind, tags, body, ts, due_at FROM items" + f" WHERE room = $1 AND {filter_clause}" + f" ORDER BY ts DESC LIMIT 20", + evt.room_id, + ) + if not rows: + await evt.reply(f"{title}: nothing here.") + return + lines = [f"**{title} ({len(rows)}):**"] + for r in rows: + tags = [t for t in (r["tags"] or "").split(",") if t] + prio = _priority_label(tags) + display_tags = [t for t in tags if not t.startswith("priority:")] + tag_str = (" #" + " #".join(display_tags)) if display_tags else "" + prio_str = f" *[{prio}]*" if prio else "" + kind_str = "" if r["kind"] == "task" else f" _({r['kind']})_" + lines.append( + f"- `{r['id']}`{kind_str}{prio_str}{tag_str}: {r['body']} " + f"_({_format_age(r['ts'])})_" + ) + await evt.reply("\n".join(lines)) + + async def _handle_setup(self, evt: MessageEvent, body: str) -> None: + cfg = await self._get_config(evt.room_id) + body = body.strip() + if not body or body == "show": + verbs = cfg.get("verbs", {}) + if not verbs: + await evt.reply( + "No room shortcuts configured.\n" + "Add one with: `!setup add [#tag…]`\n" + "Example: `!setup add buy task #buy`" + ) + return + lines = ["**Room shortcuts:**"] + for v, spec in verbs.items(): + tags = spec.get("tags", []) + tag_str = " ".join(f"#{t}" for t in tags) + lines.append(f"- `!{v}` → `!{spec.get('kind', 'task')} {tag_str}`") + await evt.reply("\n".join(lines)) + return + parts = body.split() + if parts[0] == "add" and len(parts) >= 3: + verb, kind = parts[1].lstrip("!"), parts[2] + if verb in UNIVERSAL_VERBS: + await evt.reply(f"`{verb}` is a universal verb — pick a different name.") + return + if kind not in ("task", "sidequest", "remind"): + await evt.reply(f"Kind must be task, sidequest, or remind (got `{kind}`).") + return + tags = [t.lstrip("#") for t in parts[3:] if t.startswith("#")] + cfg.setdefault("verbs", {})[verb] = {"kind": kind, "tags": tags} + await self._set_config(evt.room_id, cfg) + tag_display = " ".join(f"#{t}" for t in tags) + await evt.reply(f"✅ Added: `!{verb}` → `!{kind} {tag_display}`") + return + if parts[0] == "remove" and len(parts) >= 2: + verb = parts[1].lstrip("!") + if verb in cfg.get("verbs", {}): + del cfg["verbs"][verb] + await self._set_config(evt.room_id, cfg) + await evt.reply(f"🗑 Removed shortcut `!{verb}`.") + else: + await evt.reply(f"No shortcut `!{verb}` configured.") + return + await evt.reply( + "Usage:\n" + "- `!setup` or `!setup show` — list shortcuts\n" + "- `!setup add [#tag…]` — add a shortcut\n" + "- `!setup remove ` — drop a shortcut" + ) diff --git a/wiki/README.md b/wiki/README.md new file mode 100644 index 0000000..31fea58 --- /dev/null +++ b/wiki/README.md @@ -0,0 +1,117 @@ +# wiki + +Documentation-lookup Matrix bot. Points at any +[Quartz](https://quartz.jzhao.xyz/)-rendered docs site, periodically +fetches its `contentIndex.json`, and answers queries in chat. + +Designed to be community-portable — works against any Quartz site you +configure it for, not just `docs.ariege.io`. Adjust `docs_url` per +instance. + +## Commands + +``` +!ask # full-text search the docs, top 3 with snippets +!doc # open a specific page (exact slug or fuzzy title) +!wiki # status: doc count, last refresh, source URL +!wiki refresh # force re-index now (admin nicety) +``` + +## Examples + +``` +!ask how do I shut the water off +!ask alpaca feeding winter +!ask power outage +!doc emergency/water-emergency +!doc water emergency # fuzzy title match works too +!wiki # are we up to date? +``` + +The bot replies with markdown links to the doc pages, so clicking +through opens the full doc in a browser. + +## How it works + +Quartz emits `/static/contentIndex.json` as part of its standard build +— a flat `{slug: {title, content, tags}}` map of every published page. +The plugin fetches that file on a timer (default every 10 minutes), +keeps an in-memory inverted index, and scores searches by: + +- Title hits: 5 points each +- Content hits: 1 point + 0.1 × frequency + +Top N (default 3) results come back with a short snippet around the +first match. **No LLM is involved** in v1 — pure deterministic keyword +search. Phase 2b / future work may add an LLM synthesis step (RAG) +once the inference layer is up. + +## Config + +`base-config.yaml` (override per maubot instance from the UI): + +```yaml +docs_url: https://docs.ariege.io # Quartz site base URL +index_path: /static/contentIndex.json # standard Quartz path +refresh_minutes: 10 # re-fetch cadence +max_results: 3 # !ask hit limit +snippet_chars: 160 # snippet window +site_name: Castle Docs # human-readable label in output +``` + +For internal-network deployments (the recommended posture — see below), +set `docs_url: http://` instead of the public URL. + +## Deployment posture (Château du Faune) + +Both `docs.ariege.io` and the maubot daemon run on **cfaun**. The bot +hits the docs site over the host's loopback / internal network, so: + +- No WAN dependency — the bot works during internet outages +- The fetch is fast (no TLS handshake to the public internet) +- If `docs.ariege.io` is down externally, the bot is unaffected +- Same applies if a future inference node (e.g. a ZeroClaw box) lives + on the internal network: it can hit the same internal URL + +If you're deploying elsewhere, point `docs_url` at whichever URL the +bot's host can actually reach. + +## Build + iterate + +```sh +cd ~/dev/maubot-plugins/wiki +zip -j ../wiki.mbp maubot.yaml base-config.yaml *.py +``` + +Upload via maubot UI → Plugins → click existing → upload new `.mbp`. +**Hit Save on the instance** after upload (the standard maubot +facepalm). For a new instance, edit the config to point at your docs +site and save. + +## Known limitations (v1) + +- **No LLM synthesis.** Returns matched passages, not a synthesized + answer. RAG (`!ask` → cited synthesized answer) is the natural Phase + 2b enhancement when the inference node is live. +- **Stopwords are minimal.** A query like "how do I" mostly matches + stopwords and may return weak results — phrase queries with the + actual content words ("water shutoff", "winter feeding"). +- **No spell correction on content terms.** Title fuzzy match works + for `!doc`; for `!ask` you need to spell the keywords correctly. +- **No personalization.** Everyone in the room sees the same hits. +- **No multi-site support per plugin instance.** One Quartz site per + maubot instance — to serve a second docs source, install a second + instance with a different config. + +## Adopting for a different docs site + +This plugin is intentionally protocol-agnostic at the content layer — +anything that emits a `{slug: {title, content}}` JSON map will work. +For non-Quartz docs sites, you can either: + +1. Adapt the upstream build to emit a compatible `contentIndex.json` +2. Fork this plugin's `_refresh()` to parse your site's index shape + +Common alternates worth considering for adopters: MkDocs (with the +mkdocs-material search plugin), Docusaurus, mdBook, or a custom +generator. diff --git a/wiki/base-config.yaml b/wiki/base-config.yaml new file mode 100644 index 0000000..60d6903 --- /dev/null +++ b/wiki/base-config.yaml @@ -0,0 +1,21 @@ +# Wiki lookup config. Point at any Quartz-emitted site: +# `docs_url` + `index_path` together resolve to the contentIndex.json +# the bot uses for search. Page links are constructed from docs_url + slug. + +docs_url: https://docs.ariege.io +index_path: /static/contentIndex.json + +# How often to re-fetch the content index, in minutes. Lower = fresher +# but more network chatter. Site refreshes typically happen on git push, +# so a few minutes lag is normal. +refresh_minutes: 10 + +# Max results returned per `!ask` query. +max_results: 3 + +# Snippet window around the first match in `!ask` output, in characters. +snippet_chars: 160 + +# Human-readable label for the docs site, used in bot output. +# E.g. "Castle Docs", "Co-op Wiki", "Operations Manual". +site_name: Castle Docs diff --git a/wiki/maubot.yaml b/wiki/maubot.yaml new file mode 100644 index 0000000..aa1eebe --- /dev/null +++ b/wiki/maubot.yaml @@ -0,0 +1,9 @@ +maubot: 0.1.0 +id: dev.aiolabs.wiki +version: 0.1.0 +license: AGPL-3.0-or-later +modules: + - wiki +main_class: WikiBot +database: false +config: true diff --git a/wiki/wiki.py b/wiki/wiki.py new file mode 100644 index 0000000..b8e7948 --- /dev/null +++ b/wiki/wiki.py @@ -0,0 +1,234 @@ +import asyncio +import difflib +import re +import time +from typing import Optional + +from maubot import MessageEvent, Plugin +from maubot.handlers import command +from mautrix.util.config import BaseProxyConfig, ConfigUpdateHelper + + +class Config(BaseProxyConfig): + def do_update(self, helper: ConfigUpdateHelper) -> None: + helper.copy("docs_url") + helper.copy("index_path") + helper.copy("refresh_minutes") + helper.copy("max_results") + helper.copy("snippet_chars") + helper.copy("site_name") + + +_CMD_RE = re.compile(r"^!(ask|doc|wiki)(?:[ \t\r\n]+(.*))?$", re.DOTALL) +_TOKEN_RE = re.compile(r"[a-z0-9]+", re.IGNORECASE) +_STOPWORDS = frozenset({ + "a", "an", "and", "as", "at", "be", "by", "for", "from", "how", "i", + "in", "is", "it", "of", "on", "or", "than", "that", "the", "this", + "to", "was", "what", "when", "where", "which", "who", "why", "with", + "do", "does", "did", "are", "we", "you", "our", "my", "me", +}) + + +def _tokens(s: str) -> list[str]: + return [t.lower() for t in _TOKEN_RE.findall(s) if t.lower() not in _STOPWORDS] + + +def _make_url(base: str, slug: str) -> str: + return f"{base.rstrip('/')}/{slug.lstrip('/')}" + + +def _snippet(content: str, terms: list[str], width: int) -> str: + if not content: + return "" + lc = content.lower() + first = -1 + matched_term = None + for t in terms: + idx = lc.find(t) + if idx != -1 and (first == -1 or idx < first): + first = idx + matched_term = t + if first == -1: + return content[:width].strip() + ("…" if len(content) > width else "") + start = max(0, first - width // 2) + end = min(len(content), start + width) + chunk = content[start:end].strip().replace("\n", " ") + if matched_term: + chunk = re.sub( + rf"(?i)\b({re.escape(matched_term)})\b", + r"**\1**", + chunk, + ) + prefix = "…" if start > 0 else "" + suffix = "…" if end < len(content) else "" + return f"{prefix}{chunk}{suffix}" + + +class WikiBot(Plugin): + config: Config + _index: dict + _slug_titles: list[tuple[str, str]] + _last_refresh: float + _refresh_task: Optional[asyncio.Task] + + @classmethod + def get_config_class(cls): + return Config + + async def start(self) -> None: + await super().start() + self.config.load_and_update() + self._index = {} + self._slug_titles = [] + self._last_refresh = 0.0 + self._refresh_task = asyncio.create_task(self._refresh_loop()) + + async def stop(self) -> None: + if self._refresh_task: + self._refresh_task.cancel() + await super().stop() + + async def on_external_config_update(self) -> None: + self.config.load_and_update() + + async def _refresh_loop(self) -> None: + try: + while True: + try: + await self._refresh() + except Exception: + self.log.exception("wiki refresh failed; will retry") + await asyncio.sleep(self.config["refresh_minutes"] * 60) + except asyncio.CancelledError: + raise + + async def _refresh(self) -> None: + url = _make_url(self.config["docs_url"], self.config["index_path"]) + async with self.http.get(url) as resp: + resp.raise_for_status() + data = await resp.json(content_type=None) + new_index = {} + slug_titles = [] + for slug, entry in data.items(): + title = (entry.get("title") or slug).strip() + content = entry.get("content") or "" + new_index[slug] = { + "title": title, + "content": content, + "tags": entry.get("tags") or [], + } + slug_titles.append((slug, title.lower())) + self._index = new_index + self._slug_titles = slug_titles + self._last_refresh = time.time() + self.log.info("wiki refresh: %d docs from %s", len(new_index), url) + + def _search(self, query: str, limit: int) -> list[tuple[float, str, dict]]: + terms = _tokens(query) + if not terms: + return [] + hits = [] + for slug, doc in self._index.items(): + title_lc = doc["title"].lower() + content_lc = doc["content"].lower() + score = 0.0 + for t in terms: + if t in title_lc: + score += 5.0 + if t in content_lc: + score += 1.0 + 0.1 * content_lc.count(t) + if score > 0: + hits.append((score, slug, doc)) + hits.sort(key=lambda x: x[0], reverse=True) + return hits[:limit] + + def _lookup(self, query: str) -> tuple[Optional[str], Optional[dict]]: + q = query.strip().lower() + if not q: + return None, None + if q in self._index: + return q, self._index[q] + for slug in self._index: + if slug.lower().endswith("/" + q) or slug.lower() == q: + return slug, self._index[slug] + candidates = [t for _, t in self._slug_titles] + match = difflib.get_close_matches(q, candidates, n=1, cutoff=0.6) + if match: + for slug, title in self._slug_titles: + if title == match[0]: + return slug, self._index[slug] + return None, None + + @command.passive(regex=_CMD_RE) + async def dispatch(self, evt: MessageEvent, match) -> None: + verb = match[1].lower() + body = (match[2] or "").strip() + if verb == "ask": + await self._handle_ask(evt, body) + elif verb == "doc": + await self._handle_doc(evt, body) + elif verb == "wiki": + await self._handle_wiki(evt, body) + + async def _handle_ask(self, evt: MessageEvent, body: str) -> None: + if not body: + await evt.reply( + f"Usage: `!ask ` — search {self.config['site_name']}." + ) + return + if not self._index: + await evt.reply("Wiki index isn't ready yet; try again in a moment.") + return + hits = self._search(body, self.config["max_results"]) + if not hits: + await evt.reply(f"No matches in {self.config['site_name']} for that.") + return + terms = _tokens(body) + snippet_chars = self.config["snippet_chars"] + lines = [f"**{self.config['site_name']} — {len(hits)} match(es):**"] + for score, slug, doc in hits: + url = _make_url(self.config["docs_url"], slug) + snip = _snippet(doc["content"], terms, snippet_chars) + lines.append(f"- **[{doc['title']}]({url})** — {snip}") + await evt.reply("\n".join(lines)) + + async def _handle_doc(self, evt: MessageEvent, body: str) -> None: + if not body: + await evt.reply("Usage: `!doc ` — open a specific page.") + return + if not self._index: + await evt.reply("Wiki index isn't ready yet; try again in a moment.") + return + slug, doc = self._lookup(body) + if slug is None or doc is None: + await evt.reply( + f"No page matches `{body}`. Try `!ask {body}` for a fuzzy search." + ) + return + url = _make_url(self.config["docs_url"], slug) + snippet_chars = self.config["snippet_chars"] * 2 + preview = (doc["content"] or "").strip().replace("\n", " ") + if len(preview) > snippet_chars: + preview = preview[:snippet_chars].rstrip() + "…" + await evt.reply(f"**[{doc['title']}]({url})**\n{preview}") + + async def _handle_wiki(self, evt: MessageEvent, body: str) -> None: + body = body.strip().lower() + if body == "refresh": + try: + await self._refresh() + await evt.reply(f"🔄 Refreshed {len(self._index)} docs.") + except Exception as e: + await evt.reply(f"Refresh failed: {e}") + return + if body in ("", "status"): + age = int(time.time() - self._last_refresh) if self._last_refresh else None + age_str = f"{age}s ago" if age is not None else "never" + await evt.reply( + f"**{self.config['site_name']}** — {len(self._index)} docs, " + f"last refresh: {age_str}\n" + f"Source: {_make_url(self.config['docs_url'], self.config['index_path'])}\n" + f"Use `!ask ` or `!doc `." + ) + return + await evt.reply("Usage: `!wiki` (status) or `!wiki refresh` (re-index).")