palinode

# Palinode 🧠

**Persistent long-term memory for AI agents — with provenance.**

*A palinode is a poem that retracts what was said before and says it better.
That's what memory compaction does.*

Git-native. Markdown-first. No database required.

---

## The Problem

AI agents wake up with amnesia every session. They don't remember who you are, what you're working on, or what was decided yesterday. You waste turns re-explaining context that should already be there. Current solutions either don't scale (flat files), produce uncurated noise (vector-only stores), or lock you into opaque databases you can't inspect.

## The Solution

Palinode gives your agent **14 MCP tools and a memory directory** — the agent decides what to remember, what to search, and when to consolidate. No rigid pipeline. No framework lock-in. Just tools, files, and git.

An MCP server works with Claude Code, Cursor, Codex, Antigravity, OpenClaw, or any MCP client. A session skill auto-captures milestones during coding. A deterministic executor handles compaction — the LLM proposes operations, the executor applies them. The LLM never touches your files directly.

Storage is typed markdown with YAML frontmatter. Search is hybrid BM25 + vector. History is git. Works with any LLM backend. If every service crashes, `cat` still works.

### Built for model step-changes

Most AI systems accumulate "compensating complexity" — workarounds for the last model's weaknesses that become constraints when a better model arrives. Palinode is designed to survive model upgrades:

- **Tools are the interface, not a pipeline.** Your agent calls `palinode_search` when it needs context, not because a pipeline forces 300 tokens of search results into every turn. Smarter models make better retrieval decisions on their own.
- **The executor is deterministic.** The LLM proposes KEEP/UPDATE/MERGE/SUPERSEDE/ARCHIVE. The executor validates and applies. Swap the LLM, keep the executor. ([ADR-001](docs/ADR-001-tools-over-pipeline.md))
- **Files and git don't depend on any model.** Markdown, YAML, `git blame` — none of this changes when Mythos or GPT-5 drops.
- **Auto-injection is optional scaffolding.** The OpenClaw plugin injects context automatically today (useful for current models). As models get smarter and use tools proactively, the injection pipeline becomes unnecessary — the tools remain.

### What it costs

| Turn | What happens | Tokens |
|---|---|---|
| Turn 1 | Core memory injected (people, projects, decisions) | ~4,200 |
| Every turn after | Relevant search snippets for your message | **~300** |
| Trivial messages ("ok", "yes", "👍") | Nothing — skipped automatically | **0** |

~300 tokens per turn. Less than a sentence of output. The alternative — re-explaining your project every session — costs the same tokens with none of the benefit.

---

## What Makes Palinode Different

Most agent memory systems are opaque databases you can't inspect, flat files that don't scale, or graph stores that require infrastructure. Palinode is **memory with provenance** — the only system where you can `git blame` every fact your agent knows.

### No other production system has these:

- **Git blame/diff/rollback as agent tools** — not just git-compatible files, but `palinode_diff`, `palinode_blame`, and `palinode_rollback` as first-class MCP tools your agent can call. [DiffMem](https://github.com/search?q=diffmem) and Git-Context-Controller are PoCs; Palinode ships 14 MCP tools including 5 git operations.

- **Operation-based compaction with a deterministic executor** — the LLM outputs structured ops (KEEP/UPDATE/MERGE/SUPERSEDE/ARCHIVE), a deterministic executor applies them. The LLM never touches your files directly. [All-Mem](https://arxiv.org/search/?query=all-mem+memory) does something similar on graph nodes; Palinode does it on plain markdown with git commits.

- **Per-fact addressability** — every list item gets an invisible `<!-- fact:slug -->` ID that survives git operations and is targetable by compaction. memsearch has per-chunk (heading-level); Hermes has per-entry (delimiter). Nobody has inline fact IDs.

- **4-phase injection pipeline** — Core → Topic → Associative → Triggered. Individual phases exist elsewhere (Letta core, LangMem search, Zep graph, ADK preload), but no system combines all four. [Perplexity deep research confirms](docs/perplexity-landscape-2026-03-31.md): "No widely documented system matches a four-phase pipeline with exactly the requested semantics."

- **If every service crashes, `cat` still works** — your memory is markdown files in a directory. Rebuild the index from files anytime.

---

## Features

> ✅ = production-ready &nbsp; 🧪 = implemented, beta &nbsp;

### Memory Storage ✅
- **Typed memories** — people, projects, decisions, insights, research (not flat text blobs)
- **Layered structure** — files split into Identity (`name.md`), Status (`-status.md`), and History (`-history.md`)
- **Fact IDs** — persistent, unique IDs (`<!-- fact:slug -->`) for precise auditing and compaction
- **YAML frontmatter** — structured metadata, categories, entity cross-references
- **Git-versioned** — every memory change has a commit, `git blame` your agent's brain
- **Graceful degradation** — vector index down → files still readable, grep still works

### Capture ✅
- **Session-end extraction** — auto-captures key facts from conversations to daily notes
- **`-es` quick capture** — append `-es` to any message to route it into the right memory bucket
- **Inbox pipeline** — drop PDFs, audio, URLs into a watch folder; they appear as research references

### Recall
- ✅ **Core memory injection** — files marked `core: true` are always in context
- ✅ **Tiered injection** — full content on turn 1, summaries on subsequent turns (saves tokens)
- ✅ **Hybrid search** — BM25 keyword matching + vector similarity merged with Reciprocal Rank Fusion
- ✅ **Content-hash dedup** — SHA-256 hashing skips re-embedding unchanged files (~90% savings)
- 🧪 **Temporal decay** — re-ranks results based on freshness and importance (beta — decay constants need tuning)
- 🧪 **Associative recall** — spreading activation across entity graph (beta)
- 🧪 **Prospective triggers** — auto-inject files when trigger contexts match (beta)

### Compaction 🧪
- **Operation-based** — LLM outputs JSON ops, deterministic executor applies them
- **Layered files** — Identity (slow-changing) / Status (fast-changing) / History (archived)
- **Weekly consolidation** — cron-driven, local LLM (OLMo/vLLM), git commits each pass
- **Security scanning** — blocks prompt injection and credential exfiltration in memory writes

### Integration ✅
- **OpenClaw plugin** — lifecycle hooks for inject, extract, and capture
- **MCP server** — 13 tools for Claude Code and any MCP client
- **FastAPI server** — HTTP API for programmatic access
- **CLI** — command-line search, stats, reindex

---

## Architecture

```mermaid
graph TD
    subgraph Capture
        T[Telegram] --> OC[OpenClaw Agent]
        S[Slack] --> OC
        W[Webchat] --> OC
        CC[Claude Code] --> MCP[MCP Server]
    end

    subgraph Processing
        OC -->|session end| EX[Extract to daily notes]
        OC -->|"-es" flag| QC[Quick capture + route]
        MCP -->|palinode_save| SV[Write markdown file]
    end

    subgraph Storage
        EX --> MD[~/palinode/*.md]
        QC --> MD
        SV --> MD
        MD -->|file watcher| IDX{Index}
        IDX -->|embed| VEC[(SQLite-vec)]
        IDX -->|tokenize| FTS[(FTS5 BM25)]
    end

    subgraph Recall
        Q[Search query] --> HYB{Hybrid search}
        HYB --> VEC
        HYB --> FTS
        HYB -->|RRF merge| RES[Ranked results]
    end
```

### Stack

| Layer | Technology | Why |
|---|---|---|
| Source of truth | Markdown + YAML frontmatter | Human-readable, git-versioned, portable |
| Semantic index | SQLite-vec (embedded) | No server, single file, zero config |
| Keyword index | SQLite FTS5 (embedded) | BM25 for exact terms, stdlib — 

[truncated…]