Structured-Memory-Engine

<p align="center">
  <img src="assets/banner.jpg" width="600" alt="Structured Memory Engine">
</p>

<p align="center">
  <strong>Your AI agent forgets everything between sessions. SME fixes that.</strong>
</p>

<p align="center">
  <a href="https://www.npmjs.com/package/structured-memory-engine"><img src="https://img.shields.io/npm/v/structured-memory-engine?color=green" alt="npm"></a>
  <img src="https://img.shields.io/badge/tests-1%2C451-brightgreen" alt="tests">
  <img src="https://img.shields.io/badge/recall-5ms-blue" alt="recall">
  <img src="https://img.shields.io/badge/cost-%240%2Fmo-orange" alt="cost">
</p>

```bash
npm install structured-memory-engine
```

## Quick Start (30 seconds)

```bash
# Install
npm install -g structured-memory-engine

# Initialize a workspace
cd ~/my-project
sme init

# Query your memory
sme query "what did we decide about the database?"

# Get auto-context for AI messages
sme context "summarize the migration plan"
```

That's it. No API keys, no config, no cloud. Everything runs locally.


```js
const sme = require('structured-memory-engine').create({ workspace: '.' });
sme.index();
sme.context('What did Sarah say about the migration plan?');
// → Ranked, cited, confidence-scored context from 60+ meeting transcripts in ~5ms
```

## The Problem

AI agents have amnesia. Every session starts from zero. Your agent doesn't remember what you decided yesterday, who you talked to last week, or what matters most to you. The workarounds — stuffing everything into system prompts, manually searching files, hoping the context window holds — don't scale.

## What SME Does About It

| Capability | What it does | Tangible benefit |
|-----------|-------------|-----------------|
| **Auto-Recall** | Injects relevant memories into every agent turn automatically | **Zero manual searching** — agent just *knows* things without being asked |
| **Confidence Scoring** | Tags facts as confirmed, inferred, or outdated with decay over time | **No more stale info served confidently** — outdated facts are deprioritized 6x vs confirmed |
| **Entity Graph** | Tracks relationships between people, projects, and topics | **Ask about "Sarah" → also get "Nexus" context** — 40-60% more relevant results on entity-heavy queries |
| **6-Signal Ranking** | Scores results by keyword match + recency + confidence + type + file weight + entity overlap | **Top result is the right result** — not just the one with the most keyword hits |
| **Contradiction Detection** | Smart detection with same-file exclusion, temporal awareness, and proximity checks | **Real contradictions, not false alarms** — 80%+ fewer false positives vs naive negation matching |
| **Contradiction Resolution** | CLI commands to resolve conflicts without editing source files | **One command fixes it** — `resolve 42 --action keep-newer` instead of hunting through markdown |
| **Configurable Decay** | Tunable half-life and decay rate per workspace | **Your workspace, your rules** — fast-moving projects decay in 90 days, core knowledge lives forever |
| **Memory Lifecycle** | Automatic decay, reinforcement, staleness detection, and pruning | **Self-cleaning index** — frequently-used memories get stronger, stale ones fade. Zero maintenance. |
| **Auto-Capture** | Detects decisions, preferences, and facts from conversation and saves them | **Never "remember to write it down" again** — 3 captures/turn, SHA-256 dedup, zero friction |
| **Transcript Ingestion** | Parses meeting recordings into tagged, searchable markdown | **60 meetings → searchable in one command.** Every decision, action item, and quote indexed. |
| **Token Budgeting** | Retrieves only what fits in a configurable token window | **No context overflow** — relevant memories in 1,500 tokens, not 50,000 |
| **Offline / Zero Cost** | SQLite FTS5 + local embeddings, no API calls | **$0/month forever.** No rate limits, no API keys, no vendor lock-in |
| **Query Intent Detection** | Classifies queries as action, reasoning, temporal, or factual — applies specialized scoring per type | **"What should I focus on?" pulls action items** — not random keyword matches |
| **Priority File Injection** | Action queries automatically surface open-loops and recent self-reviews regardless of keyword match | **Critical context never buried** — your todo list always surfaces when you ask for priorities |
| **Rule Chunk Penalty** | Detects policy/rule content and deprioritizes it for factual queries | **"What did I buy?" returns purchases** — not your Amazon account rules |
| **Forward-Looking Temporal** | "What's coming up in March?" searches for future events even when no files are dated in March | **Future planning works** — finds upcoming events, deadlines, and milestones |
| **Recall Test CLI** | Built-in test harness: `npm run test:recall` scores 6 standard queries with anti-term detection | **Regression-proof** — every change validated against real recall quality |
| **Synonym Expansion** | Bridges vocabulary gaps — "money" finds portfolio content, "health" finds bloodwork | **Natural language works** — no need to use exact keywords from stored memories |
| **Result Diversity** | File cap (3), heading cap (2), cosine similarity filter (0.80) | **No duplicate context waste** — every injected chunk adds unique information |
| **Recall Benchmarking** | 10-case test suite with content/type hit rate grading | **Quantified recall quality** — `sme benchmark` reports exactly what's working and what isn't |
| **Memory Promotion** | Finds high-value daily memories for elevation to long-term storage | **Important facts don't fade** — auto-identifies what deserves permanent memory status |

## Before & After

**Without SME:**
```
User: "What did we decide about the database migration?"
Agent: "I don't have context on that. Could you remind me?"
```

**With SME (auto-recall):**
```
User: "What did we decide about the database migration?"

## Recalled Context (auto-injected, 3 chunks, 847 tokens)
- [decision] Going with PostgreSQL on AWS for the main database. Sarah confirmed parameters.
  Source: memory/2026-02-20.md:45 | confidence: 1.0
- [fact] Target connection pool size 50, failover monitoring via CloudWatch alerts
  Source: memory/2026-02-21.md:23 | confidence: 0.95
- [action_item] Sarah to send final migration runbook by Friday
  Source: ingest/nexus-standup-feb19.md:112 | confidence: 0.85

Agent: "We decided on PostgreSQL on AWS. Sarah confirmed the parameters — pool size 50 with
        CloudWatch failover monitoring. She owes us the final migration runbook by Friday."
```

**The difference:** The agent answered with specifics, citations, and confidence levels — without being asked to search. That context was auto-injected before the agent even started thinking.

## How It Works

Every time your agent receives a message, SME runs a 6-step pipeline in <50ms:

1. **Extract** — Key terms and entity names from the user's message + recent conversation
2. **Expand** — Entity graph adds related entities (mention "Sarah" → also match "Nexus")
3. **Query** — Dual FTS5 search: AND query for precision, OR query with alias expansion for recall
4. **Rank** — 6-signal scoring: keyword relevance + semantic similarity + recency + type priority + file weight + entity overlap, multiplied by confidence^1.5
5. **Budget** — Top chunks selected within a token limit (default 1,500), cleanly truncated
6. **Inject** — Formatted as cited context with confidence warnings and contradiction flags

Markdown files are always the source of truth. The SQLite index is derived and fully rebuildable. SME never modifies your files.

## Benchmarks

Measured on Apple M3 Max, 69GB RAM, Node v24.13.0. Run `npm run bench` to verify on your hardware.

| Operation | Dataset | Avg | p95 | Notes |
|-----------|---------|-----|-----|-------|
| Full index | 100 files → 500 chunks | 31ms | — | Cold start |
| Incremental reindex | 2/100 files changed | 2.2ms | — | mtime-based skip |
| Query (FTS5) | 10 qu

[truncated…]