pi-memory-stone

A global pi extension that preserves and retrieves useful memory across pi sessions. Raw pi session JSONL files remain the source of truth; the extension builds a searchable, privacy-safe SQLite+FTS5 index with backreferences to exact session entries.

Status

MVP vertical slice — deterministic indexing, FTS5 search, conservative injection, commands, tools, and tests are complete. LLM extraction, embeddings, and historical backfill are deferred.

Quick Start

Install globally from npm:

pi install npm:pi-memory-stone

Package page: https://www.npmjs.com/package/pi-memory-stone

Then restart pi or run /reload, and verify it is active:

/memory-status

Alternative installs:

# Install directly from GitHub
pi install git:github.com/nikolasp/pi-memory-stone

# Manual global extension checkout
git clone https://github.com/nikolasp/pi-memory-stone ~/.pi/agent/extensions/pi-memory-stone

Scope note: pi install -l ... / pi install --local ... writes to the current project's .pi/settings.json and only loads there. For all projects, run pi install npm:pi-memory-stone without --local (user settings) or use ~/.pi/agent/extensions/.

Architecture

~/.pi/agent/memory/memory.db      # SQLite + FTS5 (WAL mode)
pi-memory-stone package source
├── src/
│   ├── index.ts                  # Entry point: hooks, lifecycle
│   ├── db/                       # SQLite connection, migrations, CRUD
│   ├── indexing/                 # Deterministic JSONL parser, agent_end handler
│   ├── retrieval/                # FTS search, hybrid ranking, injection builder
│   ├── commands/                 # /memory-* slash commands
│   ├── tools/                    # LLM-callable tools
│   ├── vault/                    # Optional Obsidian-compatible markdown vaults
│   ├── privacy/                  # Secret redaction, sensitive path filtering
│   └── config/                   # Project identity, settings
└── test/                         # tests across memory, privacy, portable, and vault helpers

How It Works

1. Indexing (agent_end)

Every time the agent finishes a response, the extension:

• Reads new session entries since the last indexed position
• Parses turns deterministically (no LLM): extracts user prompts, assistant responses, tool calls, errors
• Redacts secrets (API keys, tokens, passwords, private keys) before storage
• Skips sensitive files (.env, keys, certs, node_modules, .git)
• Stores structured turn_summary and error_resolution records in SQLite
• Maintains index_state per session file so indexing is incremental

2. Retrieval (before_agent_start)

Before each agent turn, the extension:

• Builds a focused query from the user's prompt
• Uses memory.injectionMode to choose automatic or manual injection
• In auto mode, searches records via FTS5
• Ranks by hybrid score: FTS match + same-project boost + recency decay + kind weight + confidence
• Injects top results (max 5, threshold-limited) plus any refs selected with /memory-inject
• In manual mode, skips search and injects only refs selected with /memory-inject
• Tracks auto-injected refs to prevent feedback loops
• Logs every injection for audit via /memory-last

3. Storage Model

Records:

Kind	Description
turn_summary	Concatenated user prompt + assistant response + tools used
error_resolution	Tool errors with context
decision	Explicitly remembered decisions (LLM/tool)
preference	User preferences (LLM/tool)
task	Tracked tasks (LLM/tool)
session_summary	Session-level summary (future)

Scopes:

Scope	Visibility
project	Visible within the same project (git repo root)
global	Visible across all projects (explicit opt-in only)

Statuses: active, soft_forgotten, hard_forgotten, superseded

Commands

Command	Alias	Description
/memory-status	/stone-status	Show index statistics, record counts by kind, config
/memory-status --verbose		Include per-kind record breakdown
/memory-search <query>	/stone-search	Search memory for relevant records
/memory-open <id>	/stone-open	Open a specific memory record by reference ID
/memory-inject <id> [id ...]	/stone-inject	Manually inject specific refs into future turns this session
/memory-clear-injected	/stone-clear-injected	Clear manually injected refs for this session
/memory-mode <auto|manual>	/stone-mode	Override injection mode for this session
/memory-last	/stone-last	Show the last memory injection packet
/memory-forget <id>	/stone-forget	Soft-forget a record (hide from searches)
/memory-forget <id> --hard		Permanently delete (with confirmation)
/memory-export [path] --format json|md	/stone-export	Export active records to a portable JSON or Markdown file (--all includes inactive records)
/memory-import <memory-export.json>	/stone-import	Import records from a JSON export, remapping project-scoped records to the current project by default
/memory-import <memory-export.json> --preserve-project		Import records while preserving exported project IDs
/memory-import <memory-export.json> --global		Import all records as global memories
/memory-backup [path]	/stone-backup	Copy the SQLite memory database to a timestamped backup file
/memory-vault-init [--project|--personal]	/stone-vault-init	Initialize an Obsidian-compatible markdown vault
/memory-vault-sync [--project|--personal]	/stone-vault-sync	Generate vault pages from active memory records
/memory-vault-status [--project|--personal]	/stone-vault-status	Show vault path, page counts, registry, and last sync
/memory-vault-capture-url <url> [--project|--personal]	/stone-vault-capture-url	Capture a web page into the vault as a source page
/memory-on		Enable memory injection for this session
/memory-off		Disable memory injection for this session

Tools

memory_search

Search memory stone for relevant records. Use before making decisions to recall past context.

Parameters:
  query   Search query text
  kind?   Filter: decision | preference | task | error_resolution | turn_summary | session_summary
  scope?  Filter: project | global
  limit?  Max results (default 5)

memory_open

Open a specific memory record by its reference ID.

Parameters:
  ref     Memory record reference ID (from search results or injection packets)

memory_remember

Explicitly store a memory record. Only use when the user explicitly asks to remember something.

Parameters:
  kind         Record kind: decision | preference | task | error_resolution | turn_summary | session_summary
  text         Memory text to store
  scope?       project (default) | global
  tags?        Comma-separated tags
  importance?  0-1 (default 0.5)

memory_forget

Soft-forget a memory record by its reference ID. Hard deletion requires explicit user confirmation via command.

Parameters:
  ref     Memory record reference ID
  hard?   Request permanent deletion (requires confirmation)

Portable Export, Import, and Backup

Use JSON export/import when you want a portable, reviewable memory transfer between machines:

/memory-export --format json
/memory-import memory-export.json

Markdown export is for human review outside SQLite:

/memory-export memory-export.md --format md

Use a database backup before pruning or hard deletion:

/memory-backup

Import defaults are intentionally practical for machine moves: project-scoped records are remapped to the current project. Use --preserve-project to keep exported project IDs, or --global to import everything as global memory.

Knowledge Vaults

Memory vaults are optional Obsidian-compatible markdown projections of active memory records. SQLite remains the source of truth; generated pages may be overwritten by /memory-vault-sync.

# Project-local vault, written only after explicit init or capture request
/memory-vault-init --project
/memory-vault-sync --project
/memory-vault-status --project

# Capture a web page source into the vault
/memory-vault-capture-url https://example.com/article --project

# Capture resolves known source formats before generic HTML extraction.
# Examples: GitHub Gist pages are fetched from gist.githubusercontent.com/raw,
# GitHub blob URLs are fetched from raw.githubusercontent.com, and raw Markdown
# is preserved as Markdown.

# Natural-language capture also works from normal prompts:
# "Capture this article into vault https://example.com/article"
# "Add page to personal vault https://example.com/article"

# Private personal vault for global memories
/memory-vault-init --personal
/memory-vault-sync --personal

Default locations:

<repo>/.memory-stone/vault/              # project vault
~/.pi/agent/memory/vaults/personal/      # personal vault

Initial layout:

index.md
WIKI_SCHEMA.md
records/{decisions,preferences,tasks,error-resolutions,turn-summaries,session-summaries}/
sources/                       # captured web source pages
meta/registry.json

URL capture writes curated source notes into sources/. Raw provenance packets (manifest, metadata, fetch attempts, original artifact, extracted markdown) are stored outside the Obsidian vault under .memory-stone/source-packets/ for project vaults or ~/.pi/agent/memory/source-packets/personal/ for personal vaults. Capture extracts HTML articles with Mozilla Readability, converts to Markdown, redacts secrets, and marks extraction quality as good or weak with warnings.

Privacy & Safety

Secret Redaction

All text is redacted before storage. Patterns covered:

• OpenAI API keys (sk-...)
• GitHub tokens (ghp_..., ghs_...)
• AWS access keys (AKIA...)
• JWT tokens
• Generic API keys and secrets in key=value assignments
• Private keys (PEM format)
• Password/secrets in assignments
• Connection strings (credentials removed)

Path Filtering

Files at these paths are never indexed:

• .env*, .envrc
• Keys and certificates (.pem, .key, .crt)
• SSH keys (id_rsa, id_ed25519, .ssh/)
• AWS credentials (~/.aws/)
• GPG keys (~/.gnupg/)
• Dependency dirs (node_modules, .git, dist, build, .next, etc.)

Cross-Project Safety

• Records default to project scope
• Cross-project memory requires explicit global flag
• Global promotion refuses sensitive patterns (paths, secrets, hostnames, implementation details)
• memory_open never sends raw excerpts to LLM automatically

Ranking

Results are ranked by a hybrid score:

Factor	Effect
FTS5 match quality	Base score (normalized reciprocal rank)
Same project	×1.5 boost
Global scope	×1.2 boost
Kind weight	Decision ×1.5, Preference ×1.3, Error ×1.4
Recency	Exponential decay, half-life 7 days
Confidence	Direct multiplier
Importance	0.5–1.5x multiplier

DB Schema

sessions         — Indexed session metadata
records          — Structured memory records (with FTS5 index)
record_fts       — Full-text search index (contentless)
file_activity    — File read/write/edit/bash tracking
injections       — Audit log of memory injections
index_state      — Per-session indexing progress
jobs             — Background job queue
schema_migrations— Versioned migration tracking

Storage: ~/.pi/agent/memory/memory.db (SQLite, WAL mode, busy timeout 5s).

Tests

cd ~/.pi/agent/extensions/pi-memory-stone
npm test
npm run typecheck

These scripts are also runnable from a pi package clone installed from git; the required script runners are regular dependencies because pi package installs omit devDependencies.

84 tests across 11 test files:

Suite	Tests	Focus
config.test.ts	2	Project-root config discovery and caching
db.test.ts	4	FTS semantics and recent-file activity queries
indexing.test.ts	2	Incremental session indexing and file-activity FK integrity
parser.test.ts	11	Turn parsing, file activity detection, error extraction
portable.test.ts	5	JSON/Markdown export, JSON import, SQLite backup
privacy.test.ts	23	Secret redaction, sensitive path filtering
ranking.test.ts	18	Hybrid ranking, cross-project filtering, injection formatting
session-state.test.ts	5	Injection mode config, session ref selection, manual-only injection
tools.test.ts	2	Tool visibility and forgetting safety
vault.test.ts	11	Vault path resolution, initialization, sync, URL capture, registry generation
warnings.test.ts	1	Runtime warning suppression

Configuration

Project settings in .pi/settings.json:

{
  "memory": {
    "enabled": true,
    "maxInjectedRecords": 5,
    "maxInjectedTokens": 1000,
    "scoreThreshold": 0.3,
    "crossProjectEnabled": false,
    "injectionMode": "auto"
  }
}

injectionMode accepts:

Value	Behavior
auto	Default. Automatically searches relevant memories and also includes refs selected with /memory-inject.
manual	Disables automatic search-based injection. Only refs selected with /memory-inject are injected.

For manual-only memory, keep enabled: true and set injectionMode: "manual".

Deferred (Future Slices)

• LLM extraction (decisions, preferences, tasks)
• Historical backfill (/memory-backfill)
• Embedding-based semantic search
• Vault backlinks/lint/search/capture commands
• /memory-edit, /memory-supersede
• Rich TUI memory browser
• Multi-machine sync
• Daemon-mode background worker for LLM extraction
• /memory-prune-missing
• .pi/memoryignore and ~/.pi/agent/memoryignore

License

MIT — see LICENSE for details.