Storage Architecture

How AIWG storage is organized

Storage Backends — Overview

AIWG persists artifacts (memory pages, knowledge-base entries, activity log, reflections, provenance records, research corpus, sandbox identities) through a pluggable storage adapter system. By default everything lives on the local filesystem under `.aiwg/`. With one config file you can route any subsystem to:

Obsidian vault — your AIWG memory becomes searchable via the graph
Logseq graph — pages flow into your daily journal workflow
Fortemi — legacy MCP semantic-memory storage adapter (Rust + Postgres + pgvector); deprecated for index/search routing
A different filesystem location — heavy artifacts on a secondary drive
Any future backend (S3, WebDAV, Notion, AnythingLLM — tracked, not yet shipped)

Each subsystem is configured independently. You can keep the activity log on local disk for compliance while routing memory pages into Obsidian and the research corpus to a network share.

Fortemi Integration Planes

Do not conflate storage configuration with AIWG's two file-based Fortemi integrations:

Plane	Configuration / command	Semantics
Static index	`aiwg index sync`, `aiwg index query`	Rebuildable local AIWG v2 search and graph cache consumed through Fortemi Core
Portable shard conversion	`aiwg index export --format fortemi-shard`	Explicit v2 index-to-shard conversion; server compatibility is profile-scoped and requires published-package plus real server-import evidence
Live MCP persistence	`"type": "fortemi"` in `.aiwg/storage.config`	Reads and writes configured subsystems through a running Fortemi service

Success in one plane is not evidence for another. In particular, `aiwg storage test` exercises live MCP persistence only; it does not validate the static index contract or a Knowledge Shard.

Quick start

If you do nothing, AIWG works exactly like before — every subsystem persists under `.aiwg/`.

For private per-project memory that should not be committed, AIWG can resolve the `memory` subsystem from a user-level project registry under `~/.aiwg/projects/` when no project-local `.aiwg` directory is present. See `docs/user-level-project-memory.md`.

To change behavior, create `.aiwg/storage.config`:

{
  "$schema": "https://aiwg.io/schemas/storage.config.v1.json",
  "version": "1",
  "roots": {
    "research": "/mnt/archive/aiwg-research"
  },
  "backends": {
    "memory":      { "type": "obsidian", "vault": "~/vaults/main", "folder": "AIWG/memory" },
    "kb":          { "type": "logseq",   "graph": "~/.logseq/graphs/work" },
    "activity_log":{ "type": "fs" }
  }
}

Then verify:

aiwg storage show           # effective config + resolved paths
aiwg storage list-backends  # which adapters are READY vs STUB
aiwg storage test memory    # round-trip write/read/list/delete probe

Subsystems

AIWG defines eight subsystems. Each one can be configured independently in `storage.config`.

Subsystem	Default location	What lives here
`memory`	`.aiwg/memory/`	Consumer-framework semantic memory (pages, lessons)
`reflections`	`.aiwg/reflections/`	Agent-loop reflections, ralph-reflect output
`kb`	`.aiwg/kb/`	Knowledge-base entity/concept pages
`activity_log`	`.aiwg/activity.log`	Cross-framework chronological event log
`provenance`	`.aiwg/provenance/`	W3C PROV records
`research`	`.aiwg/research/`	Research corpus — papers, summaries, citations
`media`	`.aiwg/media/`	Media artifacts (curator framework)
`sandbox_identity`	`~/.config/aiwg/sandbox-agents.json`	Persistent agent-instance identities (host-level)

Configuration shape

`.aiwg/storage.config` is JSON, validated against `https://aiwg.io/schemas/storage.config.v1.json`.

{
  "version": "1",                         // required; v1 is the only currently-supported version
  "roots": {                              // optional path overrides for fs-backed subsystems
    "research": "/mnt/archive/aiwg-research"
  },
  "backends": {                           // optional per-subsystem backend selection
    "memory":      { "type": "obsidian", "vault": "~/vaults/main" },
    "kb":          { "type": "logseq",   "graph": "~/.logseq/graphs/work" },
    "activity_log":{ "type": "fs" }
  },
  "fallback": "cache_and_warn"            // optional: cache_and_warn (default) | block
}

`roots` redirects the physical filesystem location of an `fs`-backed subsystem. No effect on non-`fs` backends.
`backends` picks the adapter type per subsystem. Subsystems not listed default to `fs`.
`fallback` controls behavior when an external backend is unreachable. `cache_and_warn` (default) queues writes under `.aiwg/.storage-cache/` and continues; `block` refuses the write.

Every backend's required fields are documented in `docs/storage/backends/<type>.md`.

CLI surface

Command	Purpose
`aiwg storage show`	Print the effective config and resolved physical paths per subsystem
`aiwg storage list-backends`	Inventory of compiled-in adapters with READY/STUB status
`aiwg storage test <subsystem>`	Round-trip write/read/list/delete probe — proves connectivity
`aiwg storage migrate <subsystem> --from … --to …`	Copy entries from one backend to another. See `docs/storage/migration.md`
`aiwg doctor`	Validates `.aiwg/storage.config` (schema + credential walk + reachability probes)

Per-subsystem CLIs route through the configured adapter:

Command	Subsystem
`aiwg memory`	`memory`
`aiwg reflections`	`reflections`
`aiwg kb`	`kb`
`aiwg activity-log`	`activity_log`
`aiwg provenance`	`provenance`
`aiwg research-store`	`research`

Each one supports `path / list / get / put / delete / append-log` (where applicable). See per-skill documentation for usage.

Security

The schema actively forbids credential-named properties at every nesting depth:

token, password, secret, apiKey, api_key,
accessKey, accessKeyId, secretAccessKey

Loading a `storage.config` containing any of these throws an error pointing at the offending path. Tokens, passwords, and API keys must come from environment variables or your OS keychain, never from `storage.config`.

See `docs/storage/security.md` for the full security model — credential handling, path traversal rejection, atomic writes, doctor validation.

What's implemented vs deferred

Backend	Status	Notes
`fs`	READY	Default; local filesystem
`obsidian`	READY	Direct fs writes against vault; refuses `.obsidian/` config dir
`logseq`	READY	Direct fs writes against graph; YAML→`property::` transform
`fortemi`	READY (alpha, legacy for search)	Routes persistence through Fortemi MCP tools; index/search uses Fortemi Core by default; use `--backend local` for legacy fallback
`notion`	STUB (#959)	Planned — REST + external_id upsert
`anythingllm`	STUB (#960)	Planned — multipart upload + cache mirror
`s3`	STUB (#962)	Planned — phase 3 (bulk artifacts)
`webdav`	STUB (#963)	Planned — phase 3 (Nextcloud/ownCloud)

`aiwg storage list-backends` shows the live status. Declaring a STUB backend in `storage.config` produces a clear error citing the tracking issue.