Storage & PKM Integration

Storage & PKM Integration

By default AIWG persists everything under `.aiwg/` in your project. That's fine for most projects. But when you want AIWG memory to flow into your existing personal-knowledge system — Obsidian, Logseq, Fortemi — or you want to move heavy artifacts (research papers, media caches) onto a secondary drive, AIWG has a pluggable storage system that handles this with one config file.

This guide walks through the common moves.


When to set this up

Skip this if:

  • You're new to AIWG and want to focus on the SDLC workflow first
  • `.aiwg/` is fine where it lives (you're not running out of space, you don't use a separate PKM)

Set it up when:

  • You already use Obsidian or Logseq daily and want AIWG memory in your graph
  • You're acquiring a large research corpus and your project drive is filling up
  • You're running multiple projects and want a single shared memory store

Five-minute setup: route AIWG memory into Obsidian

You have an Obsidian vault at `~/vaults/main` and want AIWG memory pages to land in `AIWG/memory/` inside it.

1. Create `.aiwg/storage.config`:

   {
     "version": "1",
     "backends": {
       "memory": {
         "type": "obsidian",
         "vault": "~/vaults/main",
         "folder": "AIWG/memory"
       }
     }
   }

2. Verify:

   aiwg storage show              # confirms memory routes to Obsidian
   aiwg storage list-backends     # confirms obsidian is READY
   aiwg storage test memory       # round-trip write/read/list/delete probe

3. (Optional) Move existing AIWG memory into the vault:

   aiwg storage migrate memory \
     --from fs:.aiwg/memory \
     --to obsidian:~/vaults/main \
     --to-folder AIWG/memory \
     --dry-run

   # Re-run without --dry-run when the preview looks right

That's it. Every memory write from AIWG now lands in your vault. Obsidian's file watcher picks up changes within seconds.

The same pattern works for `kb`, `reflections`, `provenance`, etc. — just change the subsystem key in `backends`.


Three-minute setup: relocate the research corpus to a different drive

You have a research corpus that's getting big. You want it on `/mnt/archive/` instead of taking space on the project drive.

1. Add a path override in `.aiwg/storage.config`:

   {
     "version": "1",
     "roots": {
       "research": "/mnt/archive/aiwg-research"
     }
   }

2. Migrate existing data:

   aiwg storage migrate research \
     --from fs:.aiwg/research \
     --to fs:/mnt/archive/aiwg-research

3. Verify:

   aiwg storage show           # research → /mnt/archive/aiwg-research
   aiwg research-store path    # confirms the new resolved path

The research corpus now lives on the secondary drive. Same access pattern from AIWG; nothing else changes.


Subsystems you can route

SubsystemWhat lives hereDefault location
`memory`Consumer-framework semantic memory (pages, lessons)`.aiwg/memory/`
`reflections`Agent-loop reflections`.aiwg/reflections/`
`kb`Knowledge-base pages`.aiwg/kb/`
`activity_log`Cross-framework event log`.aiwg/activity.log`
`provenance`W3C PROV records`.aiwg/provenance/`
`research`Research corpus`.aiwg/research/`
`media`Media artifacts`.aiwg/media/`
`sandbox_identity`Persistent agent identities`~/.config/aiwg/sandbox-agents.json`

Each subsystem can be configured independently. Mix and match.


Backends available today

BackendStatusUse case
`fs`READY (default)Local filesystem. Free, fast, no external deps
`obsidian`READYYou already use Obsidian as your PKM
`logseq`READYYou already use Logseq as your PKM
`fortemi`READY (alpha)First-party AIWG semantic-memory; Postgres + pgvector

Stub-tracked (planned, refuses with a clear error if configured today):

  • `notion` (#959), `anythingllm` (#960), `s3` (#962), `webdav` (#963)

Security: tokens never go in storage.config

Every credential — API tokens, passwords, S3 keys — comes from environment variables, never from `.aiwg/storage.config`. The schema actively rejects credential-named properties at every nesting depth, and `aiwg doctor` does a recursive walk as defense-in-depth.

This means: it's safe to commit `.aiwg/storage.config` to git. It's NOT safe to put credentials in it.

See `docs/storage/security.md` for the full security model.


What to do next

  • Configure another subsystem — repeat the five-minute setup with `kb`, `reflections`, etc.
  • Read the per-backend page — `docs/storage/backends/<type>.md` covers env vars, caveats, and setup specifics.
  • Migrate in chunks — `aiwg storage migrate` resumes on retry, so you can run it incrementally on a large corpus.
  • Verify with the doctor — `aiwg doctor` validates `storage.config` and probes each declared backend.

For the full reference, see `docs/storage/`.