Storage Backends — Migration Guide

Prompt-first procedure: Describe the outcome you want in your agent conversation. The agent should select and load the appropriate AIWG assets, explain material changes, request any needed approval, and report verification evidence. Exact commands and flags appear only in the CLI reference.

Current CLI versus scalable protocol: This page documents the currently

shipped path-oriented the agent-owned storage operation command. Its JSONL receipt and

pathname-only resume behavior do not satisfy the scalable migration contract.

The versioned `aiwg.storage-migration/v1` coordinator is documented in

Scalable Storage Migration Protocol; it provides

revision/digest receipts, tombstones, atomic batches, online replay, parity

verification, approval-bound cutover, and rollback, but is not yet wired into

this CLI command. Do not use the legacy command as scale-out certification

evidence.

the agent-owned storage operation copies entries from one backend to another. Use it when you decide to move a subsystem's persistence — for example, AIWG memory was on the local filesystem, you've now installed Obsidian, and you want the existing pages in your vault.

Synopsis

Use AIWG to complete this documented outcome: Synopsis
Have it inspect the current state, explain the plan, ask before material
changes, and report the result with verification evidence.

Spec format: `<type>:<location>`

Type	Location format	Example
`fs`	directory path (relative or absolute)	`fs:.aiwg/memory` or `fs:/mnt/store`
`obsidian`	vault directory	`obsidian:~/vaults/main` (use the to-folder option for subfolder)
`logseq`	graph directory	`logseq:~/.logseq/graphs/work`
`fortemi`	MCP server name	`fortemi:fortemi`

`fortemi:<server-name>` targets the legacy Fortemi MCP storage adapter only. It does not create or update Fortemi Core index/search caches; run the agent-owned index operation for that path.

Optional flags:

• `the from-folder option <subfolder>` — Obsidian-only, applies to source
• `the to-folder option <subfolder>` — Obsidian-only, applies to destination
• the dry-run option — preview operations; no writes

Examples

Move AIWG memory from filesystem to an Obsidian vault

Use AIWG to complete this documented outcome: Move AIWG memory from filesystem to an Obsidian vault
Have it inspect the current state, explain the plan, ask before material
changes, and report the result with verification evidence.

Relocate the research corpus to a secondary drive

Use AIWG to complete this documented outcome: Relocate the research corpus to a secondary drive
Have it inspect the current state, explain the plan, ask before material
changes, and report the result with verification evidence.

Move agent-loop memory into Fortemi MCP storage

Use AIWG to complete this documented outcome: Move agent-loop memory into Fortemi MCP storage
Have it inspect the current state, explain the plan, ask before material
changes, and report the result with verification evidence.

(`fortemi:<server-name>` references an MCP server registered via the agent-owned mcp operation.)

Ingest the research corpus into a memory store

The provider-neutral corpus command selects textual research records, preserves Markdown frontmatter as adapter metadata, and skips binary attachments rather than decoding them as UTF-8. Previewing never connects to the destination:

Use AIWG to complete this documented outcome: The provider-neutral corpus command selects textual research records, preserves Markdown frontmatter as adapter metadata, and skips binary attachments rather than decoding them as UTF-8. Previewing never connects to the destination
Have it inspect the current state, explain the plan, ask before material
changes, and report the result with verification evidence.

Fortemi destinations use deterministic IDs of the form `research:<corpus-relative-path>`, so the resumable AIWG migration receipt replaces the old machine-specific `import-ids.log`. Binary attachment upload is not part of the text-based `StorageAdapter` contract; skipped paths are counted explicitly and remain in the source corpus for a future binary-attachment adapter rather than being corrupted or silently discarded.

For an authenticated internal/Enterprise service, register an HTTPS MCP server with a credential reference. Only the environment-variable name is persisted:

Use AIWG to complete this documented outcome: For an authenticated internal/Enterprise service, register an HTTPS MCP server with a credential reference. Only the environment-variable name is persisted
Have it inspect the current state, explain the plan, ask before material
changes, and report the result with verification evidence.

Set `AIWG_FORTEMI_TOKEN` in the approved process secret provider before the live run. AIWG does not read legacy Claude credential files and does not accept a token in `storage.config`. Non-loopback remote endpoints must use HTTPS.

How it works

1. Build adapters — one each from the from option and the to option specs. 2. Refuse identical locations — if both specs resolve to the same physical location, the command aborts with a clear error before touching anything. 3. Walk source — calls `source.list('')` to enumerate every entry. 4. Per-entry copy — reads from source, writes to destination. Per-entry errors don't abort the migration; they're tallied at the end. 5. Resume tracking — appends a JSONL line per successfully-migrated entry to `.aiwg/.storage-cache/migrations/<subsystem>-<from>-to-<to>.jsonl`. Re-running the same command picks up where it left off.

Resume behavior

Long migrations can be interrupted (Ctrl-C, a network hiccup, a backend rate limit). Re-running the same command:

• Reads the migration log
• Skips any entry whose path appears in the log
• Retries the rest

This means you can safely run a migration in chunks, or re-run after fixing a backend issue. The command prints `skipped=N` in the summary so you can see what the resume picked up.

Because completion is keyed only by pathname, changing an already-receipted source entry or deleting it does not update the destination on rerun. Use this legacy path only for quiesced, backup-first copies and perform an independent source/destination comparison before changing configuration.

Dry-run mode

the dry-run option lists every operation that would happen but performs no writes. Use it to:

• Sanity-check the source has the entries you expect
• Confirm the to option resolves to the intended physical location (especially with the to-folder option)
• Estimate migration size before committing

The dry-run output uses `→` to mark planned operations:

storage migrate (DRY RUN)
  subsystem: memory
  from:      fs:./aiwg/memory
  to:        obsidian:/home/user/vaults/main (folder=AIWG/memory)

  → page-001.md (would copy)
  → page-002.md (would copy)
  → research-complete/index.md (would copy)

Summary: copied=3 skipped=0 errored=0 total=3

Error handling

Per-entry errors don't abort the migration. The summary line tallies `copied / skipped / errored / total`, and the command exits non-zero when `errored > 0`.

Common causes:

Symptom	Likely cause	Fix
`read returned null`	Source entry vanished mid-migration	Re-run; resume skips already-copied
`refusing to operate on .obsidian/`	Source contained Obsidian config files	Pre-clean source, or use the to-folder option
`rate limit / 429`	Notion / AnythingLLM throttling	Wait and re-run; resume skips done entries
`MCP server not registered`	`fortemi:<name>` references unknown server	the agent-owned mcp operation first

Backwards-compatible after migration

After a successful migration, you'll typically update `storage.config` so AIWG reads from the new location going forward. The migration command itself does NOT update `storage.config` — you control that explicitly. This means:

• You can run a migration to test a backend without committing
• You can keep both source and destination in sync briefly while you validate
• You can roll back by reverting `storage.config` (the source data is untouched)

What can and can't be migrated

Source → Destination	Status
`fs` → `fs`	✓ supported
`fs` → `obsidian` / `logseq`	✓ supported
`obsidian` / `logseq` → `fs`	✓ supported
`obsidian` ↔ `logseq`	✓ supported (frontmatter transformation handled)
`fs` / file-shaped → `fortemi`	✓ supported (alpha; Fortemi tool surface unverified)
`fortemi` → file-shaped	✓ supported (alpha)
`notion` / `anythingllm` / `s3` / `webdav`	✗ adapters not yet implemented (#959/#960/#962/#963)

When the deferred adapters land, this table updates. The migrate command will refuse unimplemented backend types in the from option/the to option with a clear error citing the tracking issue.

Edge cases worth knowing

• `fs → fs` between the same root is refused. AIWG checks resolved paths and aborts before any side effect.
• Empty source is reported gracefully. No-op, exit 0.
• The migration log lives under `.aiwg/.storage-cache/migrations/` — that directory is gitignored by default. Don't commit it.
• Large migrations: each entry is read and written serially. If you have 10,000 entries against a rate-limited backend, expect minutes. Use the dry-run option first to estimate.