Storage & isolation

Why dual storage

Agent workflows have two conflicting storage needs:

Need	Tension
Fast — current session reads/writes must feel instant	Object storage RTT is too slow
Durable — files must survive session end	Ephemeral container disks lose data

Combo Agent uses local + MinIO: local for active Sessions; MinIO for cross-node / time-durable isolation and persistence.

Rendering diagram…

Layering rules

Layer	Purpose	Characteristics
Local filesystem	Hot Session scratch	Fast, single-node, lost if container is recycled
MinIO object store	Durable, shared, tenant-isolated	Higher latency, clustered, reliable

Rules (from engineering practice):

✅ Local is the first write target (speed); MinIO is async backup
✅ Don’t persist inside containers only — write to host mounts or MinIO
✅ MinIO enforces isolation; local holds active work — don’t invert roles

Strict multi-tenant isolation

Storage namespaces tenant → user → session:

minio/
├── tenant-A/
│   ├── user-1/
│   │   ├── session-abc/
│   │   │   ├── inputs/        # uploaded originals
│   │   │   ├── artifacts/     # Agent outputs
│   │   │   └── cache/         # intermediate cache
│   │   └── session-def/
│   └── user-2/
└── tenant-B/
    └── ...

Tenants — physical isolation (separate buckets or mandatory prefixes)
Users — logical isolation (path prefix + ACL checks)
Sessions — isolated by default within a user; explicit Graft / Memory can bridge
All access passes Credential checks — no direct bucket bypass

Supported backends

Backend	Notes
Local filesystem	Default; dev / small / single-node private deploy
MinIO (recommended)	Self-hosted S3-compatible
AWS S3 / Alibaba OSS / Tencent COS	Cloud-native
NFS / Ceph	Enterprise storage via S3 gateway

KB data’s three storage layers

Beyond Session files, KBs have structured artifacts:

Data	Where
Original files (PDF / DOCX / MP4)	Object store (MinIO / S3)
Parsed chunks / vectors / metadata	Search engine (Elasticsearch)
Entities / relations / graph	Search + graph DB (GraphRAG)
Session memory (KB / LTM facts)	Search (`sources=kb / fact` tags)

See Memory.

Async sync

Session file lifecycle:

1. Agent writes → local first
2. Return immediately after local write (Agent continues)
3. Background worker uploads to MinIO
4. On success → metadata marked synced
5. Cross-node failover → pull latest from MinIO to new local

Consistency

Same Session routes to one node to avoid write races
Failover: new node resumes from latest synced state on MinIO
Unsynced local changes retry with idempotency

File lifecycle

Phase	Behavior
Active	Local + MinIO copies
Session archived	Local trimmed; MinIO kept
Long-term archive (policy)	MinIO hot → cold tier
Delete	Retention policy driven (default retain; compliance cycles optional)

Encryption & compliance

In transit — TLS between components
At rest — MinIO SSE-S3 / SSE-KMS
Audit — read/write operations logged for reconciliation
Compliance — private deploy for data residency; SaaS boundaries per contract

Session
Memory
Multi-tenancy
Deploy
Code entrypoints: AgentFlow/src/storage/ · api/utils/file_utils.py

Storage & isolation

On this page