ICE Overview
The Persistent Context Engine (ICE) is a stateful intermediate layer between application logic and LLM providers. It manages session memory, context retrieval, and multi-tenant isolation.
Why ICE Exists
For Developers
Your agents remember everything. No more losing tool outputs mid-workflow. No more chatbots that forget what was said three turns ago. ICE maintains a persistent session ledger — every tool result, every turn, every document — and surfaces the right pieces at the right time, automatically.
One integration. No memory plumbing. Stop building and debugging the same pgvector + Redis + session-management stack. ICE is a drop-in layer that your app talks to exactly like any LLM endpoint. Memory becomes infrastructure you don't think about.
For Enterprises
Your customers' data never mixes. Tenant isolation is enforced at the database layer — not in application logic, not in middleware. One tenant cannot access another's session data. This is the answer your security team needs.
Smaller prompts, lower inference bills. ICE injects only what's relevant into each LLM call instead of the full history. Less tokens in, same quality out.
Runs entirely inside your network. ICE connects to the PostgreSQL instance you already operate. Nothing leaves your infrastructure.
Architecture Reference (v2.7.755)
| Component | Implementation |
|---|---|
| Context Storage | 3-tier: Hot-Cache (Redis) → Semantic Ledger (PostgreSQL + pgvector) → Cold Archive |
| Retrieval Mechanism | Asynchronous pgvector HNSW index with per-tenant RLS enforcement |
| Prompt Assembly | Deterministic multi-tiered injection: System Prompt floor → retrieved fragments → sliding window |
| Tenant Isolation | PostgreSQL kernel-level Row-Level Security (RLS) — enforced at DB, not application layer |
| Concurrency Model | Stateless kernel; horizontal scaling via shared PostgreSQL + Redis cluster |
| Streaming Protocol | Server-Sent Events (SSE) passthrough — zero additional buffering |
| Observability Stack | OpenTelemetry traces, Prometheus metrics |
| Supported Architectures | AMD64, ARM64 |