Research preprint · v0.3.0 · June 2026
ACME externalizes memory, belief, and learning from LLM weights. Experiences become auditable beliefs; feedback updates confidence; retrieval combines graph structure with vectors.
Deployed benchmark runs on Azure OpenAI GPT-4.1 with sandbox-isolated scenarios,
official LongMemEval judge prompts, and persisted benchmark_runs for reproduction.
MemoryBench overall is a four-metric capability index; baseline feedback/belief cells are N/A but count as 0.
| MemoryBench v3.1 | Overall | Retention | Feedback | Belief |
|---|---|---|---|---|
| ACME | 0.925 | 1.000 | 1.000 | 0.700 |
| RAG | 0.470 | 0.900 | — | — |
| MemGPT | 0.477 | 0.979 | — | — |
| LangGraph | 0.489 | 0.979 | — | — |
14 scenarios (incl. knowledge-update) · job 94005737 · 26 Jun 2026 · paper Table 8: job 3b31e5e3
| LongMemEval oracle | Overall | KU | Temporal | Abstain. |
|---|---|---|---|---|
| ACME | 0.876 | 0.944 | 0.803 | 0.833 |
| RAG | 0.776 | 0.875 | 0.622 | 0.600 |
| MemGPT | 0.786 | 0.861 | 0.630 | 0.600 |
500 questions · job 45623ca0 · GPT-4.1 · v5 hybrid · +10.0 pts vs RAG
Seven engines coordinated by a single orchestrator. PostgreSQL stores episodes and beliefs; Neo4j holds the semantic graph; Azure OpenAI handles extraction and reasoning.
Observation through belief promotion, with symmetric demotion when contradictory evidence arrives.
Beliefs ranked by prediction success, temporal stability, contradiction resistance, and source diversity.
Extract knowledge from a PDF — upload one paper and let GPT-5.4 build a document-local knowledge graph. Memory Chat — per-visitor agents with web browse and file upload.
Hosted on Azure Container Apps. Benchmarks require API key; health endpoint is public.
https://acme-api.blackgrass-3076f328.westeurope.azurecontainerapps.io
Health: /api/v1/health
git clone → azure/set-api-key.sh
./scripts/run_prod_benchmark.sh
LONGMEMEVAL_TYPES=all ./scripts/run_longmemeval_prod.sh