Who Stores What? Why Split Memory Custody Is One of the Sneakiest Governance Problems in Agentic AI

Enterprise LLM agents have officially left the demo era. They're now running real, long-horizon workflows — remembering customer preferences, correcting errors on the fly, enforcing policies, and getting smarter with every session.

But here's the part nobody puts on the pitch deck: the experience these agents accumulate lives in three different places, controlled by two different parties who don't fully see what the other is doing.

This split custody creates a quiet but expensive phenomenon I call the memory fork. Prompts say one thing. Weights do another. Suddenly your agent surprises everyone at the worst possible moment. Teams duplicate effort, debug in the dark, and end up in awkward 3 a.m. conversations about who actually owns the risk.

In a new formal game-theoretic model, we ask a simple but powerful question: When is this split custody arrangement stable and productive — and when does it quietly collapse into a low-trust trap?

The answer is more interesting (and more fixable) than you might expect.

The Three Tiers Nobody Talks About Enough

Modern LLM agents rely on three complementary memory layers:

• Prompt rules and playbooks — fast, fully inspectable, controlled by application teams.
• External retrieval stores (RAG caches, episodic databases, checkpointers) — medium inspectability, usually managed by the deployer or platform.
• Model weights (fine-tunes, adapters, RLHF updates) — slow, mostly opaque, controlled by the vendor.

Technically elegant. Organizationally messy.

In practice, custody is split. Application owners govern the inspectable tiers. Vendors control the weights and often observe aggregate usage traces. Neither side has full real-time visibility into the other's memory writes. The result? Prompt-visible rules drift out of sync with weight-encoded behavior. State gets duplicated across layers. Attribution becomes ambiguous. Risk becomes conveniently (or inconveniently) blurred.

You see this pattern play out again and again in GitHub issues and Stack Overflow threads across LangChain, LangGraph, CrewAI, Mem0, and similar frameworks — checkpoint corruption, manual pruning workarounds, rollback headaches, and "who changed what?" debugging sessions.

The Strategic Game Behind the Fork

Technical feasibility gets us part of the way. What we really need is incentive compatibility under split custody.

We model this as a sequential game between two rational players: the application owner (who bears task performance and liability risk) and the model vendor (who hosts the base model and can fine-tune on usage traces).

The owner chooses the memory architecture: prompt-only, hybrid, or heavy vendor delegation. The vendor then chooses fine-tuning intensity. Payoffs reflect task success, reconciliation costs, revenue, and liability — all under partial observability.

We solve it with subgame-perfect equilibrium (full information) and Perfect Bayesian Equilibrium (when fine-tuning on leaked traces is unobserved).

The Big Findings — With Real-World Grounding

1. Hybrid memory can win — but only under the right conditions.

Even when prompt-only looks cheaper in the short run, hybrid architectures can emerge as a stable equilibrium when deployers bear reactive governance costs and can moderate what the vendor learns (Proposition 1b). There's a clean threshold: hybrid makes sense when governance cost stays below a surplus-to-labor ratio.

In practice: Teams often end up doing manual pruning, guardrail additions, or checkpoint migrations after drift shows up (common in LangGraph and CrewAI issues). When these reactive costs are manageable, hybrid custody pays off — you get vendor improvements without fully surrendering control. When governance labor is cheap enough to moderate aggressive fine-tuning, you get the best of both worlds: improvement without uncontrolled divergence.

2. Observability and repair rights are superpower levers.

Contractible rights to inspect, roll back, or reconfigure inspectable tiers act as credible off-path threats. When owner liability exceeds vendor liability, these rights discipline aggressive fine-tuning and push the system toward better outcomes (Proposition 2). They also select the high-trust equilibrium when hidden fine-tuning would otherwise sustain a pessimistic trap (Proposition 3′).

In practice: Think of episodes where a poisoned checkpoint or stale entity link forces a full process restart because no one had surgical access to the tier. Contractible observability (logging, trace spans, guarded checkpointers) turns these into manageable incidents instead of production fires. Teams that negotiate these rights upfront avoid the "who owns this memory?" blame game later.

3. The low-trust trap — and why it's so sneaky (the central result).

Under undisclosed fine-tuning on leaked traces, the game admits two Perfect Bayesian Equilibria — a high-trust hybrid equilibrium and a low-trust prompt-only trap (Proposition 3).

The trap isn't driven by bad parameters. It's sustained by self-confirming owner pessimism. Expecting aggressive hidden behavior (e.g., the vendor quietly improving weights on traces the team can't see), the owner sticks with prompt-only to avoid hybrid risk. This removes the observability that would have disciplined the vendor — so the vendor does exploit leaked traces, confirming the owner's pessimism.

In practice: Imagine a team that has been burned by unexplained policy drift before. They default to prompt-only to maintain control, even though hybrid would capture useful vendor improvements. The vendor, seeing less structured feedback, leans harder on whatever traces leak through. Drift worsens. The team's pessimism is validated, and they double down on prompt-only. Everyone loses a bit of capability. Observability rights break this loop by making hidden behavior visible and correctable — turning a belief-driven trap into a manageable, high-trust equilibrium.

The hopeful part? Contractible observability doesn't just threaten bad behavior — it selects the high-trust equilibrium by making pessimistic beliefs irrelevant. You no longer have to trust what you can audit.

What Teams and Vendors Should Actually Do

Memory governance isn't a routing detail. It's a strategic design problem at the intersection of architecture, contracting, and incentives.

For application teams and architecture committees:

• Assign clear owners per memory tier with explicit reconciliation cadences.
• Prioritize contractible observability and repair rights in your vendor agreements — these are high-leverage clauses.
• Lean into hybrid architectures with automation when governance costs sit in the right range. The "improvement without loss of control" regime is achievable when moderation binds; when it doesn't, observability matters even more.

For vendors:

• Increase transparency on evaluation deltas when traces inform updates. Trust compounds faster than hidden improvements.
• Offer inspectable adapter layers (customer-visible LoRA checkpoints and similar) to make hybrid custody both technically feasible and attractive.

The bigger picture is genuinely exciting. Split memory custody doesn't have to be a permanent source of friction. With thoughtful design and contracting, organizations can move from fragile equilibria to durable, high-trust partnerships that unlock the full potential of agentic AI.

Final Thought

Agent memory under split custody is a new but representative governance challenge in information systems. The model shows the low-trust trap is not inevitable. The teams and vendors that get observability, repair rights, and tiered permissions right won't just ship more reliable agents — they'll build the kind of resilient, accountable systems that define the next era of enterprise AI.

Formal theory with illustrative traces. Full working paper (proofs + construct-check appendix) available on request.