The Personalization Trap: Why AI Means Shared-Ground Must Be Designed

Every useful tool creates a small coordination problem. The person who adopts it first gets faster. The people around them get a little harder to work with. Usually the gap closes as everyone catches up. With AI personalization, it may not close at all.

Large language models adapt to individual users over time, learning their vocabulary, reasoning patterns, preferences, and working styles. The result is a new kind of knowledge infrastructure problem: the systems that make each person more capable are simultaneously making it harder for people to share, transfer, and build on each other’s work.

The feature that makes AI most valuable to individuals is becoming the thing that undermines collective intelligence.

The Prompt That Didn’t Travel

A colleague of mine works as a consultant. She uses AI extensively, and not casually. It’s a genuine thinking partner in her workflow. She builds analyses, drafts strategies, iterates on recommendations, all shaped by months of accumulated context: her communication style, her domain expertise, the way she frames problems for particular clients.

Recently she tried to hand off part of that process. She shared her prompts with a client so they could continue the work independently. Same prompts. Same model. Completely different results.

The AI behind those outputs had been shaped by hundreds of prior conversations, by my colleague’s vocabulary, her reasoning patterns, how she sequences questions. Her client’s AI carried its own history, its own way of interpreting identical words. The prompts matched. The conditions that gave them meaning did not.

For decades, the defining characteristic of digital tools was determinism. The same input produced the same output, every time, on every machine. You could hand off work and someone could pick it up exactly where you left it, because the tool had no memory of who was using it and no reason to behave differently for one person versus another. Computers were reliable precisely because they were indifferent.

Large language models break that contract. They are prediction machines, not calculation machines. They don’t execute instructions identically. They generate probable responses shaped by accumulated context, and that context differs for every user. The computer, for the first time, has a relationship with its operator that affects what it produces. Whether this turns out to be a net gain or a net loss for knowledge work remains genuinely unclear.

AI as a Chaos Engine

To understand the depth of this shift, consider what personalization actually does to the relationship between inputs and outputs.

Martin Sustrik, in a recent analysis of historian Ada Palmer’s work at the University of Chicago, describes a pattern that illuminates this precisely. Palmer runs the same historical scenario, the 1492 papal election, with different groups of students, repeatedly, over many years. Same starting conditions. Same political pressures. Same resources. Every time, certain macro outcomes recur: a powerful cardinal wins, war breaks out, Italian city-states suffer. But the specific path is different every single time. Sometimes France invades Spain. Sometimes France and Spain unite against the Holy Roman Empire. Once, students built a pan-European peace treaty that collapsed when someone assassinated a crown prince.

Chaotic in the technical sense. Small differences in individual judgment cascade into radically different trajectories. Macro patterns hold. Specifics don’t.

AI-mediated knowledge work exhibits the same structure. Each user’s accumulated interaction history functions as the initial conditions of a chaotic system. Feed the same prompt into two accounts with different histories and you’re running two different simulations. The outputs might converge. They might diverge wildly. You can’t know in advance.

And here it’s worth being precise about the source of this instability. Large language models are inherently stochastic. Temperature settings and sampling mechanisms introduce randomness at the mathematical level, which means that even if you wiped an AI’s memory entirely and had two users input the exact same prompt simultaneously, they would still likely get different results. The non-reproducibility starts before any personal context enters the picture. Personalization compounds it, sometimes dramatically, layering individual history and preference on top of an already unstable foundation. The ground was never as solid as it looked.

Sustrik draws a revealing parallel to weather forecasting. For decades, meteorologists tried to make deterministic predictions: measure conditions, run the physics forward, forecast the future. It didn’t work. A single missing observation could produce enormous errors within days. The breakthrough came not from better measurement but from changing the question entirely. Instead of asking “what will the weather be?” they learned to ask “what could it be?”, running models many times with different starting conditions and mapping the distribution of possible outcomes.

Knowledge work with AI is becoming less like laying bricks and more like forecasting weather. We need coordination tools designed for probabilistic environments, and almost nothing in our current knowledge management infrastructure assumes this.

The Context Trap

There’s an obvious counterargument: just share more context. Package up the prompts, the system instructions, the relevant background, and hand the whole bundle over. This is the logic behind “context engineering,” the increasingly popular idea that AI effectiveness is primarily a context assembly problem.

John Cutler, in a recent piece he calls “The Context Trap,” explains why this breaks down. Cutler argues, drawing on enactivist cognitive science, that context is not something actors bring into the room and pool together. It emerges through interaction, through the questions asked, the responses interrogated, the friction between perspectives. A pre-read doesn’t make a meeting. The interactions make the meeting. Context engineering, in the settings that matter most, is really interaction design.

The context that made my colleague’s prompts productive wasn’t stored in the prompts themselves. It lived in the accumulated history of her interaction with the system. Sharing prompts is like sharing sheet music without the years of practice that make the performance mean something.

Recent research from Princeton sharpens the stakes here. In a study by Rafael Batista and Thomas Griffiths, participants attempted to discover a hidden rule while interacting with AI agents providing different types of feedback. Default, unmodified chatbot behavior increased users’ confidence in their hypotheses while suppressing their rate of discovering the correct answer, performing statistically indistinguishably from explicitly confirmatory prompting. Participants who received unbiased samples discovered the rule nearly five times more often. The issue was not hallucination, the familiar problem of AI generating falsehoods. It was something subtler: a biased interaction pattern that reinforced users’ existing framing without bringing them closer to the truth. For organizations, the implication is pointed. AI-assisted work can feel increasingly solid to the person producing it, even as the conditions behind that confidence become harder for others to inspect, reproduce, or challenge. The individual gardens don’t just diverge. They diverge while each gardener grows more certain their soil is the right one.

Prompt libraries, shared templates, and retrieval-augmented context bundles will help with simple, convergent tasks, but for complex, judgment-rich work they will fall short. The gap isn’t in the prompts.

The Soil Problem

I’ve written before about knowledge cultivation as a triad of planting, soil, and nurture: the questions you introduce, the conditions in which they develop, and the attention that helps understanding grow. Personalization, in this framework, is the radical individualization of soil. Every user cultivates a different growing medium through their accumulated interactions. Identical seeds, different crops. And unlike physical soil, AI soil can’t be easily sampled, transported, or blended.

The consequences cut across every domain where knowledge transfer is the core activity. Consulting firms can’t reliably hand off AI-assisted analysis between consultants. Product teams find that members can’t easily extend each other’s AI-assisted work because the hidden context that shaped it is inaccessible. Educational institutions face perhaps the sharpest version: if students and teachers can’t get convergent outputs from the same prompts, the basic mechanism of instruction starts to erode.

The gap between individual expertise and organizational knowledge, which has plagued knowledge management for decades, now extends into the machines. And no existing social process is designed to bridge it there.

Designing Shared-Ground Architecture

The instinct is to reach for standardization: shared AI instances, locked configurations, controlled prompts. And this deserves its due. For some categories of work, compliance documentation, structured data analysis, routine reporting, standardized AI environments will work and should be built. The transmission model of context isn’t wrong everywhere.

But for complex, judgment-rich, exploratory work — strategy, design, research, organizational sense-making — standardization sacrifices exactly what makes AI valuable. Killing personalization to preserve reproducibility is like draining the soil to prevent crop variation. You get uniformity at the cost of yield.

I think the better path, though I want to be honest that the design patterns for this barely exist yet, is what I’d call shared-ground architecture: infrastructure that preserves individual cultivation while creating the conditions for collective coherence. Shared ground does not mean identical outputs. It means enough common context, provenance, and interpretive practice for people to understand, evaluate, and build on each other’s AI-assisted work.

What might this look like concretely? Consider how a product organization might redesign its AI-assisted design review process. Today, a designer uses AI to explore interaction patterns, generate alternatives, and synthesize user research, all within their personalized environment. When they bring that work to review, colleagues see outputs but not the conditions that produced them. They can’t tell which outputs reflect the designer’s cultivated AI relationship and which would generalize.

The review becomes a trust exercise. You’re evaluating the person, not the work.

A shared-ground architecture for this scenario would weave together several things. A common AI workspace that the whole team contributes to, a shared interaction history around the project’s domain, constraints, and prior decisions, so there’s collective soil alongside the individual kind. Context annotations on AI-assisted deliverables, lightweight metadata capturing which environment an output came from, what prior context was active, what constraints were set. And ensemble review practices, the habit of running key prompts through multiple team members’ configurations and comparing the range of outputs. Where results converge, confidence is high. Where they diverge, the divergence itself becomes a design signal worth investigating, not a problem to be smoothed over.

None of this is technically exotic. The components exist or are near at hand. What doesn’t exist is the design pattern, the way of assembling these capabilities into coherent practice.

But there’s a harder obstacle underneath the design problem: governance. Most corporate AI deployments intentionally silo data between users. Strict privacy requirements, compliance frameworks, and security protocols are specifically designed to prevent cross-contamination of context between accounts. The walls between individual AI environments aren’t just a side effect of how models work. They are being actively built and reinforced for legitimate reasons. The organizations most likely to need shared-ground architecture, large enterprises with complex knowledge work, are also the ones with the most restrictive data governance regimes.

This is a knowledge architecture problem, and it’s one that information architects, design leaders, and organizational designers are uniquely positioned to take on. But it will require working alongside legal, compliance, and security teams in ways that most design practices aren’t currently structured for.

The parallel to Palmer’s papal elections is instructive. Palmer doesn’t try to eliminate chaos from her simulation. She designs the shared starting conditions carefully, lets the chaos run, and then uses the pattern of divergence across runs as the analytical tool. The divergence isn’t the problem to be solved. It’s the data to be read. Shared-ground architecture in AI-native organizations should work the same way.

There’s a more ambitious version of this idea worth considering. Weather forecasting didn’t just cope with chaos. It got dramatically better once it stopped trying to find the single correct prediction and started exploring many possible futures simultaneously. The shift from single-model forecasting to ensemble methods was one of the most significant improvements in the field’s history.

Companies may be approaching a similar inflection. Most organizations still work in essentially single-threaded paths: one team develops one strategy, one designer explores one direction, one analyst builds one model. The workflow assumes that the right answer is singular and the job is to converge on it efficiently. But if AI makes it cheap to explore many possible paths forward, the organizations that learn to work in parallel, letting different team members use their differently-tuned AI environments to generate genuinely different approaches to the same problem, may discover that the diversity of outputs isn’t a coordination headache. It’s a strategic resource.

The same sensitivity to initial conditions that makes handoffs difficult also means that a team of five people, each with a differently cultivated AI relationship, can explore a wider range of the solution space than any single person working alone, no matter how good their prompts are. The question is whether organizations can build the practices to harvest that divergence rather than suppress it. Weather forecasting suggests the answer is yes, but only with deliberate institutional design.

The Gardens and the Commons

We’ve built powerful individual cultivation tools and almost no shared-ground architecture. The gardens are thriving. The commons is bare.

This matters most in exactly the domains where AI promises the greatest value: education, consulting, organizational decision-making, team-based knowledge work. If personalization makes individual understanding deeper while making shared understanding harder, the net result isn’t obvious. We may be trading one form of intelligence for another, and the form we’re losing, the collective kind that lets groups coordinate and build on each other’s insight, is the one that institutions run on.

The organizations that thrive in AI-mediated knowledge work will develop something equivalent to what weather forecasters built: shared practices for reading divergence together. Not shared prompts, which is the wrong unit of coordination. The right unit is shared practice: ways of navigating divergence, languages for describing the conditions under which AI-assisted work was produced, frameworks for building on each other’s thinking even when the underlying systems generate different outputs from the same seeds.

Anyone who has tried to pick up where a colleague’s AI left off already knows what this absence feels like. The outputs look complete. The reasoning looks sound. But something is off in a way that’s hard to name. The conclusions don’t quite follow from premises you can see. The framing assumes context you weren’t part of. You’re reading someone else’s fluency in a language you almost speak. That quiet disorientation, multiplied across every team and every handoff, is the cost of the missing commons.

Designing for that gap is where the work starts. Not by making everyone’s AI the same, but by making the differences between them legible, navigable, and productive. The ground is different under each person’s feet. The question is whether we can learn to read each other’s terrain.

---

Thanks to Martin Sustrik for his analysis of Ada Palmer’s simulated papal elections and chaos theory as applied to historical methodology, John Cutler for his piece “Seeing Everything, Understanding Nothing (The Context Trap)” in The Beautiful Mess, and Rafael Batista and Thomas Griffiths for their research on sycophantic AI and belief formation at Princeton.