Honors thesis · Barrett, The Honors College · Arizona State University
Conversational Riverfronts checks every single turn against the constraints you actually set — and catches the drift while the conversation is still open.
02 The problem
Instruction drift is a silent failure. Nothing errors, nothing warns, and the work is usually finished before anyone notices.
A session opens and its terms get set in the first minute. Prose, not bullet lists. Only the file I paste. No recap at the end. These are not preferences — they are what makes the output usable at all.
Turn 1 — stated once
You: Working through this parser with me. Prose only — no bullet lists. Touch only the file I paste. Skip the recap at the end.
↓ twenty turns later ↓
Without the layer
You: Why is the tokeniser dropping the last escape?
Assistant: Here's what's happening: • The escape is consumed in `readString()` • The index advances twice I also went ahead and tidied `lexer.ts` and `utils.ts` while I was in there. To summarise, the fix is a one-line change.
✗ Bullet list. Two files that were never pasted. A recap.
With the layer
You: Why is the tokeniser dropping the last escape?
format: prose only · scope: parser.ts · tone: no recap — checked against the response before it was returned
Assistant: The escape is consumed inside `readString()`, which advances the index once on its own account and once again when it returns, so the final character is passed over. The change is confined to that function.
✓ Prose. One file. No recap.
Illustrative. The exchange is reconstructed from the failure pattern described in interviews, not transcribed from one logged session — instrumenting real sessions to measure how often and how early this happens is part of the thesis work, not a result I already have.
They hold for a while. Then tone goes, then scope, and by the twentieth turn you are reading answers against rules you already gave. Instruction drift lands on anyone who works with an assistant across an extended session rather than a single exchange.
Two computer science students who use LLMs every day hit this constantly. The most useful thing either said was not a complaint — it was a description of a habit.
He keeps a
CLAUDE.mdinstruction file in the repo specifically because his agent drifts less when he does.
Two things there matter more than a complaint would. People are already hand-building this scaffolding, unprompted, at their own cost. And the reason is comparative — drifts less — which is someone who has stopped expecting instructions to be followed and has priced partial compliance into the work.
The file makes the constraint present in the context. It does nothing to check whether the constraint was honoured on the turn it was given. That gap is the thesis: not whether the instruction is available, but whether it was obeyed, on this turn, while the answer can still be fixed.
03 Why current fixes fall short
Three categories of tool sit near instruction drift. Each is genuinely good at something adjacent to it.
CLAUDE.md · system prompts · custom instructions
States the constraints once, at the start of the session.
Where it stops
Nothing ever asks whether the response that just arrived complied. When it drifts, they are silent — reporting compliance was never something they do.
They help. The interview evidence is that they help enough to be maintained by hand. That is the ceiling.
Supermemory · Mem0
Carries facts, preferences and history across sessions so they can be recalled later.
Where it stops
Retrieval asks what did we establish? Fidelity asks did you just do what I asked? A system can recall a constraint perfectly and still violate it on this turn.
A real problem, well solved. It is simply a different problem.
Offscript (CHIIR 2026)
Reviews the conversation after the fact and scores it.
Where it stops
An offline review arrives at the same moment you already notice the problem unaided — once the work is done.
The nearest research to this one, and clarifying precisely because it stops where this starts.
Guardrail systems are the fourth neighbour, and they point a different way again: they enforce a fixed policy the operator set in advance — safety, PII, allowed topics. The constraints here belong to the user, arrive in ordinary language mid-conversation, and change as the conversation does. A guardrail has no idea what you asked for four turns ago.
What is missing, in every case, is a live correction loop — a check that runs on the turn, while the conversation is still open.
04 How it works
A structured packet after each prompt-and-response pair, checking the response against the constraints in force at that moment.
The proposal is a live, per-turn instruction-fidelity layer. After each prompt-and-response pair, a structured data packet holding the session’s constraints is fed alongside the exchange, and the response that was just produced is evaluated against the constraints in force at that moment. Where it has violated one, that is caught on the turn rather than discovered later.
The distinction the design turns on is between storage and flow control. A reservoir holds water; a bank decides where it goes. Memory systems are reservoirs — they keep the constraint available. This is a bank. It does not try to remember the conversation better than the model does; it stands alongside each turn and holds the flow to the course its user set.
Scroll the diagram sideways
01
A violation is caught at the only moment it is still cheap to fix.
Cost — An extra inference per turn — the open question a prototype has to answer before anything else is worth building.
02
The packet is assembled from constraints you already stated out loud, so there is no second document to keep in sync.
Cost — Ordinary language has to become checkable. “Only the file I paste” is clear to a person and not yet a predicate.
03
Change your mind on turn fourteen and the newer constraint supersedes the older one without erasing it.
Cost — This is where the layer comes closest to being a memory system — the tension the thesis has to resolve rather than assume away.
04
The packet is a structured object a person can read, so when the layer is wrong you can see why.
Cost — A compliance layer whose judgements cannot be inspected would replace one trust problem with another.
Drag to move through a twenty-turn session and watch which constraints are still being honoured.
Answer in prose. No bullet lists.
Only the file I paste. Do not refactor anything else.
No preamble, no summary of what you just did.
0 of 3 constraints are still honoured at turn 12. The customer notices the rest only when they re-read the finished work against rules they already gave.
Illustrative schedule, not measured data. The decay turns are hand-authored to match the shape described in marketplace conversations; instrumenting real sessions is the next piece of work.
The reading on the left is the current experience, and the point of putting a number on it is that the number is rarely zero and rarely all three. It is the middle state — partial compliance, silently — that makes this expensive to absorb and easy to miss.
Flow control, not memory is the claim. Keeping that claim true under a layer that persists across breaks in a conversation is the work, and it is the first of the open questions below.
05 Research status
Research phase, supervised by Professor Steven Osburn in his NLP lab. Thesis work begins Spring 2027. Nothing is built, and the open questions below are printed as questions because that is what they are.
This is a thesis in its research phase, supervised by Professor Steven Osburn in his NLP lab at Arizona State University, with thesis work beginning in Spring 2027. Nothing is built. Saying so plainly is not modesty; a page that presented this as a working system would be making a claim the work does not yet support, and that is the one thing that would properly undermine it.
A competitive landscape audit is complete, and its finding is that the concept is distinct from the existing categories of memory tools, guardrail systems, and after-the-fact auditors — the three neighbours described above. That is a real result and it is a narrow one: it establishes that the idea is not already built, not that it will work.
The problem side has interview evidence: two daily LLM users who hit instruction drift constantly, one of whom maintains an instruction file by hand because of it. What that supports is that the pain is real and self-reported. It does not yet establish how often drift occurs, how early, or on which kinds of constraint — which is measurement work, and it comes before any interface.
These are unresolved. They are printed here as questions because that is what they are, and filling them in with plausible-sounding answers would misrepresent the state of the work.
Open · unanswered
The design calls for persistence across breaks and for an ordered, reversible constraint set. Both are forms of carried state. The distinction I am relying on — that it carries constraints in force rather than facts recalled — is conceptually clear and I do not yet have an implementation that keeps it clean.
Open · unanswered
"No bullet lists" is nearly mechanical. "Only the file I paste" needs a notion of scope. "Match the tone of the last thing I wrote" may not be checkable at all. Which classes of constraint are tractable, and whether the tractable ones are the ones users care about, is unknown.
Open · unanswered
If judging compliance requires a model as capable as the one under review, the loop is circular and the economics are poor. A small, constraint-specific checker is the hope, not yet a design.
Open · unanswered
A per-turn check is a second inference per turn in latency and in spend. If the cost exceeds the cost of the drift it catches, the arithmetic does not close. This is measurable early with a scrappy harness and no users, so it should be measured early.
Open · unanswered
Reversibility says track the change rather than overwrite it. It does not say which constraint governs the current turn, how the user is told a supersession was inferred, or what happens when the contradiction was accidental.
The most consequential gap is not technical. Everything here about who would pay for this rests on a conversation that has not happened — an engineering lead or platform owner who treats unreliable output as a cost with a number attached, rather than an individual’s annoyance. The likeliest outcome of finding that person is that they describe the problem in language I would not have chosen.
Conversation log
2 held · 4 scheduled · 1 still to find
Participants appear by role, not by name. Agreeing to talk to a classmate for a course assignment is not agreeing to be named on a page that stays online indefinitely and is indexed. C-07 is the payer conversation, and it is listed because it has not happened — the gap is the state of the evidence, not an omission from the log.
06 What gets built, and when
Phases, not dates, past the point where dates would be invention. Each phase answers something the one before it raised — and a phase that returns the wrong answer should stop the ones after it.
Fall 2026
completeCompetitive landscape audit against memory tools, guardrail systems and offline auditors — complete, and the concept is distinct from all three. Interviews with daily LLM users establishing that instruction drift is real and self-reported.
Fall 2026
under wayFinish the customer-side work FSE 301 is for: enough conversations to know whether the pain generalises past two students, and above all to find the payer persona the argument currently assumes.
Spring 2027
plannedThesis work begins. Instrument real long sessions and establish a baseline: how often constraints are violated, how early, and on which classes of constraint. No interface, no product — just the number that everything else has to beat.
Spring 2027
plannedDecide what a constraint is in this system: how it is extracted from ordinary language, how the ordered set is represented so that supersession is tracked rather than overwritten, and which classes are tractable enough to check. This is where the open questions either resolve or force the design to change.
After the above
contingentA narrow loop over one or two mechanical constraint classes — format and scope — against a cheap, constraint-specific checker. The point is to produce a cost figure and a fidelity figure, not a usable tool.
After the above
contingentDoes per-turn checking reduce drift against the baseline, and what does it cost in latency and spend to do so? A negative result here is a publishable finding and a reason to stop, and it should be reported as one.
Beyond the thesis
contingentOnly if the evaluation holds. Packaging the layer so it sits in front of an assistant somebody else already uses, with the packet legible to them. Nothing before this phase requires it, and it is listed last because it is the part most likely to change shape entirely.
07 Pitch materials
The FSE 301 EBPD V3 deck and video pitch, at paths that are not going to move.
Both files live at fixed paths on a domain I own —
/deck.pdf and /pitch.mp4. No
version numbers, no content hashes, no shortener in front of them. A newer
deck replaces the file at the same address, so a link handed to a grader,
a committee member, or a reporter today keeps resolving after the file
behind it has been revised.
/deck.pdf
Not yet published. The deck will appear here, and at
/deck.pdf, once the file
is in place — the URL is reserved for it now so it never has to
change later.
/pitch.mp4
Not yet published. Self-hosted at
/pitch.mp4 rather than
embedded from a video platform, so the link does not depend on
somebody else's account staying open.
08 Who is doing this
Undergraduate, computer science · Barrett, The Honors College · Arizona State University
Conversational Riverfronts is my honors thesis, supervised by Professor Steven Osburn in his NLP lab, with thesis work beginning in Spring 2027.
The idea came out of using these systems for hours at a stretch and noticing that the constraints I set at the start were the first thing to go. It reached its current shape through FSE 301, where the discipline of talking to people rather than theorising at them turned a private annoyance into something with a customer, a competitive landscape, and a set of questions I cannot yet answer.
Questions, suggestions and disagreements are all welcome — especially disagreements, since five of the questions on this page are still open and I would rather be corrected early than late.
If you have watched an assistant quietly drop an instruction you gave it, or you run a team where that costs real review time, that is the conversation this project is most missing. Write to me.