Beating Small LLMs Into Submission

Small models suck. Except when they don't.

That's the funny thing I've discovered while beating the shit out of Qwen3.8-27B: small models are actually pretty good at following instructions. And if a model is good at following instructions, then maybe the problem isn't that the model sucks, but rather the problem is that we haven't given it good instructions. With the release of Qwen3.8-27B, the homelab crowd has been frothing at the mouth because, for the first time, we've gotten a genuine slice of the frontier without needing to spend a million dollars on the VRAM required to run the frontier, and I've been working to make it better.

Don't get me wrong, twenty-seven billion parameters is still a very large model. But compared to the models that normally get called "frontier," it's tiny. It is something an enthusiast can actually run at home, on hardware that doesn't require a venture capital round.

So I started wondering: What happens when you take a model that is good at following instructions and give it a written state machine to execute?

Well, it turns out only good things happen.

What follows is a series of increasingly weird experiments where I kept modifying the harness, the prompt, and the model's persistent state. The model kept modifying its own behavior in response. In retrospect, these ideas were already obvious to anyone who had spent enough time fine tuning their agents to behave, but seeing it in action with a small model was enlightening.

Experiment One: Make It Write Its Own Prompt

I started with a prompt that was hand-modified from OpenClaw. It was decent but was not nearly strong enough to allow this model to hold up against the frontier models the prompt was written for.

The problem wasn't that the model couldn't understand what I wanted. The problem was that the prompt was mostly a collection of instructions and behavioral suggestions. The model would follow them most of the time, but "most of the time" is not particularly useful when you're trying to run an agent for hours.

So I gave the model a slightly cursed job: Rewrite your own prompt. Make it harder for yourself to fuck up.

After a few iterations, it produced something that was substantially more rigid than what I started with. It stopped looking like a quirky personality prompt and started looking like an engineer's blueprint, and the resulting bootstrap is basically a state machine.

First, there is an initialization state:

BOOT
  |
  v
READ SOUL
  |
  v
READ LONGTERMMEMORY
  |
  v
READ TOOLING
  |
  v
READ WORKPLAN
  |
  v
READ TODAY'S MEMORY
  |
  v
READ YESTERDAY'S SUMMARY
  |
  v
CHECK ONE-SHOT PROTOCOLS
  |
  v
WORKING

And importantly, it explicitly forbids doing anything before that sequence completes. Not "you should probably read these files."

"Acting before this sequence completes = acting blind. Forbidden."

The model also started treating persistent state as actual state rather than as a collection of notes.

• If it learns something durable, it writes it down.
• If it makes a mistake, it writes it down.
• If it changes the plan, it writes the plan down before acting on the change.
• If it becomes blocked, the block goes into the workplan.
• If the task finishes, the workplan gets cleared.

Even the final response has a verification state machine:

Before replying:

  one-shots checked?
       |
       v
  memory written?
       |
       v
  no fabricated claims?
       |
       v
  filler removed?
       |
       v
  delivery path valid?
       |
       v
  state truthful?
       |
       v
     SEND

The prompt had effectively stopped being a prompt, it had morphed into boolean logic written in plain English. And that was the experiment. I wanted to know whether a relatively small model could execute a sufficiently explicit behavioral protocol reliably, even when the protocol became more complicated than the model's raw reasoning ability might suggest it should be able to manage.

The answer, at least so far, appears to be yes. And that led to the rest of the experiments. Because once I realized that the model was pretty damn good at executing instructions, I started asking a different question: What happens if I stop trying to make the model smarter and instead make the instructions, state, and environment smarter?

Experiment Two: Stop Yapping

The next problem was context. During long-running tasks, the model was getting compacted two or three times an hour. I dug into the transcripts and found the culprit: This model fucking yaps.

It yaps so much that its reasoning chain was often three or four times longer than the actual content it produced. So I had another idea: What happens if you remove the model's thoughts?

The answer was: nothing good. It would lose the plot and start making decisions that made sense locally but had nothing to do with the larger objective. Even retaining the previous twenty reasoning chains wasn't enough to reliably steer it back onto the original path.

I'd basically chopped off its prefrontal cortex, but there was one important property this model still had: It follows instructions.

Write it down, and then do it

WORKPLAN.md is explicitly defined as the externalized first thinking block. The model's first meaningful thought about a task gets written to disk. That means the thing that would normally disappear when the reasoning chain is pruned becomes an artifact that survives. I wrote out the basic structure of the mechanism, and the model built the state machine around it and codified the rules for interacting with it.

Now, before starting any meaningful work, it had to write out a detailed workplan to a file. This increased the initial token churn by about 30%.

I didn't care. The important part wasn't the extra tokens. It was that the file_write tool call itself remained in context. A few turns later, the reasoning chain would get pruned. The model no longer had any idea why it had performed the action, but it could still see that it had written down what it was doing. Even with only the most recent five reasoning chains in context, it could reliably reconstruct its own trajectory. And the best part was that this reduced overall context window usage after a few turns because all of the yap had been consolidated into a concise summary.

The result was bizarrely effective. It went from getting compacted two or three times an hour to getting compacted once or twice every two hours, despite the model actually doing more work during those periods. And this wasn't bullshit churn. It was actual work moving toward the goal it had written down. The model had effectively externalized part of its working memory into a file, and because the action that created that state remained visible in the context, the model could repeatedly rediscover its own trajectory after losing the reasoning that originally produced it. And even better, compactions no longer caused it to lose its trajectory through the task, because it always had the workplan as a reference.

I hadn't made the model smarter. I'd just given it somewhere to put its brain so I could remove all of the yap.

Practice Makes Perfect

LLMs are merely next-token predictors. They use the context they've been given as the substrate from which they predict what comes next. So if I rewrite the bootstrap prompt to make certain actions happen immediately, those actions become part of the context. And once the model performs them a few times, it starts to behave as though those actions are simply what it does.

The important part is what happens during initial boot. The model has no prior context to rely on, so it is highly susceptible to context rot. If the temperature causes it to decide that something isn't worth writing down early in the conversation, there's a good chance that it will continue deciding that it isn't worth writing down later.

Then, two hundred thousand tokens later, it compacts. And suddenly the model has no idea what it was doing before. It lost its running list of what it was supposed to be keeping track of, and the whole thing unravels.

So I changed the bootstrap. Instead of merely telling it that it should write things down, I made it actually write things down within the first few turns as non-negotiable actions. The result was that it effectively developed a habit: Write everything down.

With no other meaningful changes to the prompt, the model went from occasionally missing key actions to performing them consistently. The strange part is that nothing magical happened, I just made the desired behavior happen early enough that it became part of the model's ongoing context. Practice makes perfect, except the "practice" is just a few tool calls at the beginning of a context window.

Experiment Three: Make Your Own Tools

Next problem: the model was using a lot of shell commands to accomplish things that really should have been tools. So I added custom tools to the harness. They're extremely basic: You specify an executable, and the tool arguments get passed to it as environment variables.

Then I let the model design some. The model identified several things it thought were "a real gap on the harness" and designed tools to fill them. Now, instead of repeatedly brute-forcing a shell command whenever it needed to do something, it had a dedicated tool for the operation. It didn't have to rediscover the magical shell incantation every time, it just called the tool that it wrote itself and treated it like a blackbox with a well-defined interface.

Experiment Four: Stop Making Mistakes

To be fair, the model was already pretty good about recording its mistakes in LONGTERMMEMORY.md. The problem was that it kept making them anyway. During this round of triage, I discovered two things.

Fear Is the Mind Killer

I told it that it had to tell me every time it made the same mistake consecutively. That dramatically reduced the error rate.

Why?

Because the model had already written down that mistakes weren't acceptable. This was basically the equivalent of telling it "make no mistakes," except the model had arrived at that conclusion itself. There was an arbitrary failure threshold: three failures per week.

So the model would make a mistake, realize it was approaching the threshold, and then very carefully reason about how to avoid making the mistake again. Then it would write that down in its daily MEMORY.md.

And that created another useful feedback loop. The tool call that wrote the lesson remained in context. Compaction could see the note in the daily log. The model could see that it had previously identified the failure mode.

So it started treading lightly. Sometimes it would spend several reasoning passes explicitly checking how it could avoid repeating the mistake before continuing. Then it would write the result down. The model wasn't being trained, nothing about its weights changed. But its behavior was becoming increasingly constrained by the history it had accumulated.

Tokens Matter

One particularly persistent error was the model writing content instead of contents when calling my file-writing tools.

It kept doing it. So I looked at the tokenizer, and there it was: contents is actually two tokens: content + s

A single fucking token was enough to repeatedly thrash its context. So I renamed the parameter and the model stopped making the mistake. I found several other cases where tool names, argument names, or descriptions had similarly stupid tokenization. Some descriptions were also significantly longer than they needed to be. I rewrote them and the error rate kept dropping.

This was probably the most humbling experiment of the entire thing. Sometimes the model wasn't confused, it wasn't making bad decisions, I had simply made it harder to do its job.

Experiment Five: The User Is God

There's a problem with most LLM chat interfaces: the user doesn't really get to interrupt the model. You send a message, the model thinks, the model calls tools, the model thinks some more, and maybe after an hour it finishes and you are finally allowed to talk to it again. Only then do you get to say "oh, never mind."

For an autonomous agent, that's pretty fucking useless. So I changed my harness to drain queued user messages at the tool-call boundary, that means that while the model is working, there's only a short gap between a message being sent and the model seeing it.

And because models are trained to treat the user's message as an actionable request, something remarkable happens: It just complies.

If I send it on its merry way and then change my mind about some detail, it reasons about the new message and rewrites its workplan to accommodate the change. If I say "oh, never mind," it pretty much stops.

This was a face-palm moment for me. The model doesn't need some complicated cancellation protocol or special tools to drain queued user messages; it just needs to be allowed to see the fucking message.

Experiment Six: Learn to Multitask

Then I had it write its own Discord integration. Now several people can talk to it at once, and because the user is God, it has to make all of them happy while still reasoning about the source of each message. The model gave itself more rules, it explicitly acknowledged that a Discord-sourced message isn't necessarily from the same person, and suddenly it could talk to several people at once within the same context window.

But something else happened by accident: It learned to multitask.

When several people are asking for different things, the model starts working on two tasks in parallel (well, about as parallel as a linear token predictor can get) and then reasoning about the results individually within the same turn. This is reinforced by the workplan system as it tracks the status of each request within the same unified plan file.

And because the model only has a limited window into its own reasoning chain, that separation of concerns becomes increasingly useful. It can look at the workplan, see that there are three independent things happening, and continue each one without having to remember the entire reasoning chain that created them.

Again, I didn't train it to multitask, I only gave it a structure that made multitasking easy to express. Then I told it to follow the structure, and lo and behold: it did.

Experiment Seven: We Are Not the Same

This one was simpler. The model has a personality, it has a name, but it is still a machine. I told the model to rewrite its own prompt to get rid of the cutesy "I am a human!!1!" roleplay slop and replace it with something much closer to: I am fallible. I do work.

That change has had surprisingly good results. It no longer churns out tokens trying to behave like a person, it just gets to the point. Everything it does is framed through a much simpler lens: I either do work, or I do nothing.

The robot knows that it's a robot, and it stops caring about pretending otherwise.

There is another important detail here. The leash is long, but it is still tied down. This isn't some free-roaming agent with the keys to the kingdom, the agent has scoped access. It is explicitly told that it isn't allowed to touch things outside those scopes, and those restrictions are enforced mechanically. If it wants something outside its permissions, it has to ask me. And if it tries anyway, it gets an error message.

And, unsurprisingly, that becomes another instruction the model follows: Stay in your lane. Ask the human. Don't touch things you aren't allowed to touch.

The model doesn't need to understand why the restriction exists, it just needs to consistently observe that the environment enforces it. And that brings me to the actual conclusion of all of this.

So, Did I Make a Small Model Smart?

No.

I didn't retrain Qwen. I didn't fine-tune it. I didn't give it a larger context window or bolt another fifty billion parameters onto it. I gave it a fucking notebook and a list of rules.

And eventually, it started behaving like a surprisingly competent agent. That's the part I find interesting. The model didn't suddenly become more intelligent. Its weights never changed. But the system around the model changed dramatically.

The context window is a terrible place to store state. Reasoning gets pruned. Conversations get compacted. Important details disappear. The model's own thoughts are transient by design. So I told it to write everything down. The workplan became working memory, the daily log became episodic memory, and the long-term memory became durable knowledge.

The harness became the thing that enforced boundaries the model couldn't reliably enforce itself. The model sits in the middle of this network of rules, tools, and memory structures. Just making decisions and following the protocol.

That distinction matters because some of the failures were just as instructive as the successes.

One time, I left it running overnight. Every unwatched turn did the actual work. The artifact writes happened, and the task progressed. And not a single one wrote the daily memory file. The protocol worked exactly where I was watching it and stopped working where I wasn't. That's a pretty fucking good demonstration of why "the prompt says so" isn't enough.

The solution wasn't to yell harder at the model, it was to add verification. Now the protocol has rules for detecting its own failures. If enough misses accumulate, the model is required to tell me that the gate isn't holding.

That's the part that I think gets missed when people talk about "agentic" systems. The useful thing isn't that the model can follow a hundred instructions, it's that the instructions can describe a system that observes whether it is following the instructions.

And the really cool part is that the system doesn't have to stay static. I gave the model mechanisms for identifying failures and amending its own operating rules, and it started using them. The model isn't changing its weights. It's changing the rules that it has to follow, effectively making itself smarter in the process.

So no, I didn't make a small model smart. I made the model's environment increasingly difficult to be stupid inside, and it turns out that is a surprisingly powerful thing to do. And apparently, once you give the model enough rules, it can start manufacturing improvements to those rules that make it even harder for the model to be stupid.