GGUFun

Lately I was wondering if it would be possible to build a gguf binary file that, when loaded by ollama, would always answer with the same text, whatever the input.

I don’t mean something like an actual LLM with a strongly worded system prompt, I mean an actual deterministic output.

Turns out: it can be done, and it’s not even too complex. You zero out the impact of the attention heads and MLPs on the residual stream, and hide everything inside the unembedding matrix.

Since everything is zeroed out, the last embedding at the end of the computation will be the embedding of the last token received by ollama, which is the system token marking the start of the assistant’s turn.

The unembedding matrix then maps that embedding to the token containing the entire answer. Done.

Next step: building snake in a gguf.

Even this one can be done. It’s a little bit more complex, and to be honest I would have never been able to build this one without the help of AI.

But it’s there, it works, and makes me wonder: who should I ask to have someone build a .gguf that runs Doom? (just kidding: I don’t think it can be done today).

P.S.

I’ll write a more complex and complete explanation soon, but the game can be played, and the code can be seen at the addresses above.