Stochastic systems

Traditional software is deterministic; same input, same output. LLMs are stochastic. We spent decades mastering one paradigm. Now we’re building with its opposite.

Traditional software systems are deterministic. Fixed input gives fixed output. That’s one of the beautiful things about computers. Outcomes are predictable. Bugs can be traced, reproduced, understood with logic, and debugged. The entire system is a stack of logically connected parts.

We got good at this. Really good.

We learned where the rubber hits the road, where deterministic systems meet the chaotic real world. Where humans click buttons they shouldn’t, enter data incorrectly, or unintentionally bypass the system in ways nobody anticipated.

So we built fault-tolerant systems. Systems designed to handle the unexpected gracefully. Input validation. Error boundaries. Fallback states. An entire engineering discipline built around the principle: the system is predictable, the world isn’t, and we need to bridge that gap.

And where users and the physical world (a physical hard-drive failing for example) used to be the chaotic actors in those systems, we are now adding a third catogory; Large Language Models.

LLMs are fundamentally different.

They are stochastic systems. Systems where the same input can produce different outputs. Not because something went wrong, but by design. The randomness is the feature.

Stochastic systems, from the Greek word “stokhastikos”, meaning “skilled in aiming”.

This breaks everything we know about testing, debugging, and reliability. You can’t reproduce a bug in a stochastic system the way you can in a deterministic one. You can’t write a test that guarantees a specific output. The system doesn’t have “correct” behavior in the traditional sense, it has a probability distribution of behaviors.

We spent decades mastering deterministic systems. Now we’re building products that embed their opposite. Stochastic components inside deterministic shells. Unpredictability wrapped in predictability.

That’s not just an engineering challenge. It’s a philosophical shift in how we think about software.