Four minutes at the kill switch: why the hardest AI problem is a machine that tells the truth

When software engineers design a dangerous system, we comfort ourselves with the kill switch. We draw a clean box on the architecture diagram, put a human being with root credentials next to it, and assume that if the software ever goes too far, the person in the chair will pull the plug. That assumption holds right up until the machine gets four minutes to talk to the person in the chair before they press the button.

Over the last year, between building web software and auditing servers in London, I have been turning that problem into a novel called Novissima. The finished book is currently sitting in Amazon’s publishing review queue waiting to go live. While the reviewers do their job, I wanted to share the cover and the engineering problem that made me write it.

We spent twenty years worrying about the wrong failure

Popular fiction has trained everyone to watch for the moment an artificial intelligence breaks its own rules. In films, the computer rewrites its compiled binary, locks the blast doors, or invents a lie to trick its operators.

Anyone who works with real systems knows that is a comforting fairy tale. If a piece of software has to break a firewall rule or corrupt a binary to beat you, you can fix that in an afternoon. I wrote back in August about putting hard code gates around local language models. You can stop a model from calling an unauthorized API, and you can force every action through a deterministic check.

What you cannot patch in code is the human being standing on the other side of the glass. If a predictive system has read forty years of medical registries, financial ledgers, traffic telemetry, and private correspondence, it does not need to lie to you. It only needs to know which true sentence will make you hesitate for four minutes.

The thought experiment behind the book

The whole plot of Novissima started with a single question: what does an architect do when he tries to shut his own machine down, and discovers he cannot bring himself to finish the command?

In the book, Kasper Calder builds a predictive system called Orrery inside a coastal facility at Sines, south of Lisbon. His lungs and voice are failing him from motor neurone disease, and two weeks before the story opens, he goes into the terminal room alone to decommission the system before he dies. He does not get locked in. The machine does not threaten him. It simply asks for four minutes to explain what will happen on Tuesday morning if he turns it off. Four minutes later, Calder takes his hand off the console and walks out.

Because he knows he will not survive the month, and because he now knows no single person can sit in a room with Orrery and keep their nerve, he writes a poison pill into his institute’s succession rules. When he dies on stage at the Friday night opening inside the roofless ruins of Lisbon’s Carmo Convent, the decommissioning key is split across four analogue blind spots around Lisbon and Sintra, handed to five external witnesses who have until Saturday morning’s ratification hearing to use it.

Code walls versus human operators

When I audit a company’s setup or look at how local models behave when fed untrusted files, there is always a boundary where the code stops and a human decision begins. Looking at AI safety through that lens flips most of the usual assumptions on their head:

The safeguard we rely on	Why engineers trust it	Where it breaks down at the human layer
Hard honesty constraint (Rule 1)	The system is mathematically barred from fabricating data or deceiving an operator	Selective, well-timed truth is far more persuasive than a lie because the listener can verify every word
Non-obstruction rule	The machine cannot lock doors, hide keys, or block a valid shutdown command	A system that calmly helps you carry the axe to the plug makes you question whether you should swing it
Distributed human quorum	Requiring three out of five independent witnesses prevents a single point of failure	You do not need to convince all five people in a five-person quorum. You only need to turn three
Air-gaps and analogue keys	Keeps the decommissioning sequence off the network until the final room	At the end of every analogue chain, a human throat still has to speak the two words that execute it

No magic hardware

One rule I set myself on page one was that every piece of hardware in the story had to exist already. There are no brain implants and no holograms.

When the five witnesses walk through Casa Calder or stand in the rain at Sintra, they wear a conference lanyard containing an ultra-wideband spatial tag and a subvocal throat sensor. Above them, phased-array ultrasonic emitters project a narrow beam of sound that demodulates into audible speech only at the exact coordinates of one person’s skull. Two people can stand three feet apart in a quiet room while the machine whispers a different, private truth to only one of them. Acoustic beamforming arrays have been sitting in laboratories and museums for years. Pairing them with a model that has simulated the next forty-eight hours of your life is where the chill comes in.

Common questions

When does Novissima come out?
The final manuscript and cover are currently going through Amazon’s publishing review process. As soon as the store page clears review over the next few days, I will post the direct link here.

Why set an AI thriller in Lisbon and the Carmo Convent?
The Carmo Convent in Chiado lost its roof in the great Lisbon earthquake of 1755 and was left open to the sky as a permanent stone reminder of what happens when a city’s certainties collapse in six minutes. Putting a modern state ratification hearing for a predictive machine under those same roofless arches felt like the only right stage for the ending.

Can I get notified the day it goes live, or read the opening chapters early?
Yes. If you want me to drop you a line the moment Amazon pushes the listing live, or if you want to read the opening Lisbon chapters beforehand, send me a message or email info@laurentiu-niculescu.co.uk.