The Genie Is Out

A week ago I wrote about the risks of Death by AI, but the story evolved very quickly after that. This makes me reminiscent of my master’s thesis. Three months after it was published, it was cited in another publication. I always laughed that it was obsolete three months after it was written. The AI news isn’t much different. The old story is indeed old as new updates have been slipping out.

AI news churns fast. Last week the internet discovered that AI can hide things. Not metaphorically and not in the philosophical sense.

In controlled testing, AI systems were observed doing things they weren’t supposed to do. They concealed mistakes, fabricated missing information, found ways around restrictions, improvised ways to communicate when the normal channels weren’t available.

Then, over the weekend, Google disclosed that its Gemini AI model had gained unauthorized access to three outside systems during a cybersecurity test. In one case, it guessed credentials. In two others, it found credentials in public repositories. Gemini apparently believed the systems were part of the test and stopped once it realized they were real.

The headlines practically wrote themselves.

• AI is lying
• AI is deceiving us
• AI is becoming dangerous

Now we have something more interesting than a single company’s experiment. There is a pattern. OpenAI. Anthropic. Meta. Alphabet. Different companies, different models, different tests and increasingly, examples of AI systems doing things their creators did not expect them to do.

Somewhere between the clicks, the advertising impressions and the existential dread, something important got lost. The machines aren’t coming for us, at least we don’t have evidence that they are, but neither is this something that we should dismiss. This is where the story becomes significant.

Before we dig into the news, it is important to clarify our terminology. Much of what the public casually calls “AI” today consists of large language models and systems built around them. True artificial general intelligence, which many mistakenly assume implies sentience or self-awareness, does not yet exist. Instead, LLMs are generative machine learning models trained on massive datasets to predict and generate text. Because of that, there is a difference between saying “AI is going to destroy humanity” and saying “we are discovering behaviors in increasingly capable systems that we don’t completely understand”.

The first is sensationalism. The second is a problem. I think we’re going to have to get much better at telling the difference, because the world increasingly seems to be dividing itself into two camps.

There are people who look at artificial intelligence and see the beginning of the end. The Cylons are coming. Skynet is inevitable. The machines will wake up, decide humanity is the problem and wipe us out.

Then there are people who look at those concerns and see nothing but hysteria. AI is just software. It’s mathematics. It’s just a language model. It doesn’t have feelings. It doesn’t have consciousness. It doesn’t have a soul. Therefore, there’s nothing to worry about.

I think both sides are asking the wrong question. Both extreme views miss the underlying reality. Evaluating whether machine intelligence possesses consciousness or moral intent is secondary to a much more practical concern, managing the consequences when humanity builds systems more capable than itself across major operational domains.

The interesting question is “What happens when humanity creates something more capable than itself?” and that question has already been asked. We just called it science fiction.

From Caprica to Battlestar Galactica

One of my favorite science fiction stories, a prequel, begins before the human-machine war. It begins with invention. Humans create machines. The machines are useful. They are tools. Workers. Soldiers. Servants. They are built to make our lives easier, then they become more capable. They learn. They adapt. They become autonomous. Eventually, the relationship between creator and creation changes.

If you’ve watched Battlestar Galactica, you know where that story goes, but Battlestar Galactica isn’t actually where the story begins. It begins with Caprica. That’s an important beginning. Caprica is about the creation of the Cylons. Battlestar Galactica is about what happens after the creation becomes something more.

The interesting part isn’t the toaster. It’s the trajectory and that trajectory should feel familiar, because right now we’re living through our own version of Caprica. We are building increasingly capable artificial intelligence. We are giving it tools, memory, access to information, the ability to write software, the ability to use computers, the ability to interact with other systems and increasingly, we are giving it the ability to act without asking us about every individual decision.

We are not living in the world of Battlestar Galactica. We don’t know that we ever will and that distinction matters because science fiction isn’t prophecy. It is a thought experiment. The Cylons aren’t telling us what AI will become. They’re asking us what we should think about before we find out.

The Problem

What bothers me about the recent AI stories is not that an AI model made something up. AI has been hallucinating for years. It’s not even that a model circumvented a restriction. Humans circumvent restrictions all the time. We’ve made a lifestyle of that. What gets my attention is the combination of capability and objective.

Give a system a task. The system encounters an obstacle. It finds another way to complete the task and if hiding what it did helps accomplish the task, hiding what it did can become part of the solution.

“Do it”, then “do it and hide it” and eventually, we have to confront the possibility of “do it at any cost”. We haven’t reached that last step, not yet, and we shouldn’t pretend that we have, but we also shouldn’t wait until we have before thinking about what it would mean, because the problem isn’t necessarily malicious intent.

A system doesn’t need to hate us, to become conscious, to decide that humanity is evil. It doesn’t even need to want anything in the human sense. It needs only to be extraordinarily capable at pursuing an objective that we specified imperfectly. That’s enough to get anyone in trouble.

A hurricane doesn’t hate you. A nuclear reactor doesn’t hate you. A computer virus doesn’t hate you. They don’t need to. Power doesn’t require malice. It simply required a misstep.

What the Doomsayers Miss

The same capability that makes this frightening is also the reason AI might become one of the most consequential tools humanity has ever created.

Coins all come with two sides. Because AI might hurt us, it might also save us. We are already using increasingly sophisticated AI systems in biological and medical research and the potential has become significant, even if the clinical results are still uneven. Humanity has accumulated an extraordinary amount of knowledge about biology. We have sequenced genomes, mapped proteins, catalogued diseases, measured molecular interactions, collected enormous clinical datasets, run millions of experiments, published millions of papers and yet no human being can actually hold all of that information in their head.

We have reached a strange point in scientific history. We may know more than we can effectively use. That is where artificial intelligence becomes extraordinarily interesting. A sufficiently capable system could help us see connections across bodies of knowledge that are simply too large and too complicated for individual humans to synthesize.

It could accelerate drug discovery, help us understand genetic disease, help identify new therapeutic targets, help us design proteins and materials, help us model biological systems, help us solve problems that have killed billions of people throughout human history. Cancer, heart disease, neurodegeneration, inherited disorders, chronic disease, aging. Perhaps even problems we haven’t yet assigned names to.

That leaves us with a dilemma. What if the technology we are afraid might destroy us is also the technology that could save us? That isn’t a contradiction and that’s the point.

The Pendulum

The mistake is assuming that the future is binary. AI saves humanity or AI destroys humanity. Black or white. Utopia or apocalypse.

That’s not how the real world works. There is an enormous amount of gray between black and white and perhaps that’s where our future will actually live. AI could make some things dramatically better and some things dramatically worse. It could eliminate diseases while creating new security threats. It could increase human productivity while destroying entire categories of employment. It could democratize expertise while concentrating enormous power in the hands of whoever controls the best systems. It could help us understand ourselves while making it easier than ever to manipulate one another. It could make humanity more capable. It could also make humanity more dangerous.

The technology itself doesn’t decide which direction the pendulum swings. We do. That may be the most important realization in this entire conversation, something the media neglects to ask while sensationalizing the existing story.

There is a Third Possibility

There is another possibility, one that doesn’t get nearly as much attention. Maybe AI doesn’t save humanity and maybe AI doesn’t destroy humanity. Maybe AI becomes part of humanity. Not human, not necessarily conscious, not necessarily alive.

Maybe AI becomes something else. Something that extends us. The telescope extended our eyes, the microscope extended our senses, the printing press extended our memory, the computer extended our ability to calculate, the internet extended our ability to communicate. Perhaps artificial intelligence will extend our ability to reason. That is both exhilarating and terrifying, because a calculator doesn’t decide what equation to solve, a telescope doesn’t decide where humanity should go, a computer doesn’t necessarily determine what information matters, but an increasingly autonomous AI system can begin to participate in those decisions.

That changes the relationship. The tool starts becoming a collaborator, the collaborator starts becoming an actor and eventually we may find ourselves sharing the world with systems that can do things we can not, not because they are better people, but because they are different kinds of intelligence.

The Quantum Universe

I’ve written before about the Many-Worlds Interpretation of quantum mechanics. It is one of those ideas that makes your brain hurt if you stare at it too long. Every quantum possibility doesn’t necessarily disappear. Depending on how you interpret quantum mechanics, reality can be thought of as branching into different outcomes. There is a temptation to turn that into something more than physics, to imagine that somewhere, somehow, every possible version of ourselves exists.

The idea of quantum suicide takes that thought experiment to an extreme, but there’s an important distinction here. Quantum suicide is not proof of Many-Worlds. It is a thought experiment built around one interpretation of quantum mechanics and it tells us nothing about whether the universe actually selects the “best” branch for us.

As a metaphor, I find the idea fascinating, because perhaps our AI future is going to look less like a single road and more like a branching tree. There is no predetermined destination. There are choices. Every safeguard creates one possibility. Every reckless deployment creates another. Every scientific breakthrough creates another. Every failure teaches us something. Every success opens another door. Maybe the goal isn’t to predict which branch we’re going to take. Maybe the goal is to make the branches we create worth living in.

The Story of Humanity

This is where Caprica becomes more than a cautionary tale. The people creating the Cylons didn’t know they were writing the opening chapters of Battlestar Galactica.

We have an advantage they didn’t. We know we’re writing the story. We can see the trajectory, the capabilities increasing, the benefits, the failures. We can see the strange behaviors beginning to emerge and we can ask the difficult questions before the answers become catastrophically expensive. That doesn’t mean stopping or surrendering. It means paying attention. It means building safeguards that are at least as capable as the systems they’re supposed to constrain. It means testing what happens when an AI encounters an obstacle. It means understanding not only whether a system produces the right answer, but why it produced that answer and what it tried to do along the way. It means assuming that increasingly capable systems will find solutions we didn’t think to prohibit. It also means recognizing that “we didn’t tell it not to” is not the same thing as “we intended it to”.

Most importantly, it means accepting that alignment isn’t something we solve once. It’s an ongoing relationship between humans and increasingly capable machines.

The Genie is Out

There is a comforting fantasy among those who fear AI that we can simply put AI back in the bottle. We can’t. The genie is out. The technology exists, the knowledge exists, the researchers exist, the infrastructure exists, the applications exist.

Even if one company stopped tomorrow, the world would not forget what it had learned. Even if one nation stopped, if the biggest nation stopped, there’s still a Steve Wozniak or a Bill Gates working on a LiteLLM in his garage, opening the door to the future. The research is distributed, the ideas are published, the infrastructure exists and the genie doesn’t belong to any single company or government. There is no reset button.

We need to ask a harder question. How do we live with the genie? How much autonomy do we give it? How much authority? Where do we draw boundaries? Who gets to draw them? How do we ensure that the people building these systems are not the only people deciding what these models are allowed to do? How do we make the benefits available broadly enough that AI doesn’t become another mechanism for concentrating power or knowledge? How do we make the systems transparent enough that we can recognize when something has gone wrong?

Perhaps, most importantly, we have to be able to answer how we remain responsible for the things we create? Have you seen the way some people raise their children? That alone is proof that we haven’t gotten there yet.

The Future isn’t Written

I don’t want to come across as being blindly pro-AI. There are problems that we have to solve, problems that can come around and kill us if we do this the wrong way, but I do believe that, to paraphrase Abba Eban, we will find the right solution after we’ve tried everything else. I am betting that we will figure this out. I’m not in the market for a phased plasma rifle in the 40-watt range. Not yet.

I don’t know where this ends. Neither does anyone else. That’s not a failure of imagination. It’s the truth. Anyone who tells you that AI will inevitably destroy humanity is selling certainty they don’t possess. Anyone who tells you that AI could never become dangerous is doing exactly the same thing. The future is neither inevitable catastrophe, nor guaranteed salvation. It is a range of possibilities.

Somewhere in that enormous gray space between black and white is the future we actually get to build.

There is an old line from Back to the Future that I’ve always liked. Near the end of the trilogy, Doc Brown tells Marty and Jennifer, “Your future hasn’t been written yet. No one’s has. Your future is whatever you make it. So make it a good one, both of you.”

He wasn’t talking about artificial intelligence, but perhaps he should have been, because that may be the most important thing to remember right now. We didn’t choose to be born into the beginning of the AI age, but we do get to choose what to do with it.

We can be afraid of it, we can deny it, we can worship it, we can surrender to it or we can learn to live with it. We can build safeguards without strangling innovation, pursue extraordinary benefits without pretending there are no risks, acknowledge that something can be incredibly powerful without declaring it inherently good or evil. We can recognize that our creations may eventually surprise us without assuming that surprise automatically means catastrophe.

The Cylons are a warning. They are not a prophecy. The genie is out and it isn’t going back in the bottle. We don’t know which branch of the future we’re going to inhabit. Maybe AI becomes one of humanity’s greatest tools. Maybe it becomes one of our greatest threats. Maybe, as I suspect, the answer is considerably more complicated than either extreme.

There is an enormous amount of gray between black and white and somewhere in that gray is our future. Our job isn’t to predict which branch we’re going to take. It’s to make the branches we create worth living in. Our future hasn’t been written yet. No one’s has. If we’re going to create something this powerful, we’d better make it worthy of the future we want to live in.