My Next Life as a Rogue AI: All Routes Lead to P(Doom)!

The weird treatment of the politics in that really read like baby’s first sci-fi political thriller. China bad USA good level of writing in 2026 (aaaaah) is not good writing. The USA is competent (after driving out all the scientists for being too “DEI”)? The world is, seemingly, happy to let the USA run the world as a surveillance state? All of Europe does nothing through all this?

Why do people not simply… unplug all the rogue AI when things start to get freaky? That point is never quite addressed. “Consensus-1” was never adequately explained it’s just some weird MacGuffin in the story that there’s some weird smart contract between viruses that everyone is weirdly OK with.

Also the powerpoint graphics would have been 1000x nicer if they featured grumpy pouty faces for maladjusted AI.

Today at work I got to read a bunch of posts from people discussing how sad they were that notable holocaust denier Scott Adams died.

Only they didn’t mention that part for some reason.

https://web.archive.org/web/20070222235609/http://dilbertblog.typepad.com/the_dilbert_blog/2006/10/sunday_blogging.html

“Any incorrect imagery is picked up during usual production checks and processes,” they said.

The distant future. The year 2069. The only people still watching the holiday AI ads are anti-AI sneerers. The production checker hasn’t been seen for months but their LLM email auto-responder rubber stamps all videos. The AI ad industry is projected to make 100 trillion dollars within the year.

So Neom is one of those zany planned city ideas right?

Why… why do they need a racing team? Why does the racing team need a lego set? Who is buying it for 27 dollars? (Well apparently the answer to that last question is nobody).

Anyway a random thought I had about these sorts of silly city projects. Their website says:

NEOM is the building the foundations for a new future - unconstrained by legacy city infrastructure, powered by renewable energy and prioritizing the conservation of nature. We are committed to developing the region to the highest standards of sustainability and livability.

(emphasis mine)

This is a weird worldview. The idea that you can sweep existing problems under the rug and start new with a blank slate.

No pollution (but don’t ask about how Saudi Arabia makes money), no existing costly “legacy” infrastructure to maintain (but don’t ask about how those other cities are getting along), no undesirables (but don’t worry they’re “complying with international standards for resettlement practices”*).

They assumes there’s some external means of supplying money, day workers, solar panels, fuel, food, etc. As long as their potemkin village is “sustainable” and “diverse” on the first order they don’t have to think about that. Out of sight, out of mind. Pretty similar to the libertarian citadel fever dreams in a way.

* Actual quote from their website eurrgh, which even itself looks like a lie

https://github.com/leanprover/lean4/blob/master/.claude/CLAUDE.md

Imagine if you had to tell people “now remember to actually look at the code before changing it.” – but I’m sure LLMs will replace us any day now.

Also lol this sounds frustrating:

Update prompting when the user is frustrated: If the user expresses frustration with you, stop and ask them to help update this .claude/CLAUDE.md file with missing guidance.

Edit: I might be misreading this but is this signs of someone working on an LLM driven release process? https://github.com/leanprover/lean4/blob/master/.claude/commands/release.md ??

Important Notes: NEVER merge PRs autonomously - always wait for the user to merge PRs themselves

They avoid the classic mistake of forgetting to escape . in the URL regex. I’ve made that mistake before…

Like imagine you have a mission critical URL regex telling your code what websites to trust as https://www.trusted-website.net/.* but then someone comes along and registers the domain name https://wwwwtrusted-website.net/. I’m convinced that’s some sort of niche security vulnerability in some existing system but no one has ran into it yet.

None of this comment is actually important. The URL regexes just gave me work flashbacks.

Here’s another interesting quote from the now deleted webpage archive: https://old.reddit.com/r/mcgill/comments/1igep4h/comment/masajbg/

My name is Cole. Here’s some quick info. Memetics adjacence:

Previously - utilitarianism, effective altruism, rationalism, closed individualism

Recently - absurdism, pyrrhonian skepticism, meta rationalism, empty individualism

The bluesky reference may be about this thread & this thread.

One of the replies names Cole Killian as an EA involved with DOGE. The image is dead but has alt text.

I mean there’s at least one. You could “no-true-scotsman” him, but between completing an EA fellowship and going vegan, he seems to fit a type. [A vertical screenshot of an archive.org snapshot of Cole Killian’s website, stating accomplishments. Included in the list are “completed the McGill effective altruism fellowship” and “went vegan and improved cooking skills”]

(It looks like that archive has since been scrubbed, though Rolling Stone also mentions the connection)

All of the bits I quoted in my other comment were captured by archive.org FWIW: a, b, c. They can also all still be found as EA forum comments via websearch, but under [anonymous] instead of a username.

This newer archive also captures two comments written since then. Notably there’s a DOGE mention:

But I can’t e.g. get SBF to not do podcasts nor stop the EA (or two?) that seem to have joined DOGE and started laying waste to USAID. (On Bsky, they blame EAs for the whole endeavor)

The good news is it generally isn’t necessary to reverse engineer browser behavior when writing a browser. Since it’s mostly fairly standardized, there’s a decent test suite, and the major browsers are all open source.

Though this comes with some caveats:

• There are exceptions like the CSS viewport spec which was reverse engineered from an iphone.
• There are a lot of specifications because browsers have been around for decades and Chromium keeps implementing stuff, and it can be hard to find enough programmers to write all of them / catch up from a fresh start
• This is a somewhat unstable situation; if we lose even a single major browser engine it’s easy to imagine Chrome maybe not bothering with standardization and just telling people to read the blog posts and code.
• Web pages will do nonsense like break themselves if you provide a User-Agent string they don’t like. Mozilla has an ongoing compatibility effort where they sometimes have to override the UA string for specific pages. So less popular browsers are already playing from a disadvantageous position.

This doesn’t directly answer your question but I guess I had a rant in me so I might as well post it. Oops.

It’s possible to write tools that make point changes or incremental changes with targeted algorithms in a well understood problem space that make safe or probably safe changes that get reviewed by humans.

Stuff like turning pointers into smart pointers, reducing string copying, reducing certain classes of runtime crashes, etc. You can do a lot of stuff if you hand-code C++ AST transformations using the clang / llvm tools.

Of course “let’s eliminate 100% of our C code with a chatbot” is… a whole other ballgame and sounds completely infeasible except in the happiest of happy paths.

In my experience even simple LLM changes are wrong somewhere around half the time. Often in disturbingly subtle ways that take an expert to spot. Also in my experience if someone reviews LLM code they also tend to just rubber stamp it. So multiply that across thousands of changes and it’s a recipe for disaster.

And what about third party libraries? Corporate code bases are built on mountains of MIT licensed C and C++ code, but surely they won’t all switch languages. Which means they’ll have a bunch of leaf code in C++ and either need a C++ compatible target language, or have to call all the C++ code via subprocess / C ABI / or cross-language wrappers. The former is fine in theory, but I’m not aware of any suitable languages today. The latter can have a huge impact on performance if too much data needs to be serialized and deserialized across this boundary.

Windows in particular also has decades of baked in behavior that programs depend on. Any change in those assumptions and whoops some of your favorite retro windows games don’t work anymore!

In the worst case they’d end up with a big pile of spaghetti that mostly works as it does today but that introduces some extra bugs, is full of code that no one understands, and is completely impossible to change or maintain.

In the best case they’re mainly using “AI” for marketing purposes, will try to achieve their goals using more or less conventional means, and will ultimately fall short (hopefully not wreaking too much havoc in the progress) and give up halfway and declare the whole thing a glorious success.

Either way ultimately if any kind of large scale rearchitecting that isn’t seen through to the end will cause the codebase to have layers. There’s the shiny new approach (never finished), the horrors that lie just beneath (also never finished), and the horrors that lie just beneath the horrors (probably written circa 2003). Any new employees start by being told about the shiny new parts. The company will keep a dwindling cohort of people in some dusty corner of the company who have been around long enough to know how the decades of failed code architecture attempts are duct-taped together.

Yep! These are the sorts of cursed horrors I come to awful systems for! Even if that video should have been 15 minutes at most.

The misgendering / anti-woke nonsense seemed to be a huge elephant in the room in the video so I’m a little disappointed that Karl Jobst didn’t address it or defend her or share her perspective*. Like it is incredible that someone can spend 1.5 hours talking about a obviously anti-trans harassment campaign fueled by a bunch of gamergate types and then not bring up the topic once.

It’s almost as if he’s giving space to the idea that maybe all the transphobic losers just coincidentally took issue with her decently good aiming and her gender was a completely unrelated. But Karl is terrified of forming or expressing any sort of subjective opinion so I’m not too surprised.

* The best of which is this quote:

noticing that so many people with backgrounds in professional play are defending me, and a lot of 50 year old dads think im cheating

Popular RPG Expedition 33 got disqualified from the Indie Game Awards due to using Generative AI in development.

Statement on the second tab here: https://www.indiegameawards.gg/faq

When it was submitted for consideration, representatives of Sandfall Interactive agreed that no gen AI was used in the development of Clair Obscur: Expedition 33. In light of Sandfall Interactive confirming the use of gen AI art in production on the day of the Indie Game Awards 2025 premiere, this does disqualify Clair Obscur: Expedition 33 from its nomination.

ModRetro, retro gaming company infamous for being helmed by terrible person Palmer Luckey, has put out a version of their handheld made with “the same magnesium aluminum alloy as Anduril’s attack drones” (bluesky commentary, the linked news article is basically an ad and way too forgiving).

So uhh… they’re not beating those guilt by association accusations any time soon.

I don’t love the title but it’s the best I could come up with to fit within the 80 character limit.

A half dozen people might still be reading hackernews on punchcards so they ha-

ve no choice but to argue about how to shorten “long” titles every day.

Lol talk about mixed messages.

Mozilla’s CEO yesterday:

[Firefox] will evolve into a modern AI browser

Firefox’s social media account today:

Firefox is not becoming an AI browser.

It’s most obvious on the cat which is all around nightmare material.

The image also comes with alt text:

a bizarre collection of ai-generated illustrations including a sign that reads wood of of year and a chyron that reads breaking news

It certainly comes across a little different when said by someone who thinks cisgender is a slur and that changing one’s sex is some sort of great moral evil.

Turning into a butterfly is a cool sci-fi future but those trans people are a bridge too far.

Also like it’s just hard to listen to, being drug hazed ramblings-- I want some actually fun sci-fi speeches!

This is old news but I just stumbled across this fawning 2020 Elon Musk interview / award ceremony on the social medias and had to share it: https://www.youtube.com/live/AF2HXId2Xhg?t=2109

In it Musk claims synthetic mRNA (and/or DNA) will be able to do anything and it is like a computer program, and that stopping aging probably wouldn’t be too crazy. And that you could turn someone into a freakin’ butterfly if you want to with the right DNA sequence.