Erland Wittkotter

Safe Development of Hacker-AI Countermeasures – What if we are too late?

So far, I have hesitated to post sensational warnings on using smartphones in malware-based cyberwars designed to decapitate governments. Still, I wrote this paper out of concern that this could happen – and cheap marketing talk does not and will not change that.

I don’t know who hacked me, but I assume they got this post. It is spicy (and... (read 7290 more words →)

Hacker-AI is AI used in hacking. It is an advanced super-hacker tool (gaining sysadmin access in all IT devices), able to steal access credentials, crypto-keys, and any secret it needs. It could use every available feature on a device. It could also help malware to avoid detection as a digital ghost and make itself irremovable on all devices it visited. The new quality of Hacker-AI arises from the speed at which it develops new malware solutions on every available platform (CPU/OS).

In previous posts on “Hacker-AI and Digital Ghosts – Pre-ASI” and “Hacker-AI – Does it already exist?”, I have hypothesized that Hacker-AI is feasible, already developed, or potentially ready to be deployed. This... (read 4104 more words →)

Indeed, I didn't see that or forgot about it -- I was pulling my memory when responding. So you might be right that Mayhem is doing RCE.

But what I remember distinctively from the DARPA challenge (also using my memory): their "hacking environment" was a simplified sandbox, not a full CISC with a complex OS.

In the "capture-the-flag" (CTF) hacking competition with humans (at DEFCON 2016), Mayhem was last. This was 6 years ago, we don't know how good it is now (probably MUCH better, but still, it is a commercial tool used in defense).

I am more worried about the attack tools developed behind close doors. I read recently about Bruce Schneier ("When AI... (read more)

Replying toImproved Security to Prevent Hacker-AI and Digital Ghosts

Improved Security to Prevent Hacker-AI and Digital Ghosts

If we have Hacker-AI on a developer machine, we have a huge problem: Hacker-AI could sabotage basic security implementations via hidden backdoors or weak compilation tools. However, I am not giving up on seeking solutions to convince humans/experts that there are no hidden/late modifications or interferences from Hacker-AI. Trust must be earned; this could only come from constantly scrutinized source code and development tools.

The problem is “Trust”. Trust must be earned - Open source will help us to gain the trust of experts - I hope that this will carry some weight
If Hacker-AI (Type 2) would be on developer machines - then we have a huge problem -- but I am not giving up on finding solutions for that situation (I was close to posting on that topic: i.e., on what can we do if Hacker-AI is sabotaging our security implementations) I am not giving up (yet)
Do we have a choice? We must develop security – otherwise, I don’t really want to think about in what world we could live if Hacker-AI is for real.

Yap, I checked out Mayhem a while ago -- it seems to be an automated test case generation engine.
From what I read - they are probably not using Reverse Code Engineering (RCE) tools for their analysis; instead, they use the source of an app – still, this is pretty good. Their software found some vulnerabilities, but honestly, I am a bit disappointed they didn’t find more (because they are likely not going lower level). However, they deserve a lot of credit for being a pioneer in cyber-defense. The Challenge was 6 years ago -- So, is Mayhem all we have? No further progress, no breakthroughs?

My post was inspired by Deepmind’s “Reward is... (read more)

Why has Hacker-AI not been discussed more?

Well, we should start now ...

In 2018, I was listening to the change in US Nuclear Posture -- saying that US could retaliate nuclear on a cyber attack. At that time I thought that kind of counter-threat is way out of line - "disproportional" -- A cyberwar via hacking would be an inconvenience -- rebooting a few machines and being prepared to replace some harddrives. But now I understand better. I believe the security establishment knew much more about our vulnerabilities than what they were sharing with us. Why were they quiet?

4 years later -- do they have a plan? Is nuclear deterrence the only tool that is keeping us safe? Leadership looks different to me.

I just read on BBC news: $10.5 Trillion annual damage by 2025 from cyber-crime.

Yes, we must have a technical solution for Hacker-AI asap.

You mention proactive, preventative steps in damage mitigation.

There are other methods:

deterrence though reliable investigation and criminal prosecution .. and making instigators pay
preventing that local events become global
Most important: accepting that this is a problem

What I don't understand: Why are we quiet about this problem? It seems the people knowing about this problem (a long time) are not even dare to call for help. wow ... how courageous.

In its simplest version: Hacker-AI (Type I) uses AI for hacking arbitrary IT systems, i.e., finding vulnerabilities automatically, e.g., elevating privileges/user roles (sys-admin or system), stealing encryption keys, hiding attacking tools from being detected, or creating code that is irremovable on (once) occupied systems. It would create exploits automatically. In an advanced version, Hacker-AI (Type II) would be actively operating malware on computer systems with sys-admin rights, trying to stay hidden from detection, and making itself irremovable from any later deployed software.

In my post “Hacker-AI and Digital Ghosts – Pre-AGI”, I argued that Hacker-AI and Digital Ghosts are feasible and a serious threat to our civilization. In my follow-up post, “Improved Security... (read 3060 more words →)

Replying toHacker-AI and Digital Ghosts – Pre-AGI

Improved Security to Prevent Hacker-AI and Digital Ghosts

I have posted "Improved Security to Prevent Hacker-AI and Digital Ghosts" -- providing a technical solution for dealing with Hacker-AI and Digital Ghosts.

In a previous post on “Hacker-AI and Digital Ghosts – Pre-ASI”. I discussed multiple problems contributing to our vulnerabilities – facilitating the development of an AI application (Hacker-AI) that could threaten our freedom, privacy, and quality of life.

I hypothesized that a Hacker-AI/Digital Ghost finds and uses (new) vulnerabilities in existing software or modifies software to create backdoors or sleeper code. The core hypothesis was that Hacker AI is flexible enough to find ways to manipulate every system (i.e., any OS, CPU, or device type). With Reverse Code Engineering, it could not just steal crypto-keys but also modify other software and try to avoid detection by manipulating the OS and removing all data... (read 3525 more words →)

Replying toHacker-AI and Digital Ghosts – Pre-AGI

Overestimating to which degree Hacker-AI could make itself undetectable? And do I potentially underestimate the effort of making a digital ghost undetectable for malware detection? I disagree because I have answered the following questions for myself.

(1) How conceptionally different are the various operating systems? In 1,000s of details, I believe they are different. But the concepts of how they are designed and written are similar among all multitasking/multithreading OS. Even multi-processing OS are built on similar but extended concepts.

(2) If we ask kernel developers: could they keep app(s) operational but disappear them? I believe he would have multiple ideas on how he could do that. Then we could ask what he could... (read more)

Replying toHacker-AI and Digital Ghosts – Pre-AGI

I think you are massively overestimating the ability of even a very strong narrow hacker AI to hide from literally everyone.

I seriously hope you are right, but from what I’ve learned, Reverse Code Engineering (RCE) is not done quickly or easily, but it’s a finite game. I know the goal; then, I believe you can define rules and train an AI. RCE is labor-intensive; AI could save me a lot of time. For an organization that hires many of the brightest IT minds, I m convinced they ask the right questions for the next iteration of Hacker-AI. I may overestimate how good Hacker-AI (already) is, but I believe you underestimate the motivation... (read more)