jimrandomh comments on [link] [poll] Future Progress in Artificial Intelligence - Less Wrong

8 Post author: Pablo_Stafforini 09 July 2014 01:51PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (89)

You are viewing a single comment's thread. Show more comments above.

Comment author: jimrandomh 10 July 2014 01:51:25PM 5 points [-]

This is not magic, I am not a layman, and your beliefs about computer security are wildly misinformed. Putting trojans on large fractions of the computers on the internet is currently within the reach of, and is actually done by, petty criminals acting alone. While this does involve a fair amount of thinking time, all of this thinking goes into advance preparation, which could be done while still in an AI-box or in advance of an order.

Comment author: XiXiDu 10 July 2014 05:01:29PM 0 points [-]

This is not magic, I am not a layman, and your beliefs about computer security are wildly misinformed. Putting trojans on large fractions of the computers on the internet is currently within the reach of, and is actually done by, petty criminals acting alone.

Within moments? I don't take your word for this, sorry. The only possibility that comes to my mind is by somehow hacking the Windows update servers and then somehow forcefully install new "updates" without user permission.

While this does involve a fair amount of thinking time, all of this thinking goes into advance preparation, which could be done while still in an AI-box or in advance of an order.

So if I uploaded you onto some alien computer, and you had a billion years of subjective time to think about it, then within moments after you got an "Internet" connection you could put a trojan on most computers of that alien society? How would you e.g. figure out zero day exploits of software that you don't even know exists?

Comment author: Lumifer 10 July 2014 05:38:55PM 3 points [-]

Within moments?

Well, what's going to slow it down? If you have a backdoor or an exploit, to take over a computer requires a few milliseconds for communications latency and a few milliseconds to run the code to execute the takeover. At this point the new zombie becomes a vector for further infection, you have exponential growth and BOOM!

Comment author: Nornagest 10 July 2014 05:27:28PM *  3 points [-]

The only possibility that comes to my mind is by somehow hacking the Windows update servers and then somehow forcefully install new "updates" without user permission.

Wouldn't have to be Windows; any popular software package with live updates would do, like Acrobat or Java or any major antivirus package. Or you could find a vulnerability that allows arbitrary code execution in any popular push notification service; find one in Apache or a comparably popular Web service, then corrupt all the servers you can find; exploit one in a popular browser, if you can suborn something like Google or Amazon's front page... there's lots of stuff you could do. If you have hours instead of moments, phishing attacks and the like become practical, and things get even more fun.

How would you e.g. figure out zero day exploits of software that you don't even know exists?

Well, presumably you're running in an environment that has some nontrivial fraction of that software floating around, or at least has access to repos with it. And there's always fuzzing.

Comment author: Lumifer 10 July 2014 05:41:01PM 2 points [-]

if you can suborn something like Google or Amazon's front page...

Also, nowadays if you can suborn the cell towers taking over all the smartphones becomes fast and easy.

Comment author: jimrandomh 11 July 2014 02:41:20AM 0 points [-]

When you are a layman talking to experts, you should actually listen. Don't make us feel like we're wasting our time.

Comment author: [deleted] 11 July 2014 05:02:17AM 0 points [-]

Care to address his valid response point in the 2nd paragraph?

Comment author: jimrandomh 11 July 2014 01:08:25PM 0 points [-]

Nornagest already answered it; the sets of software in and outside the box aren't disjoint.