They are using machine learning to comb the darknets, capturing about 300 threats a week.

About 90% hack application and backdoor recognition, that is for sale, and about 80% hacker forum vulnerability identification.

"These threat warnings include information on newly developed malware and exploits that have not yet been deployed in a cyber-attack"

https://arxiv.org/abs/1607.08583

A putative new idea for AI control; index here.

This is a stub of an idea, looking at "human in the loop"-type restrictions.

An assessor is a particular type of restriction. It involves a (generally) human agent, who is tasked with observing something (the behaviour of the agent, or maybe the expected outcome of its plans), and then deciding on whether to allow the AI to proceed or not.

Many of the methods suitable for preventing the hacking of restrictions can also be used to protect assessors. If you consider that the AI would have three options: "hold a gun to the assessor's head", "seduce the assessor" and "do what the assessor wants", it's clear that the first two options (the first more clearly than the second) are things that would a) be stupid to do if there were no assessor, and b) are behaviours that are very informative as to whether the assessor exists.

However, there are a few more steps we can take with assessors. First of all, the AI need not know which particular human will be an assessor, forcing it to take general human behaviour into account. This is more effective if the assessing process is not one that allows any real interactions between AI and assessor.

Indeed, the assessor need not be a human: it could be a superintelligent agent itself (a hypothetical one, if need be) that has its own motivation.

Lastly, since the assessors are agents and therefore can reach their own predictions, we could ask for the assessor's impressions to be accurate predictions of what the AI intends to do (again, this could be done by having a hypothetical superintelligence assess the accuracy of the assessors' expected predictions). We'll look at ideas about modelling and model accuracy in a subsequent post.

A putative new idea for AI control; index here.

Very much in the spirit of "if you want something, you have to define it, then code it, rather than assuming you can get if for free through some other approach."

Difficult children

Suppose you have a child, that you sent to play in their room. You want them to play quietly and silently, so you want them:

"I'll be checking up on you!"

The child, however, has modelled you well, and knows that you will look in briefly at midnight and then go away. The child has two main options:

1. Play quietly the whole time.
2. Be as noisy as they want, until around 23:59, then be totally quiet for two minutes, then go back to being noisy.

We could call the first option obeying the spirit of the law, and the second obeying the letter.

I was inspired by the recent post discussing self-hacking for the purpose of changing a relationship perspective to achieve a goal. Despite my feeling inspired, though, I also felt like life hacking was not something I could ever want to do even if I perceived benefits to doing it. It seems to me that the place where I would need to begin is hacking myself in order to cause myself to want to be hacked. But then I started contemplating whether this is a plausible thing to do.

In my own case, there are two concrete examples in mind. I am a graduate student working on applied math and probability theory in the field of machine vision. I was one of those bright-eyes, bushy-tailed dolts as an undergrad who just sort of floated to grad school believing that as long as I worked sufficiently hard, it was a logical conclusion that I would get a tenure-track faculty position at a desirable university. Even though I am a fellowship award winner and I am working with a well-known researcher at an Ivy League school, my experience in grad school (along with some noted articles) has forced me to re-examine a lot of my priorities. Tenure-track positions are just too difficult to achieve and achieving them is based on networking, politics, and whether the popularity of your research happens to have a peak at the same time that your productivity in that area also has a peak.

But the alternatives that I see are: join the consulting/business/startup world, become a programmer/analyst for a large software/IT/computer company, work for a government research lab. I worked for two years at MIT's Lincoln Laboratory as a radar analyst and signal processing algorithm developer prior to grad school. The main reason I left that job was because I (foolishly) thought that graduate school was where someone goes to specifically learn the higher-level knowledge and skills to do theoretical work that transcends the software development / data processing work that is so common. I'm more interested in creating tools that go into the toolbox of an engineer than with actually using those tools to create something that people want to pay for.

I have been deeply thinking about these issues for more than two years now, almost every day. I read everything that I can and I try to be as blunt and to-the-point about it as I can be. Future career prospects seem bleak to me. Everyone is getting crushed by data right now. I was just talking with my adviser recently about how so much of the mathematical framework for studying vision over the last 30 years is just being flushed down the tubes because of the massive amount of data processing and large scale machine learning we can now tractably perform. If you want to build a cup-detector for example, you can do lots of fancy modeling, stochastic texture mapping, active contour models, fancy differential geometry, occlusion modeling, etc. Or.. you can just train an SVM on 50,000,000 weakly labeled images of cups you find on the internet. And that SVM will utterly crush the performance of the expert system based on 30 years of research from amazing mathematicians. And this crushing effect only stands to get much much worse and at an increasing pace.

In light of this, it seems to me that I should be learning as much as I can about large-scale data processing, GPU computing, advanced parallel architectures, and the gross details of implementing bleeding edge machine learning. But, currently, this is exactly the sort of thing I hate and went to graduate school to avoid. I wanted to study Total Variation minimization, or PDE-driven diffusion models in image processing, etc. And these are things that are completely crushed by large data processing.

So anyway, long story short: suppose that I really like "math theory and teaching at a respected research university" but I see the coming data steamroller and believe that this preference will cause me to feel unhappy in the future when many other preferences I have (and some I don't yet know about) are effected negatively by pursuit of a phantom tenure-track position. But suppose also that another preference I have is that I really hate "writing computer code to build widgets for customers" which can include large scale data analyses, and thus I feel an aversion to even trying to *want* to hack myself and orient myself to a more practical career goal.

How does one hack one's self to change one's preferences when the preference in question is "I don't want to hack myself?"