Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

[Link] Weaponising Twitter bots and political algos.

1 Post author: morganism 05 March 2017 09:39PM

Comments (111)

Comment author: morganism 05 March 2017 09:42:10PM 2 points [-]

Top half of the article is about politics, but halfway down they start getting into info war and bots and AI.

"THE war of the bots is one of the wilder and weirder aspects of the elections of 2016. At the Oxford Internet Institute’s Unit for Computational Propaganda, its director, Phil Howard, and director of research, Sam Woolley, show me all the ways public opinion can be massaged and manipulated. But is there a smoking gun, I ask them, evidence of who is doing this? “There’s not a smoking gun,” says Howard. “There are smoking machine guns. There are multiple pieces of evidence.”

“Look at this,” he says and shows me how, before the US election, hundreds upon hundreds of websites were set up to blast out just a few links, articles that were all pro-Trump. “This is being done by people who understand information structure, who are bulk buying domain names and then using automation to blast out a certain message.”

One of the things that concerns Howard most is the hundreds of thousands of “sleeper” bots they’ve found. Twitter accounts that have tweeted only once or twice and are now sitting quietly waiting for a trigger: some sort of crisis where they will rise up and come together to drown out all other sources of information.

On its website, Cambridge Analytica makes the astonishing boast that it has psychological profiles based on 5,000 separate pieces of data on 220 million American voters

Bio-psycho-social profiling, I read later, is one offensive in what is called “cognitive warfare”. Though there are many others: “recoding the mass consciousness to turn patriotism into collaborationism,” explains a Nato briefing document on countering Russian disinformation written by an SCL employee. “Time-sensitive professional use of media to propagate narratives,” says one US state department white paper. “Of particular importance to psyop personnel may be publicly and commercially available data from social media platforms.”

Out on Twitter, the new transnational battleground for the future, someone I follow tweets a quote by Marshall McLuhan, the great information theorist of the 60s. “World War III will be a guerrilla information war,” it says. “With no divisions between military and civilian participation.”

By that definition we’re already there."

Comment author: DryHeap 09 March 2017 05:32:08PM 5 points [-]

I would imagine that there is evidence of bot usage on both sides of the aisle. There seems to be an effort to propagate anti-Trump messages on LessWrong, which I find to be partisan and largely out of the scope of this board.

Comment author: Viliam 09 March 2017 06:58:07PM 2 points [-]

It's happening not just in USA. In Slovakia there was recently a research published about fake accounts commenting on social networks and comment sections of major newspapers. It's quite scary how effective the guerrilla information war can be. This is how it works, in a nutshell:

You hire a few people (you really don't need many of them) working for minimal wage, speaking the language of the target country. Their task is to create dozen accounts on various websites. If names and photos are required, they just make fake names and steal someone's photo from google images. (If someone finds that out and bans the account, no problem, they create a new one).

Separately from these people, you have a few smart guys who decide what memes they want to spread. They choose the message, compose a few dozen text samples saying the same thing using different words. Then they e-mail the text samples to the minimum-wage people, whose task is essentially to keep copy-pasting these messages on many places with minor modifications, for 8 hours a day.

Imagine a group with e.g. one manager, two smart guys designing the memes, and five stupid guys who copy them. How many comments they can generate each day? How much can they influence a perception of an average reader?

Comment author: DryHeap 16 March 2017 11:22:08PM 0 points [-]

Thanks for the information! May I get a link to that research paper? My google-fu is apparently weak.

Comment author: Viliam 17 March 2017 09:39:24AM 0 points [-]

Sorry, I have a problem finding it now too. :(

Comment author: Lumifer 09 March 2017 07:52:09PM *  0 points [-]

Here are a couple of useful links - from the New York Times and the Guardian.

Information wars are quite real. How much actual impact do they have is a different question.

A fun possibility to consider: do we have Russian mole here on LW? X-D

Comment author: Viliam 10 March 2017 02:01:44PM *  1 point [-]

That form of spamming hundreds of websites by minor modifications of messages optimized for an average reader obviously wouldn't survive on LW. And optimizing messages specifically for LW wouldn't be cost-effective. The operation needs to scale well; there is a whole internet to take over, and even Kremlin's propaganda budget has its limits.

But I could imagine a multi-tiered approach. Something like categorizing the websites into a few groups, and crafting messages for each of these groups separately. The categorization itself could actually be quite simple: for each category, you choose a few "prototypical" websites, and then let a neural network assign the right category for everything else. Then you do the same thing -- have a smart guy writing a message optimized to impact people in given category, and a stupid guy to spam all websites assigned into given category with small modifications of the message -- for each category separately.

With enough categories, I could imagine that one of them could be compatible enough with LW. Probably "rationalists" is too narrow, but something like "educated people" or maybe even "STEM" could get among the first dozen categories.

Comment author: SnowSage4444 18 March 2017 05:38:26PM 1 point [-]

Whenever a place sinks into the #NeverTrump pit of stupidity, it never crawls back out.

Trump won because the left forgot how to argue rationally, and Trump knew how to make his supporters feel smart for not believing the paranoid moronic more-biased-every-day media. Trump outsmarted every leftist on the planet. At this point, he deserves America, and if he turns out to be half as bad as the libtards say he's going to be, he'll be kicked out in seconds. So why care? Why try to work people up into a paranoid frenzy, when that's failed once before and it'll fail again and again until the left learns how to admit its many flaws and mistakes and learn from them?

Comment author: gjm 19 March 2017 01:48:44AM 0 points [-]

libtards

You know what you were saying about "the #NeverTrump pit of stupidity"? Things like "libtard" are the same thing on the other side. Do you seriously believe that that sort of terminology helps produce productive discussions?

Comment deleted 11 March 2017 03:07:46AM [-]
Comment deleted 09 March 2017 08:39:26PM [-]
Comment author: Lumifer 09 March 2017 08:43:27PM 0 points [-]

I want to take a look at his data before I believe him.

Comment deleted 11 March 2017 03:43:28AM [-]
Comment deleted 11 March 2017 03:08:34AM [-]
Comment deleted 11 March 2017 02:48:15AM [-]
Comment deleted 10 March 2017 03:04:00AM [-]
Comment deleted 10 March 2017 03:33:24PM [-]
Comment deleted 11 March 2017 03:44:06AM [-]
Comment deleted 11 March 2017 03:09:12AM [-]
Comment deleted 11 March 2017 02:48:57AM [-]
Comment author: know40 26 March 2017 10:32:08PM 0 points [-]

In particular, this:

A majority of online and social media defenders of Obamacare are professionals who are "paid to post," according to a digital expert.

"Sixty percent of all the posts were made from 100 profiles, posting between the hours of 9 and 5 Pacific Time," said Michael Brown. "They were paid to post."

He began investigating it after his criticism of the former president's health insurance program posted on the Obamacare Facebook page. He was hit hard by digital activists pretending to be regular people. She reports that he evaluated 226,000 pro-Obamacare posts made by 40,000 Facebook profiles.

"Digital activists are paid employees; their purpose is to attack anyone who's posting something contrary to the view of the page owner wants expressed," he told Attkisson. "Sixty percent of all the posts were made from 100 profiles, posting between the hours of 9 and 5 Pacific Time."

Comment deleted 26 March 2017 08:47:45PM [-]
Comment deleted 26 March 2017 06:42:29AM [-]
Comment deleted 23 March 2017 04:54:45AM [-]
Comment deleted 23 March 2017 03:24:53AM [-]
Comment deleted 23 March 2017 03:07:29AM [-]
Comment deleted 22 March 2017 06:40:50AM [-]
Comment deleted 20 March 2017 07:35:24PM [-]
Comment deleted 19 March 2017 10:06:59PM [-]
Comment deleted 19 March 2017 09:44:50PM [-]
Comment deleted 19 March 2017 09:20:47PM [-]
Comment deleted 19 March 2017 08:53:06PM [-]
Comment deleted 19 March 2017 05:41:52PM [-]
Comment deleted 19 March 2017 05:27:15AM [-]
Comment deleted 18 March 2017 05:19:24PM [-]
Comment deleted 18 March 2017 08:18:30AM [-]
Comment deleted 18 March 2017 06:52:02AM [-]
Comment deleted 16 March 2017 07:30:54AM [-]
Comment deleted 15 March 2017 07:30:13AM [-]
Comment deleted 15 March 2017 07:07:55AM [-]
Comment deleted 15 March 2017 06:20:50AM [-]
Comment deleted 14 March 2017 11:32:03PM [-]
Comment deleted 14 March 2017 10:12:59PM [-]
Comment deleted 14 March 2017 07:04:31PM [-]
Comment deleted 13 March 2017 06:18:34PM [-]
Comment deleted 12 March 2017 07:44:52AM [-]
Comment deleted 12 March 2017 06:45:38AM [-]
Comment deleted 12 March 2017 06:13:44AM [-]
Comment deleted 11 March 2017 10:45:11PM [-]
Comment deleted 11 March 2017 06:17:42PM [-]
Comment deleted 11 March 2017 08:35:41AM [-]
Comment deleted 11 March 2017 08:14:09AM [-]
Comment deleted 11 March 2017 07:35:51AM [-]
Comment deleted 11 March 2017 04:22:58AM [-]
Comment deleted 11 March 2017 04:02:37AM [-]
Comment deleted 11 March 2017 03:42:53AM [-]
Comment deleted 11 March 2017 02:50:47AM [-]
Comment author: niceguyanon 10 March 2017 03:25:06PM 0 points [-]

There seems to be an effort to propagate...

I don't think so, or just a trivial amount. Looking at downvotes or lack of upvotes, I don't get that sense at all. Political talk is almost always discouraged and when it does go on, its fairly even handed.

Comment author: Lumifer 10 March 2017 03:37:06PM 0 points [-]

Looking at downvotes

There ain't no downvotes any more.

Comment deleted 18 March 2017 05:39:52PM [-]
Comment author: gjm 19 March 2017 01:45:59AM *  0 points [-]

Most of them are over-and-over-again reposts from the same person, who was banned years ago for abusive behaviour and keeps coming back and being banned again (and creating huge numbers of sockpuppets to downvote people he doesn't like, which is the main reason why there are currently no downvotes on LW).

[EDITED to clarify:] So what happens is: he posts something, his account gets zapped because he's supposed to be banned and his comments are deleted, and then he makes a new account and posts all the same things again, repeat ad nauseam. I can't imagine he thinks he's actually doing something constructive; I think he does it as a sort of fuck-you to the LW admins.

That would be the person you recently said "Did we just become best friends?" to, by the way.

Comment deleted 24 March 2017 05:57:12PM [-]
Comment author: gjm 24 March 2017 06:03:47PM 3 points [-]

Gods, Eugine, you're boring. You've been banned from LW, you can't accept that, and the best way you can think of to handle this situation is to keep posting the same fucking comments again and again and again and again because ... what? What good outcome do you expect from doing this? Literally the only thing you are doing is adding a bit of annoyance to the lives of people who never did you any harm.

Perhaps it makes you feel like you're outsmarting the moderators or something. I do hope not. Because any idiot can copy and paste things, and anyone a step or two above idiocy can write a script to do it.

Get a life and leave us in peace.

Comment author: Lumifer 24 March 2017 07:38:16PM 0 points [-]

anyone a step or two above idiocy can write a script

LOL. First, are you quite sure you want Eugine to move in that particular direction? Of course there is that, but still...

Second, I think you notions of what people a step or two above idiocy can achieve are a bit... optimistic.

Comment author: gjm 25 March 2017 02:54:24PM 0 points [-]

I rather suspect he already has moved in that particular direction. Could be wrong, though.

Yeah, I'm adopting a definition of idiocy adapted to the local context.

Comment deleted 11 March 2017 03:07:09AM [-]
Comment author: know40 26 March 2017 10:30:40PM 0 points [-]

This reads like what Scott Adams would call cognitive dissonance.

In particular it omits the fact that the (anti-Trump) CEO's of twitter and face have access to more data, and presumably better algorithms, but weren't able to swing the election. Almost as if it's easier to influence people if the direction you're trying to influence them agrees with their day-to-day observations. But that's way to scary a thought for the people at the guardian.

Comment deleted 26 March 2017 08:46:53PM [-]
Comment deleted 26 March 2017 06:22:09AM [-]
Comment deleted 26 March 2017 01:57:14AM [-]
Comment deleted 24 March 2017 02:55:30AM [-]
Comment deleted 23 March 2017 10:45:31PM [-]
Comment deleted 23 March 2017 06:01:52PM [-]
Comment deleted 23 March 2017 04:53:57AM [-]
Comment deleted 23 March 2017 03:24:22AM [-]
Comment deleted 23 March 2017 03:05:25AM [-]
Comment deleted 22 March 2017 06:39:56AM [-]
Comment deleted 20 March 2017 07:33:58PM [-]
Comment deleted 19 March 2017 10:06:27PM [-]
Comment deleted 19 March 2017 09:44:05PM [-]
Comment deleted 19 March 2017 09:20:08PM [-]
Comment deleted 19 March 2017 08:52:26PM [-]
Comment deleted 19 March 2017 05:40:53PM [-]
Comment deleted 19 March 2017 05:26:47AM [-]
Comment deleted 18 March 2017 05:18:44PM [-]
Comment deleted 18 March 2017 08:17:45AM [-]
Comment deleted 18 March 2017 06:49:22AM [-]
Comment deleted 16 March 2017 07:30:13AM [-]
Comment deleted 15 March 2017 05:49:56PM [-]
Comment deleted 15 March 2017 07:29:41AM [-]
Comment deleted 15 March 2017 07:07:21AM [-]
Comment deleted 15 March 2017 06:20:01AM [-]
Comment deleted 14 March 2017 11:31:34PM [-]
Comment deleted 14 March 2017 10:12:37PM [-]
Comment deleted 14 March 2017 07:03:56PM [-]
Comment deleted 13 March 2017 06:16:55PM [-]
Comment deleted 12 March 2017 07:44:18AM [-]
Comment deleted 12 March 2017 06:45:06AM [-]
Comment deleted 12 March 2017 06:13:09AM [-]
Comment deleted 11 March 2017 10:44:12PM [-]
Comment deleted 11 March 2017 06:17:03PM [-]
Comment deleted 11 March 2017 08:35:07AM [-]
Comment deleted 11 March 2017 08:13:33AM [-]
Comment deleted 11 March 2017 07:35:13AM [-]
Comment deleted 11 March 2017 04:22:21AM [-]
Comment deleted 11 March 2017 04:01:57AM [-]
Comment deleted 11 March 2017 03:42:23AM [-]
Comment deleted 11 March 2017 02:47:25AM [-]