See Kelsey Piper's discovery that Opus 4.7 can reliably identify her as the author of unpublished text.
Story from my past: at university, I once partook of a game called the "assassins' guild". It was a kind of Battle Royale. Fifty-odd participants would each be circularly assigned two "targets" from the other forty-nine, and instructed to "kill" them (for example, by writing "knife" on a stick and poking them with it, or by shooting them with a nerf gun). You'd be told their halls of residence, so you could find them there, if needed.
Your targets were revealed at 09:00 on the first day. I found my target's Facebook page, found a post announcing her going to uni, and saw she was studying a subject which shared a module with my subject. From there I was able to pull up the timetable of her subject, guess which lectures were mandatory, and notice that she had a mandatory one right now. I "killed" her at 10:00 as she left the lecture hall.
I didn't even get the first kill! Someone else pulled off the same trick even quicker than me!
This is with a smart uni student's level of skill: the only thing it took was effort. If AI is good enough to do this, then privacy removal will be very easy to perform at scale.
There will be no privacy anymore, because of a simple exploit (or molochian mechanic, if you prefer that framing).
1: The government says "We need more access to information to stop [taboo]. If you disagree then you're a [taboo] apologist or somebody who supports [taboo]"
2: The public doesn't dare to call out the government, because anyone who argues the pro-privacy position will be accused of defending [taboo], and merely being accused of immoral behaviour can ruin your life (fired from your job, debanked, blacklisted from flying). It's socially costly to try to defend privacy, but you can easily virtue signal by looking for those who are brave enough to do so, and accusing them of being evil.
Life is split into two halves, our public lives, and our private lives. The former is fake and performative, the latter is real, and filled with illegal behaviour (almost always benign, acceptable in the context in which it occurs, or occuring around people who understand and accept the risks. Laws apply globally and generally, but context only exists locally and specifically, and if you act appropriately locally you act inappropriately globally, and vice versa)
The only safety which has ever e...
2Curated. This is a fairly simple point that I hadn't seen expressed before.
I'd previously thought a bunch about how privacy would change due to having more-of-people's-lives available to be read online, and by having the scale to process large amounts of that data and look for patterns, etc.
I'd thought about intelligence being generally powerful. I'd thought somewhat about what it meant, for most people to have access to more intelligence. But not about how the raw intelligence-at-scale applied to privacy. The "gaydar at scale" example makes the point evocatively.
I have to point out here that author David Brin has been writing about this for years. His nonfiction book on the issue, which also goes into detail about his solution to how to organize a world in which the technology for mass surveillance is possible, is called The Transparent Society.
I once thought it was reasonable to pick up the phone and call someone when I wanted to talk to them, and to pick up my phone when it rang; things have changed, and someone thinking about what's possible could have seen the dilution of that signal into noise coming.
I realize that I don't know what exactly did drive this change? I've mostly seen it attributed to a generational thing - "millennials and people younger than that don't like phone calls" - and while I am in fact a millennial who doesn't like phone calls, I don't know what made it different in th...
Thanks for writing this post! I agree this is important.
it might help you to start behaving as if you're being watched and things about you are more obvious than they once were
I've been banging this drum for a while:
- 2023-02 Preparing for Less Privacy
- 2011-04 Giving up on Privacy
"develop new immigration rules that cities across America would be comfortable cooperating with" -- this is the "make government super-efficient because, of course, we can be certain it is always doing what we want" argument. The alternative is "always make sure there's some sand in the gears of government because when it is not doing what we want things can go really badly, so let's limit the velocity with which that can happen".
I actually wonder whether information asymmetries will even be sustainable in the long run, or whether we'll eventually approach a world where access to information is relatively equalized (that will all depend on how centralized things become). If things do become fairly equalized, the issue may shift from information asymmetries and how to control them to attention asymmetries. I argued something like that here.
I think you hit on this some by shifting the argument from "what can be known?" to "what should be done with what is known?" But I think the ne...
I wonder if there are technological shields that can be developed - using intelligence - to protect / shield privacy. Similar to VPNs. Like suggest parts of my face I could cover to hide my heart rate or insert noise into my call or browsing history.
This read to me as a bit inevitabilist. Putting aside for a moment that it might be hard to avoid: do we actually want to live in a world in which anyone can easily find out anything about anyone else?
Appreciate the concise but insightful post. This issue strikes me as one of the most important mundane impacts to get ahead of.
Beyond law enforcement implications, it feels like public data dumps of information (biometric, geolocation, text comms) will take google-stalking to new heights.
Expansion of panopticon capabilities seems inevitable... do you see any other plausible levers (along the lines of Overseeing the Overseers you mentioned) that keep implementation within limits acceptable to today's public?
Perhaps most importantly, it might help you to start behaving as if you're being watched and things about you are more obvious than they once were.
It's retroactively true too: anything you've ever been caught doing on camera, and any post you've ever made to a forum (under any pseudonym), will soon be linked to your identity by any entity with access and capacity. Behaving more circumspectly from now on won't protect against that, unfortunately.
and I strongly suspect that concerns about terrorism, great power conflict, and small-scale bad actors will be serious enough that these highest priority uses will not be foregone
Can you say more about why you strongly suspect this?
This post is deeply discomforting for me, but I am worried that it will be the future regardless.
I must say I disagree on your issue with the European approach. I do think that the current ignorance approach to preventing racial discrimination is the best one. Unless that sentence is completely unrelated to the previous one, and I completely missed your point. (this is highly probable, so if it is the case, please do explain what part of EU privacy law you are concerned about, and ignore the rest of this comment).
Doing option 1 relies on trusting people to...
Been thinking about writing an article like this for a decade or so, especially linking it to the sharing of information between institutions such that access to (usual) public goods that people often use to function (Gmail Accounts, Apple Accounts, X accounts, any account or sign-in where the company can decide to ban you for any reason) ever since I heard about people potentially not being let into a bar because they behaved poorly at a different bar. This combined with cancel-like culture is going to make the future a very unfun place to live until the ...
More broadly, it seems to me like there are three options for how we can react to widespread knowledge of things that were previously hidden
In some cases, there will be stalking.
Blackmail and other kind of fraud are also easy when gathering more knowledge about the target becomes easier.
In the Chinese system, there's also Social Credit Scores. The surveillance information gets used to grade your behavior and you get benefits accordingly, without really knowing what specific behavior caused the benefits.
Then once the camel's nose is in the tent the rest of the camel will follow.
I don't get this part.
Why would the acceleration of true information accessibility in one domain, eliminate legal/cultural barriers to it in another?
It's definitely true that AI can physically enable both of the following,
- Improved stylometric analysis of civilians, for people who are more likely to have ties to Hezbollah
- Maximally accurate reconstruction of unclothed 3d models of conventionally attractive public figures
Yet, these two examples feel worlds apart, to me, in their likel...
We spend enormous energy protecting bank statements at some level, treating them as surveillance risks. Yet people voluntarily share with AI systems something far more revealing: how they actually think, what they believe, what they are uncertain about, etc. A bank statement can tell you someone eats at the same place twice a week. But a long AI conversation tells you how their thinking drifts, what they are afraid of, what kinds of arguments move them, and thoughts they would never say out loud. I think the most sensitive data is no longer behavioural.
which will turn out to be much more prevalent than it seems
Duncan Sabien's recent post on "social dark matter" is relevant, and a good read. A decrease in privacy will, by Sabien's principles, most likely lead to a societal reckoning with quite a lot of social dark matter. This paints a rather hopeful picture of growing intelligence ultimately leading to growing acceptance, but the road there will probably be rather rocky.
they can (and sometimes are!) used
I would correct this to "they can be (and sometimes are!) used." I imagine this is a typo.
"start behaving as if you're being watched" -- I'm not sure what to make of the fact that the chillingness of these words isn't overwhelmingly self-evident. It is much easier for a totalitarian government to pick off a few isolated dissenters than a group that was able to gather and organize "in secret" at least up until they reached a viable size.
The worst case scenario, of course, is a system enforcing an absolute lack of privacy, when that system is owned by an enforcement agency which is not dedicated to any true prosocial goals, but to the continuation of its own power structure. And that's the end state of all surveillance and power structures because that's where the motivation and feedback reward loops trend. Anything which does NOT enforce its own monopoly on power is overtaken by things which DO enforce their own monopoly on power. The has nothing to do with the advancement of humanity or ...
The future is going to be different from the present. Let's think about how.
Specifically, our expectations about what's reasonable are downstream of our past experiences, and those experiences were downstream of our options (and the options other people in our society had). As those options change, so too our experiences, and so too our expectations of what's reasonable. I once thought it was reasonable to pick up the phone and call someone when I wanted to talk to them, and to pick up my phone when it rang; things have changed, and someone thinking about what's possible could have seen the dilution of that signal into noise coming. So let's try to see more things coming, and maybe that will give us the ability to choose what it will actually look like.
I think lots of people's intuitions and expectations about "privacy" will be violated, as technology develops, and we should try to figure out a good spot to land. This line of thinking was prompted by one of Anthropic's 'red lines' that they declined to cross, which got the Department of War mad at them; the idea of "no domestic bulk surveillance." I want to investigate that in a roundabout way, first stepping back and asking what is even possible to expect, here.
"any legal use"
Widespread access to intelligence will change privacy expectations dramatically, by allowing for 1) much more recording of information, 2) much more processing of recorded information, and 3) much more sophisticated interpretation of that information.
In American contexts, law enforcement officers have access to a wide range of information about people, but require permission to look at it (a ‘warrant’). If you’re a person of interest in a crime, they can look at your cell phone records to get evidence about whether or not you were involved in the crime, but otherwise you’re protected by the 4th Amendment from unreasonable searches and seizures. But what determines what is reasonable?
Some considerations:[1]
This has already been changed by technology becoming cheaper. It would be prohibitively expensive to have police doing stakeouts on every corner; it is not prohibitively expensive for every shopkeeper to have a CCTV system recording the street outside of their shop, and those costs continue to decline. Put together a network of those, and now a city can be under near-complete surveillance.[2]
AI continues these trends. If LLMs can review cell phone records for pennies on the dollar, it might make sense to look at a hundred times as many records. And now rather than having to have a person go camera-to-camera and track the movements of an individual thru the city, you can have a software system using facial recognition and gait analysis and spatial modeling to track whole crowds at a feasible cost.
So as a technical matter, it is already possible or it's not very far from interested parties being able to track your location at any time, if you have your phone on you or you're in a car or you're inside a city, in a 'bulk' rather than targeted way. As a social matter, this might seem pretty impolite--or it might be part of a trade most people are willing to make.
In particular, one other way that technology changes the dynamics is by making it easier for attackers to do significant amounts of damage, which raises the value of surveillance, and of predicting and catching crimes before they happen rather than investigating and punishing them after the fact.
Let's return to the interpretation of information, and look at some subtler ways that increased intelligence will change the dynamics. Many things can be inferred from 'public records' in nonobvious ways. For example, if your camera is fast enough and sensitive enough, you can measure someone's heartbeat just by watching the subtle blush-and-pale cycle of the blood in their face; standard cameras are good enough for this, and changes in heartbeat are informative about thoughts, along with other subtle changes in facial appearance.
But I'm going to talk about gaydar, because it ties back into the broader social questions of where we want society to end up. Sometimes, people can guess the sexual orientation of another person just by observing them, using both deliberate and accidental features. Gay men sometimes benefit from looking gay ("the earring in his right ear suggests--" or "his haircut implies--"), and also they have various developmental differences that can manifest in appearance. In 2018, Wang and Kosinki trained a neural network to do it off of dating photos and it substantially outperformed humans (80% success rate rather than 60% success rate.)[3][4]
So as we we get more widespread intelligence–as software gains capabilities that were formerly available only to human experts and use of that software becomes potentially widespread–we stop being able to hide some things. What do we want to do about that?
This doesn't include a line for when an expert observer could have guessed that I'd be gay, which is probably a decade earlier.
The world has changed a lot here, over the course of my lifetime! When I was a child, being gay was mostly hidden and navigating being gay required subtlety and discernment. Part of the response to the AIDS crisis, at the insistence of gay rights groups, was to prioritize patient confidentiality over stopping the spread.[5] But now, as an adult, being gay is mostly not hidden; you don’t have to use a profiling algorithm on my face, up until Facebook removed it in 2022 you could just go to my profile and see that I’ve checked the box for “interested in men”. (Or you could search my LessWrong comments, or–)
So from the perspective of the 2020s in America, it feels actually pretty benign. (But that's not universal; the situation is both worse in other countries, and the prospect of 'transvestigating models' feels much less benign.)
More broadly, it seems to me like there are three options for how we can react to widespread knowledge of things that were previously hidden:
I think there's situations where each of the three is the most appropriate option. In particular, I think the situation for acceptance of sexual minorities has been on this positive trendline in part because of increased knowledge and decreased privacy. The understanding that lots of gay people were 'normal' did a lot of normalize being gay!
I also think it's easy to find situations where purging or filtering are quite sympathetic. I in fact would strongly support my local subway system tracking who the most anti-social riders are and banning them, so that the system is cleaner and safer, and if it is cheaper and better to do so with facial recognition technology or similar 'totalitarian surveillance measures' , that seems probably worthwhile. (Similarly, it's very nice to not be murdered in a terrorist attack.) But it's also easy to see how such technologies can be deployed for undesirable ends; if border agents look thru someone's phone to try and determine if they're a member of a terrorist network, they can also determine whether they've made social media comments that are critical of Trump. A current controversy is how much local cities should sign on to surveillance networks like Flock; when many jurisdictions have committed to not cooperate with federal law enforcement on immigration enforcement, signing on with a contractor which does cooperate with federal law enforcement runs counter to those commitments.
As a fan of the truth and a believer in its efficacy, I am most biased against the third option. Yet many widely and strongly cherished parts of our society rest on it! Anti-discrimination laws that bar decision-making based on race are an example that it seems unwise to recklessly drop, and yet race is often quite easy for people or systems to infer.[6] European regulations about privacy seem to me to mostly be insisting that technology develop in this direction.
Yet my overall sense is that we cannot stick our heads in the sand for long. The highest priority uses will drive adoption of the mass surveillance technology, and I strongly suspect that concerns about terrorism, great power conflict, and small-scale bad actors will be serious enough that these highest priority uses will not be foregone. Then once the camel's nose is in the tent the rest of the camel will follow. The best way out is to fix our goals and preferences:
Rather than hoping that local entities can prevent the enforcement of immigration rules that are clearly not in their interest, develop new immigration rules that cities across America would be comfortable cooperating with, and then use the advanced technology to do so cheaply.[7]
Develop watchmen that watch the watchmen, such that people with access to bulk surveillance systems and are using them in corrupt ways are themselves found and punished.
Most controversially, become comfortable with benign deviancy (of the sort which will turn out to be much more prevalent than it seems) and align criminal standards with actual behavior (a world where the true speed limit is 20mph higher than the posted one will not be well-served by ubiquitous vehicle tracking).
Perhaps most importantly, it might help you to start behaving as if you're being watched and things about you are more obvious than they once were.
I should note that I'm thinking like an economist or systems designer, not a lawyer. There must be extensive case law on what people currently think is reasonable or unreasonable, but that's only relevant for reasons of continuity. We’re imagining the future, of what things will look like after people have adapted to their new situation, which plausibly involves major changes to the underlying laws.
Consider also the situation with cashless toll systems like EZ Pass, which have long worried privacy advocates, as they can (and sometimes are!) used to track where people travel. There's nothing fundamentally rights-violating here, tho; this could be replicated by anyone with enough eyes in enough places. (We already consent to each car having a unique identifier to make tracking easier!)
“Wait,” you might say, “60% success rate for a trait with a baseline prevalence of less than 40%? How did the human guessers do worse than always guessing ‘straight’?” In their dataset, they equalized the number of homosexual and heterosexual faces, so pure-chance guessing would have scored 50%. This is still an artificial situation (they're dating site photos, not street photos) which doesn’t take into account base rates.
A friend points out that Wang & Kosinski generated a bunch of critical responses, like this one which claims that the effect is primarily due to styling choices, rather than underlying biological features. How much of the effect is deliberate vs. accidental determines how much choice one has in whether to hide the feature, but in the current equilibrium people can "hide in plain sight", where a member of the public might just think "oh, a shark plushie", whereas a fellow transgirl will see a Blåhaj and view it as an identity marker. With more widespread intelligence, more people can more cheaply become 'in the know', and so this channel will become costlier (and, to the extent the predictive features aren't styling choices, may be impossible to hide).
As someone interested in public health, this horrifies me, but I acknowledge the ways I am a sweet summer child who grew up with the internet and the dramatic upswing in acceptance and downswing in intolerance, and decision-makers in the 80s had very different life experiences and expectations.
When I was a data scientist, I ended up looking into the contours of compliance here. Many things that aren't race are nevertheless informative about race, and so you can construct a composite out of information which individually is legal or ethical to use, which is just a proxy for race, and thus the composite is illegal or unethical to use, and so people developed statistical techniques to try to make sure this isn't happening, and that the composite is composed just of 'legitimate' influence. This involves being deliberately and willfully blind to facts about the world to achieve some social end, but that's what polite ignorance is!
There are too many horror stories of ICE misbehavior, detaining American citizens and racially profiling people on the street. Would the situation be improved or made worse by a national facial recognition database? On the one hand, Flock and similar systems would allow ICE to notice whenever someone without legal residency went out in public; on the other hand, there would be no excuse for not immediately checking the database and releasing legal residents. I think immigration reform means we can get the benefit of the latter without having to pay the costs of the former; of course, immigration reform is its own problem that deserves its own post.