Most of these kinds of posts should start with Woody Allen's 1979 quote:

More than any other time in history, mankind faces a crossroads. One path leads to despair and utter hopelessness. The other, to total extinction. Let us pray we have the wisdom to choose correctly.

Agreed, but it's not just software.  It's every complex system, anything which requires detailed coordination of more than a few dozen humans and has efficiency pressure put upon it.  Software is the clearest example, because there's so much of it and it feels like it should be easy.

I think this leans a lot on "get evidence uniformly over the next 10 years" and "Brownian motion in 1% steps".  By conservation of expected evidence, I can't predict the mean direction of future evidence, but I can have some probabilities over distributions which add up to 0.  

For long-term aggregate predictions of event-or-not (those which will be resolved at least a few years away, with many causal paths possible), the most likely updates are a steady reduction as the resolution date gets closer, AND random fairly large positive updates as we learn of things which make the event more likely.

I kind of see what you're saying, but I also rather think you're talking about specifying very different things in a way that I don't think is required.  The closer CS definition of math's "define a sorted list" is "determine if a list is sorted".  I'd argue it's very close to equivalent to the math formality of whether a list is sorted.  You can argue about the complexity behind the abstraction (Math's foundations on set theory and symbols vs CS library and silicon foundations on memory storage and "list" indexing), but I don't think that's the point you're making.

When used for different things, they're very different in complexity.  When used for the same things, they can be pretty similar.

It's fascinating (and a little disturbing and kind of unhelpful in understanding) how much steering and context adjustment that's very difficult in older/smaller/weaker LLMs becomes irrelevant in bigger/newer ones.  Here's ChatGPT4:

You

Please just give 100 digits of e * sqrt(3)

ChatGPT

Sure, here you go:

8.2761913499119 7879730592420 6406252514600 7593422317117 2432426801966 6316550192623 9564252000874 9569403709858

"Mathematical descriptions" is a little ambiguous.  Equations and models are terse.  The mapping of such equations to human-level system expectations (anticipated conditional experiences) can require quite a bit of verbosity.  

I think that's what you're saying with the "algorithms and data structures" part, but I'm unsure if you're claiming that the property specification of the math is sufficient as a description, and comparable in fidelity to the algorithmic implementation.

[note: I suspect we mostly agree on the impropriety of open selling and dissemination of this data.  This is a narrow objection to the IMO hyperbolic focus on government assault risks. ]

I'm unhappy with the phrasing of "targeted by the Chinese government", which IMO implies violence or other real-world interventions when the major threats are "adversary use of AI-enabled capabilities in disinformation and influence operations." Thanks for mentioning blackmail - that IS a risk I put in the first category, and presumably becomes more possible with phone location data.  I don't know how much it matters, but there is probably a margin where it does.

I don't disagree that this purchasable data makes advertising much more effective (in fact, I worked at a company based on this for some time).  I only mean to say that "targeting" in the sense of disinformation campaigns is a very different level of threat from "targeting" of individuals for government ops.

I don't have confidence in my models of how coherent and competent governments are at getting and using data like this.  The primary buyers of location data are advertisers and business planners looking for statistical correlations for targeting and decisions.  This is creepy, but not directly comparable to "targeted by the Chinese government".

My competing theories of "targeted by the Chinese government" threats are:

1. they're hyper-competent and have employee/agents at most carriers who will exfiltrate needed data, so stopping the explicit sale just means it's less visible.
2. they're as bureaucratic and confused as everything else, so even if they know where you are, they're unable to really do much with it.  

I think the tension is what does it even mean to be targeted by a government.

Moral weights depend on intensity of conscient experience.

Wow, that seems unlikely.  It seems to me that moral weights depend on emotional distance from the evaluator.  For some, they're able to map intensity of conscious experience to emotional sympathy (up to a point; there are no examples and few people who'll claim that somthing that thinks faster/deeper than them is vastly more important than them).