Intuitive and non-intuitive findings of systems theory, like what adding a second queue does or increasing the variability of a flow rate or  changing the size of a buffer.

• Being presented a plan/system/technology/mathematical proof and trying to find reasons why it doesn't/won't work
• Conducting simple cost-benefit analyses (mostly using the four basic arithmetic operations + dimensional analysis), with a time-limit of 5/30/60 minutes
  • This includes knowledge about expected value (and it's possible shortcomings)
  • This maybe even includes knowledge about the expected value of perfect information
• Performing Monte-Carlo calculations using something like Guesstimate, ideally on the same problem as a previous cost-benefit analysis.
• Calibration training
  • Over unknown variables, such as "Which person was born earlier", or "Which country has a bigger population"
  • By forecasting events on different timescales, ranging from a day to a month (maybe more (a decade would be great), but the exam format probably doesn't allow that)

• Reading and writing Dath Illan's spoken language. 
• Arithmetic.
• Individual and collective decision making and execution
  • How to make a good decision.
    • Individually
    • As part of a collective
  • How to recognize a good decision made by others
  • How to execute a good decision when you or others have made it - especially in the face of akrasia.

Given the following charts, statistics, and arguments based on those charts and statistics, point out the important flaws in the arguments and state what unjustified conclusions the arguers are trying to cause you to reach.

Eliezer wrote:

Eliezer Yudkowsky: That covers a lot of territory! But if I tried to zoom in at the most general level of difference, there's very much an understanding among Very Serious People that if you require anything to get access to a job (like a credential) that is difficult to get or has finite supply, people can burn the whole surplus value of the job to them in order to get it, even if that thing is of lower value to the employer. In other words, they would recognize "occupational licensing" or college as a sickness and move to prevent it while it was still getting started. In general, demanding something from somebody other than their actual job skill is recognized as a potential civilizational problem. 

Topics are selected based on cost/benefit; something that takes a long time to learn would need to be a lot more useful, or have major positive externalities to more people knowing it.

There's more that's going into a good test. A good test is not about whether a person learned a topic but about whether the person is likely perform well in whatever they want to do.

Dath Ilan is going to have many different tests for different jobs. There will be regular evaluations of how the test predicts performance in the job and the test will be adapted to optimize for predicting performance. 

Instead of picking topic by putting in "a lot of thought" the main way will be picking topics empirically by looking at whether the knowledge predicts performance.

The only thing that does need thought is thinking about how to prevent goodharting.

What topics belong on it?

That completely depends on the job. It doesn't make sense to have a uniform test that everyone is taking given that as a society you want diversity in skills in your population.

My case for trigonometry: We want to people understand social cycles. For example, heroin becomes fashionable among young people because it feels good. Time goes by and problems emerge with tolerance, addiction, and overdose. The next cohort  of young people see what happened to aunts and uncles etc, and give heroin a miss. The cohort after that see their aunts and uncles living clean lives, lives that give no warning. They experiment and find that heroin feels good. The cycle repeats.

These cycles can arise because the fixed points of the dynamics are unstable. The classic simple example uses a second order linear differential equation as a model with a solution such as $e^{at} \sin kt$. We really want people to have some sense of cycles arising from instabilities without anyone driving them. We probably cannot give simple examples of what we mean with trigonometric functions.

Well, before we can discuss the "answers" the first step is to come up with an algorithm that measures how much a given piece of civics knowledge pays rent.  Is it useful to know the nominal structure of the government (vs the 'actual' structure in practice)?  Is it useful to know the retold tales of George Washington, victorious general and American hero?  (not to mention the high probability of errors in the 'knowledge' given the time that has passed and the likely bias)

A reasonable argument could be made that in our form of democracy, civics knowledge is of little use to the average citizen.  This is because that each of us has such an infinitesimal 'vote', and each person well educated in civics has their vote drowned out.  If 1 in 1000 citizens are genuinely well educated in civics, and nearly all elections are decided by a margin greater than 0.1% (or this is below the noise floor for the voting machinery itself...), civics knowledge is useless.

If this hypothesis is true (not saying it is or isn't), the culprit would be a failure of colleges and others to be accountable to a measurable cost/benefit ratio for the things that they teach.  Instead of using measurable metrics they use arguments like "this knowledge is clearly worth it because of what it is" or "tradition says we have to keep teaching it".

This is one reason why we all had to waste some of our lifespan on trigonometry and literature instead of say learning to use Python more effectively.  Arguably knowing how to tell a machine to do your math for you effectively is thousands of times more valuable than useless derivations you won't need to know unless you become a mathematician.

Suppose it were possible to write down a list of every fact or algorithm known by a living human being.  This isn't impossible, if you could use an AI system to translate audio recordings of someone's entire life to text, and everything they ever read, you would have it in a file.  Then you would map from [text] to [common fact or algorithm] by comparing thousands of these files and to fact or algorithm written in sources like encyclopedias.  Or more likely you would find commonality with a clustering algorithm.

The "knowledge that pays the most rent" is either the most common fact or algorithm known by all humans, or the most common one that separates successful humans from failures.  (if there is a measurable difference)  

Then an education system adds the most value by teaching the or algorithms in order of greatest value to least value, or reordering in complexity tiers, and in each tier teaching the most value elements first until the time allotted for education is over.  

If civics has any value at all, this algorithm would find it.  (though as described it is subject agnostic)