Quantified Self is in it's nature about dealing with epistemology. It's not certain that you will learn something about how an AGI works by doing Quantified Self but the potential is there.
A mathematical model of how human memory works that could be produced by looking at Mnemosyth data could also potentially matter for FAI.
FAI is a hard problem and therefore it's difficult to predict, where you will find solutions to it.
How easy it is to get funding for these kind of projects?
It very much depends on the project. I don't know how hard it is to get grants for the spaced repetition problems I mentioned. I however think that if someone seeks a topic for a bachelor or master thesis, they are good topics if you want an academic career.
The daily Anki score would allow other academics to do experiments of how factor X effects memory. If you provide the metric that they use in their papers they will cite yourself.
I thought that by "problems" you meant things like the millennium problems
I don't understand why anyone would want to work on the Riemann Hypothesis. It doesn't seem to be a problem that matters.
It one of those examples that suggests that people are really bad at prioritising. Mathemacians work at it because other mathematician think it's hard and solving it would impress them.
It has a bit of Terry Pratchett's Unseen University which was created to prevent powerful wizards from endangering the world by keeping them busy with academic problems. The only difference is that math might advance in a way that makes an AGI possible and is therefore not completely harmless.
I don't understand why anyone would want to work on the Riemann Hypothesis. It doesn't seem to be a problem that matters.
Could the fact that it doesn't seem to have many practical applications is what attracts certain people towards it? It doesn't have practical applications -> it's "purer" math. You're not trying to solve the problem for some external reason or using the math as a tool, you're trying to solve it for its own sake. I remember reading studies that mathematicians are on average more religious than scientists in general and I'v...
What should a not-very-smart person do? Suppose you know a not-very-smart person (around or below average intelligence). S/he read about rationality, has utilitarian inclinations, and wants to make the world better. However, s/he isn't smart enough to discover new knowledge in most fields, or contribute very much to a conversation of more knowledgeable experts on a given topic. Let's assume s/he has no exceptional talents in any area.
How do you think a person like that could best use his/her time and energy? What would you tell the person to do? This person may be, compared to average LW readership, less capable of noticing the irrationality in his/her actions even if s/he wants to be rid of it, and less easily able to notice the flaws in a bad argument. S/he may never be able to deeply understand why certain arguments are correct, certain scientific facts have to be the way they are, and telling him/her to be unsure or sure about anything seems dangerous if s/he doesn't really understand why.
My practical advice might be:
1) If you want to give to charity, follow GiveWell recommendations.
2) Learn about the basic biases, and commit to resisting them in your own life.
3) Follow advice that has been tested, that correctly predicts a positive outcome. If a hypothesis is untestable (there's an unsensible dragon in your garage) or doesn't predict anything (fire comes from phlogiston in combustable substances), or is tested and demonstrably false (god will smite you if you say it doesn't exist), don't waste time and energy on it. If you want to improve, look for tested methods that have significant positive results relevant to the area of interest. Similarly, if a person regularly gives you advice that does not lead to good outcomes, stop following it, and if someone gives advice that leads to good outcomes, start paying attention even if you like that person less.
At a more general level, my thoughts are tentative, but might include basic LW tenets such as:
1) Don't be afraid of the truth, because you're already enduring it.
2) If all the experts in a field agree on something, they might be wrong, but you are extremely unlikely to be better at uncovering the truth, so follow their advice, which might appear to conflict with...
3) Don't trust deep wisdom. Use Occam's razor, think about simple, basic reasons something might be true (this seems good for religion and moral issues, bad for scientific ideas and understanding)
4) If you find yourself flinching away from an idea, notice that, and give it extra attention.
Note: I mean this as a serious, and hopefully non-insulting question. Most people are intellectually near-average or below-average, and I have not seen extensive discussion on how to help them lead happier lives that make the world a better place.