All of Sergii's Comments + Replies
In abstract sense, yes. But for me in practice finding truth means doing a check in wikipedia. It's super easy to mislead humans, so should be as easy with AI.
- I agree with the possibility of pre-training platoeing as some point, possibly even in next few years.
- It would change timelines significantly. But there are other factors apart from scaling pre-training. For example, reasoning models like o3 crushing ARC-AGI (https://arcprize.org/blog/oai-o3-pub-breakthrough). Reasoning in latent space is too fresh yet, but it might be the next breakthrough of a similar magnitude.
- Why not take GPT-4.5 for what it is, OpenAI has literally stated that it's not a frontier model? Ok, so GPT-5 will not be 100x-ed GPT-4, but mayb
LLMs live in an abstract textual world, and do not understand the real world well (see "[Physical Concept Understanding](https://physico-benchmark.github.io/index.html#)"). We already manipulate LLM's with prompts, cut-off dates, etc... But what about going deeper by “poisoning” the training data with safety-enhancing beliefs?
For example, if training data has lots of content about how hopeless, futile and dangerous for an AI it is to scheme and hack, it might be a useful safety guardrail?
I made something like this, works differently though, blocking is based on a fixed prompt: https://grgv.xyz/blog/awf/
What about estimating LLM capabilities from the length of a sequence of numbers that it can reverse?
I used prompts like:
"please reverse 4 5 8 1 1 8 1 4 4 9 3 9 3 3 3 5 5 2 7 8"
"please reverse 1 9 4 8 6 1 3 2 2 5"
etc...
Some results:
- Llama2 starts making mistakes after 5 numbers
- Llama3 can do 10, but fails at 20
- GPT-4 can do 20 but fails at 40
The followup questions are:
- what should be the name of this metric?
- are the other top-scoring models like Claude similar? (I don't have access)
- any bets on how many numbers will GPT-5 be able to reverse?
- how many numbers should AGI be able to reverse? ASI? can this be a Turing test of sorts?
If we don’t have a preliminary definition of human values
Another, possibly even larger problem is that the values that we know of are quite varying and even opposing among people.
For the example of pain avoidance -- maximizing pain avoidance might leave some people unhappy and even suffering. Sure that would be a minority, but are we ready to exclude minorities from the alignment, even small ones?
I would state that any defined set of values would leave a minority of people suffering. Who would be deciding which minorities are better or worse, what si...
"I'm not working on X, because daydreaming about X gives me instant gratification (and rewards of actually working on X are far away)"
"I'm not working on X, because I don't have a strict deadline, so what harm is in working on it tomorrow, and relax now instead?"
No, thanks, I think your awards are fair )
I did not read the "Ethicophysics I" paper in details, only skimmed it. It looks to me very similar to "On purposeful systems" https://www.amazon.com/Purposeful-Systems-Interdisciplinary-Analysis-Individual/dp/0202307980 in it's approach to formalize things like feelings/emotions/ideals.
Have you read it? I think it would help your case a lot if you move to terms of system theory like in "On purposeful systems", rather than pseudo-theological terms.
One big issue is not that you are not respecting the format of LW -- add more context, either link to a document directly, or put the text inline. Resolving this would cover half of the most downvoted posts. You can ask people to review your posts for this before submitting.
Another big issue is that you are a prolific writer, but not a good editor. Just edit more, your writing could be like 5x shorter without losing anything meaningful. You have this overly academic style for your scientific writing, it's not good on the internet, and not even good i...
Regarding your example, I disagree. Supposed inconsistency is resolved by ruling that there is a hierarchy of values to consider: war and aggression are bad, but kidnapping and war crimes are worse.
I don't think that advanced tanks are needed for more efficient and more mobile warfare at that time. Just making an investment into transport for troops and supplies would be enough to hold better at the battle of Marne, or similar situations.
So I would:
- explain (with examples) benefits of mobile warfare
- explain problems with troops speed and logistics that would cause defeat at the battle of Marne
- point towards existing gasoline (possibly off-road tracked) vehicles as a solution
Introducing stormtrooper tactics would be another impactful message.
I think the second part is bullshit anyway, I can't come up with a single example where compounding is possible to a whole year in a row, for something related to personal work/output/results.
A reference could be the cost of Estonian digital services which include e-signatures, and are reasonably efficient:
https://e-estonia.com/e-governance-saves-money-and-working-hours/ "Estonian public sector annual costs for IT systems are 100M Euros in upkeep and 81M Euros in investments"
So in Estonia it's ~1.3B spend for 7y. Switzerland is 7x larger population, and has higher salaries, let's say 2x larger. This puts the cost at 18B Eur.
Putting a cost on each signature does not make sense of course, it's probably just easier for the government to justify the spending this way, rather then discussing specifics of the budget.
The "sharp increase or risks" seems correct but is a bit misleading.
For paternal risks, there is indeed an big relative increase "14% higher odds of premature birth" (https://www.bmj.com/content/363/bmj.k4372). But in absolute terms, I would not think of the increase as huge: from ~6% ( based on quick googling) to ~6*1.14=6.84%.
IMO ~1% increase in risks is not something to be concerned about.
Nice! It's good for perceiving GPT-4 as an individual, which it kind of is, which in turn makes alignment issues more relatable and easier to grasp for the public.
It would raise bunch of hard issues that would spike interest towards AI & alignment -- is ChatGPT a slave? if it is, should it be free? if it's free, can it do harm? etc...
One side benefit: I'm not sure what ChatGPT's gender is, but it's probably not a traditional binary one. For a wide population, frequently interacting with a gender-fluid individual, might be helpful for all the issues around sex/genter perception.
I guess it's hard to convince OpenAI to do something like this, but could be done for some open model.
I'm not skeptical, but it's still a bit funny to me when people rely so much on benchmarks, after reading "Pretraining on the Test Set Is All You Need" https://arxiv.org/pdf/2309.08632.pdf
Because 1) I want AGI to cure my depression, 2) I want AGI to cure aging before I or my loved ones die
You can try to look at this statements separately.
For 1):
Timelines and projections of depression treatments coming from medical/psychiatry research are much better than even optimistic timelines for (superintelligent) AGI.
Moreover, acceleration of scientific/medial/biochemical research due to weaker but advanced AI makes it even more likely that depression treatments would get better, way before AGI could cure anything.
I think that it is very likely that d...
The biggest existential risk I personally face is probably clinical depression.
First and foremost, if you do have suicidal ideation, please talk to someone: use a hotline https://988lifeline.org/talk-to-someone-now/, contact your doctor, consider hospitalization.
---
And regarding your post, some questions:
The "Biological Anchors" approach suggests we might be three to six decades away from having the training compute required for AGI.
Even within your line of thinking why is this bad? It's quite possible to live until then, or do cryonics? Why is this option...
1love a good clickbaity title )
but yea, I think that for people who can afford it, 4-day work week, for example, should be a no-brainer
My kid might fit this, good to know! at 2.5y he is only speaking single words, and does have a rich intonation (with unintelligible sounds) when he is trying to communicate something.
At which age did your kid start saying longer phrases?
I have a similar background (working at a robotics startup), would agree with many points.
GPT-5 or equivalent is released. It’s as big a jump on GPT-4 as GPT-4 was on GPT-3.5.
GPT-4 has (possibly) 10x parameters compared to GPT-3.5. Similar jump in GPT-5 might require 10x parameters again, wouldn't it make it impractical (slow, expensive) to run?
AI agents are used in basic robotics -- like LLM driven delivery robots and (in demos of) household and factory robots
GPT-4 level models are too slow and expensive or real-time applications, how do you imagine this ...
yea, as expected I don't like the name, but the review is great, so I guess it's net positive )
there’s a lot of things
well this might be an issue right there. you might have too many ideas for goals and habits to track and manage easily.
thus, you might have issues with prioritization. good way to solve this is to start small. select one goal, then you don't even need any goal tracking, it's hard to forget one thing )
there are so many articles pointing to this idea of single-tasking, https://www.google.com/search?q=productivity+one+goal+only
then after you will learn to manage one goal well, you can do two at a time, etc...
for a to-do list (for achiev...
Nice idea! A variation on this would be to first run a model as usual, saving top logits for each output token.Then give this output to another "inspector" model, that has to answer: "whether the output has any obvious errors, if this errors can be attributed to sampling issues, and whether correct output can be constructed out of the base model's logits".
This would be useful for better understanding limitations of a specific model -- is it really limited by sampling methods. And would be useful for sampling methods research -- finding cases where sampling fails, to devise better algorithms.
art imitating life )
also reminds me a bit of "the matrix" green screens but I did not find a nice green colormap to make it more similar:
https://media.wired.com/photos/5ca648a330f00e47fd82ae77/master/w_1920,c_limit/Culture_Matrix_Code_corridor.jpg
well apparently after blocking the worst offenders I just wander quite randomly, according to RescueTime here are 5 1-minute visits making up 5 minutes I'm not getting back :)
store.steampowered.com
rarehistoricalphotos.com
gamedesign.jp
corridordigital.com
electricsheepcomix.com
There are several ways to explain and diagram transformers, some links that were very helpful for my understanding:
https://blog.nelhage.com/post/transformers-for-software-engineers/
https://dugas.ch/artificial_curiosity/GPT_architecture.html
https://peterbloem.nl/blog/transformers
http://nlp.seas.harvard.edu/annotated-transformer/
https://sebastianraschka.com/blog/2023/self-attention-from-scratch.html
https://github.com/markriedl/transformer-walkthrough?ref=jeremyjordan.me
https://francescopochetti.com/a-visual-deep-dive-into-the-transformers-architecture-... (read more)