B.Eng (Mechatronics)
O1 now passes the simpler "over yellow" test from the above. Still fails the picture book example though.
For a complex mechanical drawing, O1 was able to work out easier dimensions but anything more complicated tends to fail. Perhaps the full O3 will do better given ARC-AGI benchmark performance.
Meanwhile, Claude 3.5 and 4o fail a bit more badly failing to correctly identify axial and diameter dimensions.
Visuospatial performance is improving albeit slowly.
My hope is that the minimum viable pivotal act requires only near human AGI. For example, hack competitor training/inference clusters to fake an AI winter.
Aligning +2SD human equivalent AGI seems more tractable than straight up FOOMing to ASI safely.
One lab does it to buy time for actual safety work.
Unless things slow down massively we probably die. An international agreement would be better but seems unlikely.
This post raises a large number of engineering challenges. Some of those engineering challenges rely on other assumptions being made. For example, the use of energy carrying molecules rather than electricity or mechanical power which can cross vacuum boundaries easily. Overall a lot of "If we solve X via method Y (which is the only way to do it) problem Z occurs" without considering making several changes at once that synergistically avoid multiple problems.
"Too much energy" means too much to be competitive with normal biological processes.
That goalpost should be right at the top and clearly stated instead of "microscopic machines that [are] superior". "grey goo alone will have doubling times slower than optimised biological systems" is definitely plausible. E-coli can double in 20 minutes in nutrient rich conditions which is hard to beat. If wet nanotech doubles faster but dry nanotech can make stuff biology can't, then use both. Dry for critical process steps and making high value products and wet for eating the biosphere and scaling up.
Newer semiconductor manufacturing processes use more energy and materials to create each transistor but those transistors use less power and run faster which makes producing them worthwhile. Dry nanotech will be a tool for making things that may be expensive but worthwhile to build like really awesome computers.
Wet nanotech (IE:biology) is plausibly the most efficient at self-replicating but notice humans use all sorts of chemical and physical processes to do other things better. Operating in space with biotech alone for example would be quite difficult.
Your image links are all of the form: http://localhost:8000/out/planecrash/assets/Screenshot 2024-12-27 at 00.31.42.png
Whatever process is generating the markdown for this, well those links can't possibly work.
I got this one wrong too. Ignoring negative roots is pretty common for non-mathematicians.
I'm half convinced that most of the lesswrong commenters wouldn't pass as AGI if uploaded.
This post is important to setting a lower bound on AI capabilities required for an AI takeover or pivotal act. Biology as an existence proof that some kind of "goo" scenario is possible. It somewhat lowers the bar compared to Yudkowsky's dry nanotech scenario but still requires AI to practically build an entire scientific/engineering discipline from scratch. Many will find this implausible.
Digital tyranny is a better capabilities lower bound for a pivotal act or AI takeover strategy. It wasn't nominated though which is a shame.
This is why I disagree with a lot of people who imagine an “AI transformation” in the economic productivity sense happening instantaneously once the models are sufficiently advanced.
For AI to make really serious economic impact, after we’ve exploited the low-hanging fruit around public Internet data, it needs to start learning from business data and making substantial improvements in the productivity of large companies.
Definitely agree that private business data could advance capabilities if it were made available/accessible. Unsupervised Learning over all private CAD/CAM data would massively improve visuo-spatial reasoning which current models are bad at. Real problems to solve would be similarly useful as ground truth for reinforcement learning. Not having that will slow things down.
Once long(er) time horizon tasks can be solved though I expect rapid capabilities improvement. Likely a tipping point where AIs become able to do self-directed learning.
Hard drives are a good illustrative example. Here's a hardware hacker reverse engineering and messing with the firmware to do something cool.
There is ... so much hardware out there that can be bought cheaply and then connected to with basic soldering skills. In some cases, if soft-unbricking is possible, just buy and connect to ethernet/usb/power.
There's a long tail (as measured by commercial value) of real world problems that are more accessible. On one end you have the subject of your article, software/devices/data at big companies. On the other, obsolete hardware whose mastery has zero value, like old hard disks. The distribution is somewhat continuous. Transaction costs for very low value stuff will set a floor on commercial viability but $1K+ opportunities are everywhere in my experience.
Not all companies will be as paranoid/obstructive. A small business will be happy using AI to write interface software for some piece of equipment to skip the usual pencil/paper --> excel-spreadsheet step. Many OEMs charge ridiculous prices for basic functionality and nickel and dime you for small bits of functionality since only their proprietary software can interface with their hardware. Reverse engineering software/firmware/hardware can be worth thousands of dollars. So much of it is terrible. AI competent at software/firmware/communication reverse engineering could unlock a lot of value from existing industrial equipment. OEMs can and are building new equipment to make this harder but industrial equipment already sold to customers isn't so hardened.
IOT and home automation is another big pool of solvable problems. There's some overlap between home automation and industrial automation. Industrial firmware/software complexity is often higher, but AI that learns how to reverse engineer IOT wireless microcontroller firmware could probably do the same for a PLC. Controlling a lighbulb is certainly easier than controlling a CNC lathe but similar software reverse engineering principles apply and the underlying plumbing is often similar.
Alternate POV
Science fiction. < 10,000 words. A commissioned re-write of Eliezer Yudkowsky's That Alien Message https://alicorn.elcenia.com/stories/starwink.shtml
since it hasn't been linked so far and doesn't seem to be linked from the original
TLDR:autofac requires solving "make (almost) arbitrary metal parts" problem but that won't close the loop. Hard problem is building automated/robust re-implementation of some of the economy requiring engineering effort not trial and error. Bottleneck is that including for autofac. Need STEM AI (Engineering AI mostly). Once that happens, economy gets taken over and grows rapidly as things start to actually work.
To expand on that:
"make (almost) arbitrary metal parts"
Closing the loop needs much more than just "make (almost) arbitrary metal parts". "build a steel mill and wire drawing equipment", is just the start. There are too many vitamins needed representing unimplemented processes
A minimalist industrial core needs things like:
Those in turn imply other things like:
Things just snowball from there.
Efficiency improvements like carbide+coatings for cutting tools are also economically justified.
All of this is possible to design/build into an even bigger self-reproducing automated system but requires more engineer-hours put into a truly enormous git repo.
STEM AI development ("E" emphasis) is the enabler.
Simplifications can be made to cut down on vitamin cost of machine tools. Hydraulics really helps IMO:
End result: machine tools with big spindle motors and small control motors for all axes. Robots use rotary equivalent. Massive reduction in per-axis power electronics, no ballscrews, no robot joint gears.
For Linear/rotary position encoders, calibrated capacitive encoders (same as used in digital calipers) are simple and needs just PCB manufacturing. Optical barcode based systems are also attractive but require an optical mouse worth of electronics/optics per axis, and maybe glass optics too.
None of the labs would be doing undirected drift. That wouldn't yield improvement for exactly the reasons you suggest.
In the absence of a ground truth quality/correctness signal, optimizing for coherence works. This can give prettier answers (in the way that averaged faces are prettier) but this is limited. The inference time scaling equivalent would be a branching sampling approach that searches for especially preferred token sequences rather than the current greedy sampling approach. Optimising for idea level coherence can improve model thinking to some extent.
For improving raw intelligence significantly, ground truth is necessary. That's available in STEM domains, computer programming tasks being the most accessible. One can imagine grounding hard engineering the same way with a good mechanical/electrical simulation package. TLDR:train for test-time performance.
Then just cross your fingers and hope for transfer learning into softer domains.
For softer domains, ground truth is still accessible via tests on humans (EG:optimise for user approval). This will eventually yield super-persuaders that get thumbs up from users. Persuasion performance is trainable but maybe not a wise thing to train for.
As to actually improving some soft domain skill like "write better english prose" that's not easy to optimise directly as you've observed.