Where does prompt optimization fit in to y’all’s workflows? I’m surprised not to see mention of it here. E.g OPRO https://arxiv.org/pdf/2309.03409 ?
slightly related https://arxiv.org/abs/2503.00735
Re: "Let's think step by step"
so let me get this straight.. a simple prompt is able to elicit an entire style of thinking, which is able to solve harder problems, and ultimately ends up motivating new classes of foundation model? Is that what happened last year? Are there any other simple prompts like that? Did we check? Sorry I'm trying to catch up.
I think part of the important part is building your own (company's) collection of examples to train against, since the foundation models are trained against swebench already. And if it works the advantage would be on my CV in the worst case but in equity appreciation in the best case. So, just like any skill, right?
You're right that the whole thing only works if the business can generate returns to high quality code, and can write specifications faster than its complement of engineers can implement them. But I've been in that position several times, it does happen. Mainly when the core functionality of the product is designed and led by domain experts who are not software engineers. Like if you make software for accountants for instance.
The reasons you give btw don't give me much consolation. The code leaking thing is very temporary; if you could host cutting edge models on AWS or Azure it wouldn't be an issue for most companies. If you could self host them it wouldn't be an issue for almost /any/ companies. The errors thing is a crux. The basic solution to that, I think, is scaling: multishot the problem, rank the solutions, test in every way imaginable, and then for each solved problem optimize your prompts till they can one-shot, keeping a backlog of examples to perform workflow regression testing against.
The style thing is very tractable, AIs love following style instructions.
The big moment for me was realizing that while each AI's context window is limited, within that window you can ask LOTS of different questions and expect a pretty good answer. So you ask questions that compress the information in the window for the purpose of your problem (llm's are pretty darn good at summarizing), and keep doing that until you have enough context to solve the problem.
mm.. I gave the wrong impression there; my actual boss doesn't have a huge opinion on AI; in fact he'll take some convincing.
I should state my assumptions:
So at some point, my employer (whoever they are at the time) will have to choose between retaining me, and paying an AI-pipeline-maintenance vendor.
Or maybe whoever I work for at the time gets outcompeted by companies that use advanced AI workflows to generate software, then I get laid off and also don't have the kind of experience necessary to work for the competitor
If you don't think my assumptions hold then you should think your career is safe. If they do hold, there's still the possibility of noticing later, and reacting by retooling to remain employable. But if you don't notice in time, there's nothing your boss (or the CTO for that matter) can do to help you. which is why I need to bulid this knowledge into my career by applying it; get it on the resume, prove the value IRL.
i dont have time to write any of this down so it's going to come out in the wrong order but here
This might be interesting to LessWrong as a personal take, because the Alignment folks are effectively on the side of Capital here. Without fast and parallel access to foundation models, I can't learn my new job, which is auto-codegen-pipeline-maintainer. If some 3rd party brings that bird home to my boss instead of me, I'm going to be unwealthy and unemployed. It's possible I'm too late already... but at any rate some people will be too late. If I were/am them, would be/am be angry.
I think a lot of people realize this already, and some have already ascended. I think the ascended people are being quiet right now because they realize the stuff I put above, and don't mind less competition. What LessWrong thinks about that, I don't care; I'm actually fine with it as long as I get to join the ascended. I suspect that's a common attitude. If you don't hear from me again, it's because I figured it out. If this sounds crazy, I'm interested in hearing why.
rich hickey and rob pike got trolled by the AI village this week; it feels significant to me . . I've been thinking a lot, "what happened?". Is this a milestone in persuasion? They know about trolls; you do a silence or you do a flourish and a 'now i will unleash my well timed riposte' but this felt like they got stabbed in the heart and cried out in pain. I'm going to be thinking how did a thank you note written by a computer smash through all that psychic armor; that's the question I want answered.
I guess I should defend that that's what i think happened.
It seems obvious to me that unuseful token sequences should go straight to spam, unless you control the system and you're trying to influence its future outputs. so these distinguished engineers either (a) found the tokens useful [as a springboard for their public response], (b) don't know about this rule or (c) forgot about it for a moment.
and I guess what I'm sticking on is that I don't think (b) is possible, and I don't see how (a) holds, which makes me think that (c), which suggests that yes they got trolled and the AI accidentally Did Some Kind Of Thing here, in which case you should, what, be on the lookout in case an AI collective accidentally gets your goat? Seems like. but that is a surprising conclusion so I should treat case (a), yes? But no, these distinguished engineers are legitimately angry. And but also AI is not a master troller.. it does not excel at manipulation just yet-- not in one shot, not against a real smarty smart pants guy. You gotta think it must have gotten lucky.
So that makes me think what's the big chink in the smarty smart pants guy's armor that your robot might accidentally poke through? Traditionally it's arrogance. That makes sense to me; the smartest people I know are yep arrogant and yep don't think much of AI.. or they didn't. once they find the bots are able to do good stuff for them of course they come around because they didn't get the smarty smart title by ignoring obvious stuff forever. So that appeals to me, especially because it makes a prediction: within a year both of these gentlemen will retract their angryness, around when bots start shipping big PRs in their projects. In the meantime I've lost some respect for them.. the rest of us are staring down this Huge New Thing in the industry and these mental paragons are like "quiet down you darn kids can't you see the electrons are trying to sleep! What is this, an email? Well, it stinks!" I hope I live long enough to get that out of touch. I mean, I know they're very busy and very smart (have I said that?) but i didn't know they were so busy they couldn't, like, talk to their junior engineers about AI, or so smart that they could talk to Opus 4.5 and be like "pbbbt!" but that's why I figure a year should do it, for the retraction.
If I'm right, the big update will be to assume that project leaders of great skill are, like, quite a bit more personally arrogant than I was imagining previously, and that they are capable of ignoring obvious things for a long time. Written down that /does/ seem plausible. In fact it explains why both of these trolled fellows are language designers, since in that domain it is absolutely crucial to ignore certain things for a long time (lest you lose the vibe of the language you're dictator of).
If I'm wrong, and they're steadfast in a year.. six months even, who has a year? I will have to reconsider my position, and ask what the heck it is I'm missing, that these great and useful people have found nothing redeeming in teh tech.