I think it is a strong preference, which is why people who accidentally enable it or forget they enable it complained so much when we defaulted to 'auto' on Gwern.net.
Could you say more about the efficient sampling through convolutions? Bad interactive latency was a major reason I didn't spend any time exploring Bayesian approaches for my quick-and-dirty resorter
script.
I take it you are not using a conjugate approach for fast exact estimates, nor a Laplacian approximation, nor full slow MCMC, but I'm not sure what sort of speed you actually get from your approach.
I found the reasons that you gave in the case of BB, bizarrely mundane.
It is in fact a mundane topic, because you are surrounded by AI slop and people relying heavily on ChatGPT writing, making it a mundane every day observation infiltrating even the heights of wordcel culture (I've now started seeing blatant ChatGPTisms in the New Yorker and New York Times), which is why you are wrong to bend over backwards to require extraordinary evidence for what have become ordinary claims (and also why your tangents and evasions are so striking).
So, I am again going to ignore those, and will ask you again - you were sure that BB was not using ChatGPT, despite the linguistic tells and commonness of it:
These political asides are of a piece with the philosophical jabs and brags he makes in his philosophical essays.
That doesn't actually rebut my observation, unless you are claiming to have seen jibes and sneering as dumb and cliche as those in his writings from before ChatGPT (Nov 2022).
I am still waiting for an answer here.
Seems like integrated enough zoom tools would work like this
Again, they cannot, because this is not even a well-defined task or feature to zoom in on. Most tasks and objects of perception do not break down as cleanly as "the letter 'e' in 's t r a w b e r r y'". Think about non-alphabetical languages, say, or non-letter non-binary properties. (What do you 'zoom in on' to decide if a word rhymes enough with another word to be usable in a poem? "Oh, I'd just call out using a tool to a rhyming dictionary, which was compiled by expert humans over centuries of linguistic analysis." OK, but what if, uh, there isn't such an 'X Dictionary' for all X?)
Especially since the lack of examples of genuinely good writing would have made it easy to pattern-match me to the kinds of people who get enthused about LLM writing due to sycophancy and then fail to notice how bad it is.
I mean, why wouldn't people think that? You spend a while describing your setup and work as purely a hobby to cope with depression and about pleasure. Then to describe the results of this intensive many-months-long work with extreme prompt engineering, you describe most of it as being unpublishable trash - not fit for even a quick throwaway blog post:
I have mostly been writing for my own pleasure. The things I've been writing have been quite idiosyncratic to my interests and preferences, and I don't expect many people to necessarily appreciate them. That said, I do have at least one story that I've been starting to feel has enough literary merit that I might want to share it more widely someday. I've been editing it as I write, but for now, that's out of a pure love of the craft - I just enjoy working on it for its own sake, regardless of whether I'll ever publish it.
Nor do any of your examples particularly change the reader's mind.
You also don't establish any particular bona fides for you having any literary talent or taste, and you spend time describing how you are a bad writer:
It used to be that I really liked writing fiction, but couldn't write longer stories myself. I'd manage a couple of scenes and then just run out of ideas, or feel like my characters didn't come properly alive. My style of writing tends to involve immersing myself in the head of one character and writing from their perspective. This has the problem that it makes it hard to jump into the heads of the other characters at the same time, so they end up flat and lifeless. This problem can be avoided if I have a co-writer. They write one character, I write another, and then we have those characters interact. I also love the interplay of ideas when writing with somebody - I introduce an element into the story and see how the other person builds on it, and then they introduce an element of their own and I build on it. It’s amazingly fun to come up with something and see what someone else does with it...For me, co-writing works off momentum. It takes some investment to drop into that character headspace, and once I’m there, I don’t want to leave. I’ve had several brief attempts at co-writing together with someone that ended because one of us couldn’t commit enough time and energy to make it work. Or worse, it did work and I got obsessed with it, but then my partner couldn’t invest a corresponding amount of time into it and the collaboration sputtered away just as I’d gotten excited about it.
If people write it off as a lengthy description of one person's masturbatory LLM use generating midbrow (at best) fiction of no value to anyone else, they are only taking away what you seemed to want them to take away. Why should anyone read the post in full?
And why argue with you about it? Certainly I do not enjoy the many comments I have been leaving all other the Internet (not just LW) pointing out LLM confabulations, or blatant tells of ChatGPT use, only to be greeted by downvotes, mockery, lies, denial, or long-drawn-out paltering before finally acknowledging that yeah I was right all along, and that's the easiest case for arguing with LLM-corrupted users (a mere factual matter of 'did ChatGPT write this?').
That a link contains a UTM parameter referring to ChatGPT, tells us only that this link was provided in ChatGPT output, it doesn't tell us that the text around the link was written by ChatGPT as well. As for the article itself, I find nothing in its verbal style that is outside the range of human authorship... But even the emdash has its human fans (e.g. the Chicago Manual of Style). It can be a sign of a cultivated writer
I note, that aside from bending over backwards to excuse multiple blatant signs of a common phenomena which requires little evidence to promote to high posterior confidence, you still are not responding to what I said about BB and have instead chosen to go off on a tangent.
So again. Are you claiming to have seen jibes and sneering as dumb and cliche as those in his writings from before ChatGPT (Nov 2022), and thus rebutting my observation about how it sounds like he is gracelessly using ChatGPT?
The economics here seem wrong.
Poor people do not benefit less from debt than rich people do - they benefit vastly more, because they have major cashflow and liquidity issues. (I wouldn't go so far as to claim interest rates are the least important things about debt, but they are discussed disproportionately compared to aspects like credit availability.) They do not shun debt, but use it in countless highly complex forms to deal with the challenges of routinely running out of money before the next pay period, or the 'feast or famine' variance in payments that to a richer (ie. middle-class) person would barely register as a blip on their bank account balance. Arbitraging 2% vs 5% ROI is trivial compared to arbitraging 'not getting evicted' or 'not getting fired'. (Borrowing $20 for gas in the next half hour can be life-changing; getting 40 cents extra on your Vanguard ETF retirement account 50 years later is not.)
A useful reference for me for understanding this was reading Portfolios of the Poor.
Incidentally, I would note that Polonius is an aristocrat speaking to an aristocrat (and about to be killed through his own foolishness), and his advice should not be taken on finance, or perhaps anything else either.
reading above I see that my first comment could be interpreted as saying we should just do character-only tokenization during inference. This isn't what I was suggesting.
Yes, that is what I took you to mean.
Anyway, to address your revised claim & example: this may work in that specific simple 'strawberry' task. The LLMs have enough linguistic knowledge to interpret that task and express it symbolically and handle it appropriately with its tools like a 'zoom tool' or a Python REPL.
However, this is still not a general solution, because in many cases, there either is no such tool-using/symbolic shortcut or the LLM would not feasibly come up with it. Like in the example of the 'cuttable tree' in Pokemon: is there a single pixel which denotes being cuttable? Maybe there is, which could be attacked using Imagemagick analogous to calling out to Python to do simple string manipulation, maybe not (I'm not going to check). But if there is and it's doable, then how does the LLM know which pixel to check for what value?
This is not how I learned to cut trees in Pokemon, and it's definitely not how I learned to process images in general, and if I had to stumble around calling out to Python every time I saw something new, my life would not be going well.
How about the fact that the opinions in the inserted asides are his actual opinions?
I'm not sure I believe they are his 'actual opinions', but it doesn't matter to my points.
If they were randomly generated, they wouldn't be.
No one, particularly me, said they were 'randomly generated', so that again does not rebut any of my observations.
Wrong. Spelling reflects pronunciation to a considerable degree. Even a language like English, which is regarded as quite pathological in terms of how well spelling of words reflects the pronunciation, still maps closely, which is why https://en.wikipedia.org/wiki/Spelling_pronunciation is such a striking and notable phenomena when it happens.
Shape perception and other properties like color are also global, not reducible to single pixels.