Ω 04y

Hangnails are Largely Optional

Hangnails are both annoying and painful, often snagging on things and causing your fingers to bleed. Typical responses to hangnails include:

Ignoring them.
Pulling them out, which can lead to further bleeding or infection.
Trimming them with nail clippers, which often leaves a jagged edge.
Wrapping the affected finger in a bandage, requiring daily changes.

Instead, use a drop of superglue to glue it to your nail plate, IMO a far superior option. It's $10 for 12 small tubes on Amazon. Superglue is also useful for cuts and minor... (read more)

Examples of Highly Counterfactual Discoveries?

172

johnswentworth, kromem

The history of science has tons of examples of the same thing being discovered multiple time independently; wikipedia has a whole list of examples here. If your goal in studying the history of science is to extract the predictable/overdetermined component of humanity's trajectory, then it makes sense to focus on such examples.

But if your goal is to achieve high counterfactual impact in your own research, then you should probably draw inspiration from the opposite: "singular" discoveries, i.e. discoveries which nobody else was anywhere close to figuring out. After all, if someone else would have figured it out shortly after anyways, then the discovery probably wasn't very counterfactually impactful.

Alas, nobody seems to have made a list of highly counterfactual scientific discoveries, to complement wikipedia's list of multiple discoveries.

To...

(See More – 189 more words)

CuriousMeta16m10

~Don't aim for the correct solution, (first) aim for understanding the space of possible solutions

3Johannes C. Mayer11h

Ok, I was confused before. I think Homoiconicity is sort of several things. Here are some examples: * In basically any programming language L, you can have program A, that can write a file containing a valid L source code that is then run by A. * In some sense, python is homoiconic, because you can have a string and then exec it. Before you exec (or in between execs) you can manipulate the string with normal string manipulation. * In R you have the quote operator which allows you to take in code and return and object that represents this code, that can be manipulated. * In Lisp when you write an S-expression, the same S-expression can be interpreted as a program or a list. It is actually always a (possibly nested) list. If we interpret the list as a program, we say that the first element in the list is the symbol of the function, and the remaining entries in the list are the arguments to the function. Although I can't put my finger on it exactly, to me it feels like the homoiconicity is increasing in further down examples in the list. The basic idea though seems to always be that we have a program that can manipulate the representation of another program. This is actually more general than homoiconicity, as we could have a Python program manipulating Haskell code for example. It seems that the further we go down the list, the easier it gets to do this kind of program manipulation.

4Answer by AnthonyC17h

I think it's worth noting that small delays in discovering new things would, in aggregate, be very impactful. On average, how far apart are the duplicate discoveries? If we pushed all the important discoveries back a couple of years by eliminating whoever was in fact historically first, then the result is a world that is perpetually several years behind our own in everything. This world is plausibly 5-10% poorer for centuries, maybe more if a few key hard steps have longer delays, or if the most critical delays happened a long time ago and were measured in decades or centuries instead.

Thoughts on seed oil

266

dynomight

10d

This is a linkpost for https://dynomight.net/seed-oil/

A friend has spent the last three years hounding me about seed oils. Every time I thought I was safe, he’d wait a couple months and renew his attack:

“When are you going to write about seed oils?”

“Did you know that seed oils are why there’s so much {obesity, heart disease, diabetes, inflammation, cancer, dementia}?”

“Why did you write about {meth, the death penalty, consciousness, nukes, ethylene, abortion, AI, aliens, colonoscopies, Tunnel Man, Bourdieu, Assange} when you could have written about seed oils?”

“Isn’t it time to quit your silly navel-gazing and use your weird obsessive personality to make a dent in the world—by writing about seed oils?”

He’d often send screenshots of people reminding each other that Corn Oil is Murder and that it’s critical that we overturn our lives...

(Continue Reading – 4926 more words)

sapphire24m20

Strong upvoted. I learned a lot. Seriously interested in what you think is relatively safe and not extremely expensive or difficult to acquire. Some candidates I thought of but im not exactly well informed:
-- Grass fed beef
-- oysters/muscles
-- some whole grains? which?
-- fruit
-- vegetables you somehow know arent contaminated by anti-pest chemicals?

I really need some guidance here.

2JenniferRM5h

This bit caught my eye: I searched for [is olive oil cut with canola oil] and found that in the twenty teens organized crime was flooding the market with fake olive oil, but in 2022 an EU report suggested that uplabeling to "extra virgin" was the main problem they caught (still?). Coming from the other direction, in terms of a "solid safe cheap supply"... I can find reports of Extra Virgin Olive Oil being sold by Costco under their Kirkland brand that is particularly well sourced and tested, and my priors say that this stuff is likely to be weirdly high quality for a weirdly low price (because, in general, "kirklandization" is a thing that food producers with a solid product and huge margins worry about). I'm kinda curious if you have access to Kirkland EVOO and if it gives you "preflux"? Really any extra data here (where your sensitive palate gives insight into the current structure of the food economy) would be fascinating :-)

1Ann8h

Thanks for the reference! I'm definitely confused about the inclusion of "pre-prepared (packaged) meat, fish and vegetables" on the last list, though. Does cooking meat or vegetables before freezing it (rather than after? I presume most people aren't eating meat raw) actually change its processed status significantly?

1Freyja9h

Also as a brief pointer at another cool thing in Metabolical, Lustig claims that exercise is useful for weight loss mostly because of its beneficial impact on cell repair/metabolic system repair (something specific about mitochondria?) and not for the calorie deficit it may or may not create. I consider Lustig's science to be quite thorough, I like him a lot. The main point against him is that he personally doesn't look very metabolically healthy, which I would expect of someone who had spent his life investigating and theorising about what influences metabolic health.

LLMs seem (relatively) safe

JustisMills

This is a linkpost for https://justismills.substack.com/p/llms-seem-relatively-safe

Post for a somewhat more general audience than the modal LessWrong reader, but gets at my actual thoughts on the topic.

In 2018 OpenAI defeated the world champions of Dota 2, a major esports game. This was hot on the heels of DeepMind’s AlphaGo performance against Lee Sedol in 2016, achieving superhuman Go performance way before anyone thought that might happen. AI benchmarks were being cleared at a pace which felt breathtaking at the time, papers were proudly published, and ML tools like Tensorflow (released in 2015) were coming online. To people already interested in AI, it was an exciting era. To everyone else, the world was unchanged.

Now Saturday Night Live sketches use sober discussions of AI risk as the backdrop for their actual jokes, there are hundreds...

(Continue Reading – 1790 more words)

2Seth Herd12h

Nothing in this post or the associated logic says LLMs make AGI safe, just safer than what we were worried about. Nobody with any sense predicted runaway AGI by this point in history. There's no update from other forms not working yet. There's a weird thing where lots of people's p(doom) went up when LLMs started to work well, because they found it an easier route to intellligence than they'd been expecting. If it's easier it happens sooner and with less thought surrounding it. See Porby's comment on his risk model for language model agents. It's a more succinct statement of my views. LLMs are easy to turn into agents, so let's don't get complacent. But they are remarkably easy to control and align, so that's good news for aligning the agents we build from them. But that doesn't get us out of the woods; there are new issues with self-reflective, continuously learning agents, and there's plenty of room for misuse and conflict escalation in a multipolar scenario, even if alignment turns out to be dead easy if you bother to try.

2Seth Herd12h

That is a fascinating take! I haven't heard it put that way before. I think that perspective is a way to understand the gap between old-school agent foundations folks' high p(doom) and new school LLMers relatively low p(doom) - something I've been working to understand, and hope to publish on soon. To the extent this is true, I think that's great, because I expect to see some real insights on intelligence as LLMs are turned into functioning minds in cognitive architectures. Do you have any refs for that take, or is it purely a gestalt?

1quetzal_rainbow12h

If it is not a false memory, I've seen this on twitter of either EY or Rob Bensinger, but it's unlikely I find source now, it was in the middle of discussion.

Seth Herd1h20

Fair enough, thank you! Regardless, it does seem like a good reason to be concerned about alignment. If you have no idea how intelligence works, how in the world would you know what goals your created intelligence is going to have? At that point, it is like alchemy - performing an incantation and hoping not just that you got it right, but that it does the thing you want.

[Aspiration-based designs] 2. Formal framework, basic algorithm

Jobst Heitzig, Simon Dima, Simon Fischer

Summary. In this post, we present the formal framework we adopt during the sequence, and the simplest form of the type of aspiration-based algorithms we study. We do this for a simple form of aspiration-type goals: making the expectation of some variable equal to some given target value. The algorithm is based on the idea of propagating aspirations along time, and we prove that the algorithm gives a performance guarantee if the goal is feasible. Later posts discuss safety criteria, other types of goals, and variants of the basic algorithm.

Assumptions

In line with the working hypotheses stated in the previous post, we assume more specifically the following in this post:

The agent is a general-purpose AI system that is given a potentially long sequence of tasks, one by one,

...

(Continue Reading – 4618 more words)

2Charlie Steiner7h

So to sum up so far, the basic idea is to shoot for a specific expected value of something by stochastically combining policies that have expected values above and below the target. The policies to be combined should be picked from some "mostly safe" distribution rather being whatever policies are closest to the specific target, because the absolute closest policies might involve inner optimization for exactly that target, when we really want "do something reasonable that gets close to the target." And the "aspiration updating" thing is a way to track which policy you think you're shooting for, in a way that you're hoping generalizes decently to cases where you have limited planning ability?

Jobst Heitzig1h10

Exactly! Thanks for providing this concise summary in your words.

In the next post we generalize the target from a single point to an interval to get even more freedom that we can use for increasing safety further.

In our current ongoing work, we generalize that further to the case of multiple evaluation metrics, in order to get closer to plausible real-world goals, see our teaser post.

Searching for Searching for Search

Rubi J. Hudson

Ω 82mo

Thanks to Leo Gao, Nicholas Dupuis, Paul Colognese, Janus, and Andrei Alexandru for their thoughts.

This post was mostly written in 2022, and pulled out of my drafts after recent conversations on the topic. My main update from then is the framing. Rather that suggesting searching for substeps of search as an approach I'm excited about, I now see it only as a way to potentially reduce inherent difficulties. The main takeaway of this post should be that searching for search seems conceptually fraught to the point that it may not be worth pursuing.

Searching for Search is the research direction that looks into how neural networks implement search algorithms to determine an action. The hope is that if we can find the search process, we can then determine...

(Continue Reading – 1905 more words)

silentbob2h10

Aren't LLMs already capable of two very different kinds of search? Firstly, their whole deal is predicting the next token - which is a kind of search. They're evaluation all the tokens at every step, and in the end choose the most probable seeming one. Secondly, across-token search when prompted accordingly. Say "Please come up with 10 options for X, then rate them all according to Y, and select the best option" is something that current LLMs can perform very reliably - whether or not "within token search" exists as well. But then again, one might of cours... (read more)

To get the best posts emailed to you, create an account! (2-3 posts per week, selected by the LessWrong moderation team.)

Big-endian is better than little-endian

Menotim

This is a response to the post We Write Numbers Backward, in which lsusr argues that little-endian numerical notation is better than big-endian.^[1] I believe this is wrong, and big-endian has a significant advantage not considered by lsusr.

Lsusr describes reading the number "123" in little-endian, using the following algorithm:

Read the first digit, multiply it by its order of magnitude (one), and add it to the total. (Running total: ??? one.)
Read the second digit, multiply it by its order of magnitude (ten), and add it to the total. (Running total: ??? twenty one.)
Read the third digit, multiply it by its order of magnitude (one hundred), and add it to the total. (Arriving at three hundred and twenty one.)

He compares it with two algorithms for reading a big-endian number. One...

(See More – 763 more words)

Yair Halberstadt2h20

What if you are not a person, but a computer, converting a string into an integer? In that case, having a simpler and faster algorithm is important, having to start with only the beginning of a string (what the user has typed so far) is plausible, and knowing the number's approximate value is useless. So in this case the little-endian algorithm is much better than the big-endian one.

For the most part this is irrelevant. If you are a computer with only partial input, recieving the rest of the input is so much slower than parsing it that it literally does... (read more)

3localdeity7h

One aspect neither of you have explicitly addressed is the speaking of numbers; speaking, after all, predates writing. We say "one billion, four hundred twenty-eight million, [...]". Given that that's what we say, the first two pieces of information we need are "one" and "billion". More generally, we need to get the first 1-3 digits (the leftmost comma-separated group), then we need the magnitude, then we can proceed reading off all remaining digits. Given that the magnitude is not explicitly written down, we get it by counting the digits. If the digits are comma-separated into groups of 3 (and "right-justified", so that if there are 3n+1 or 3n+2 digits, then the extra 1-2 are the leftmost group), then it's generally possible to get the magnitude from your "peripheral vision" (as opposed to counting them one by one) for numbers less than, say, 1 billion, which are what you'd most often encounter; like, "52" vs "52,193" vs "52,193,034", you don't need to count carefully to distinguish those. (It gets harder around 52,193,034,892 vs 52,193,034,892,110, but manually handling those numbers is rare.) So if getting the magnitude is a mostly free operation, then you might as well just present the digits left-to-right for people who read left-to-right. Now, is it sensible that we speak "one billion, four hundred twenty-eight million, [...]"? Seems fine to me. It presents the magnitude and the most significant digits first (and essentially reminds you of the magnitude every 3 digits), and either the speaker or the listener can cut it off at any point and have an estimate accurate to as many digits as they care for. (That is essentially the use case of "partially running the algorithm" you describe.) I think I'd hate listening to "six hundred sixty three, six hundred twenty-seven thousand, four hundred twenty-eight million, and one billion", or even suffixes of it like "four hundred twenty eight million and one billion". Tell me the important part first!

2quiet_NaN8h

I think that it is obvious that Middle-Endianness is a satisfactory compromise between Big and Little Endian. More seriously, it depends on what you want to do with the number. If you want to use it in a precise calculation, such as adding it to another number, you obviously want to process the least significant digits of the inputs first (which is what bit serial processors literally do). If I want to know if a serially transmitted number is below or above a threshold, it would make sense to transmit it MSB first (with a fixed length). Of course, using integers to count the number of people in India seems like using the wrong tool for the job to me altogether. Even if you were an omniscient ASI, this level of precision would require you to have clear standards at what time a human counts as born and at least provide a second-accurate timestamp or something. Few people care if the population in India was divisible by 17 at any fixed point in time, which is what we would mostly use integers for. The natural type for the number of people in India (as opposed to the number of people in your bedroom) would be a floating point number. And the correct way to specify a floating point number is to start with the exponent, which is the most important part. You will need to parse all of the bits of the exponent either way to get an idea of the magnitude of the number (unless we start encoding the exponent as a floating point number, again.) The next most important thing is the sign bit. Then comes the mantissa, starting with the most significant bit. So instead of writing What we should write is: Standardizing for a shorter form (1.6e-19 C --> ??) is left as an exercise to the reader, as are questions about the benefits we get from switching to base-2 exponentials (base-e exponentials do not seem particularly handy, I kind of like using the same system of digits for both my floats and my ints) and omitting the then-redundant one in front of the dot of the mant

2localdeity8h

I generally agree, except I find words like "multiply" and "add" a bit misleading to use in this context. If I read a number like 3,749,328, then it's not like I take 3 million, and then take 7, multiply by 100,000, and get 700,000, and then perform a general-purpose addition operation and compute the subtotal of 3,700,000. First of all, "multiply by 100,000" is generally more like "Shift left by 5 (in our base-10 representation)"; but moreover, the whole operation is more like a "Set the nth digit of the number to be this". If this were a computer working in base 2, "set nth digit" would be implemented as "mask out the nth bit of the current number [though in this case we know it's already 0 and can skip this step], then take the input bit, shift left by n, and OR it with the current number". (In this context I find it a bit misleading to say that "One hundred plus twenty yields one hundred and twenty" is performing an addition operation, any more than "x plus y yields x+y" counts as performing addition. Because 100, by place-value notation, means 1 * 100, and 20 means 2 * 10, and 120 means 1 * 100 + 2 * 10, so you really are just restating the input.) Also, I might switch the order of the first two steps in practice. "Three ... [pauses to count digits] million, seven hundred forty-nine thousand, ...".

On Not Pulling The Ladder Up Behind You

120

Screwtape

Epistemic Status: Musing and speculation, but I think there's a real thing here.

I.

When I was a kid, a friend of mine had a tree fort. If you've never seen such a fort, imagine a series of wooden boards secured to a tree, creating a platform about fifteen feet off the ground where you can sit or stand and walk around the tree. This one had a rope ladder we used to get up and down, a length of knotted rope that was tied to the tree at the top and dangled over the edge so that it reached the ground.

Once you were up in the fort, you could pull the ladder up behind you. It was much, much harder to get into the fort without the ladder....

(Continue Reading – 2402 more words)

4Screwtape14h

Epistemic status: memories from five years ago where I was stressed and sleep deprived at the time. So, the primary thing I thought the Megameetup did was have overnight space for the people who registered for overnight and space during the day for people who registered for the day. I closed registrations when I thought we had as many people as the space could hold, and made most of my calculations and planning based on the number of people who registered. (Mostly food, but I'd also been asked to check that certain people the community had had problems with weren't attending.) I knew Solstice was going on that weekend and had coordinated a little bit with the Solstice organizer, but mostly just to know the time and location so I knew when to send people over. During the weekend- if I remember correctly, this was in the early afternoon on Saturday, so about five hours before Solstice and while the Megameetup was in full swing- people start pointing out that with registration closed, people who just planned to go to the afterparty didn't know if they were supposed to just show up or what. I don't remember the exact conversation, but basically over the course of about fifteen minutes I realized that lots of people were assuming that the megameetup would host Solstice's afterparty, and that an unknown number of people were attending Solstice who hadn't registered at all with Megameetup but expected to go to the afterparty. I have five hours to prepare for an unknown number of people to converge on us, when we were already at what I thought was capacity for the venue with a little safety margin, while simultaneously trying to keep the event I knew I was planning on course. I could try and tell people not to, but lots of people including my co-organizers have been assuming obviously the afterparty is at the Megameetup and people who went to solstice can come, even if they didn't tell Megameetup they were coming, and if Megameetup isn't hosting this then someone else

2Algon7h

You can't just say that and not elaborate!

Screwtape3h20

Attendee: knock knock Hey, is the organizer in there?

Me: Yeah, what's up?

Attendee: The fire department is here, and we think an attendee just left in an ambulance but we're not sure who or why.

Me: . . . I'll be right out.

And that's the most stressful thing that's ever happened to me as an event organizer.

2Screwtape14h

A history of the NYC Rationalist Megameetup is in my drafts. Someday I hope to finish it, ideally around when I announce 2024's iteration.

LESSWRONG
LW

Quick Takes

Popular Comments

Recent Discussion

Hangnails are Largely Optional

Assumptions

I.

LessOnline

A Festival of Writers Who are Wrong on the Internet

May 31 - Jun 2, Berkeley, CA