All of joseph_c's Comments + Replies

Isn't $\beta$ proportional to the inverse temperature, and so should be smaller now (with easier, more frequent trading)?

A math textbook leaving certain results as an exercise for the reader?

 

I think this is usually actually one of (1) the author not wanting to write out the proof (because it's boring/tedious) or (2) a proof that would make a good exercise because it is easy enough if you understand the big ideas (and coming up with good exercises is not always easy).

I recently came across Backpack Language Models and wanted to share it in case any AI interpretability people have not seen it. (I have yet to see this posted on LessWrong.)

The main difference between a backpack model and an LLM is that it enforces a much stricter rule to map inputs' embeddings to output logits. Most LLMs allow the output logits to be an arbitrary function of the inputs' embeddings; a backpack model requires the output logits to be a linear transformation of a linear combination of the input embeddings. The weights for this linear combinat... (read more)

joseph_c2-1

Most students, 48 percent, claimed to be Native American on their application....

According to Intelligent.com Managing Editor Kristen Scatton, the prevalence of applicants who claim Native American ancestry is possibly due to the popular narrative that for many Americans, a small percentage of their DNA comes from a Native American tribe.

 

Maybe these students are purposely misinterpreting "Native American" to be someone who was born and raised in the United States, perhaps with ancestors born and raised in the US as well. This is actually the older sense of the term "Native American", found, for example, in the name of the Native American Party back in the mid-1800s.

1Andrew Burns
The Amelia Bedelia defense.

includeIt is written in More Dakka:

If something is a good idea, you need a reason to not try doing more of it.

Taken at face value, it implies the contrapositive:

If something is a bad idea, you need a reason to not try doing less of it.

 

This is not the contrapositive. It is not even the opposite.

1Mateusz Bagiński
You're right, fixed, thanks!
joseph_c10

Parfit's Hitchhiker: You're stranded in the desert and Omega comes up. It will give you a ride out of the desert iff it predicts you'd give it 10,000 dollars upon reaching civilization again. You get a ride. When in civilization again, do you go over to the bank and withdraw some money? Well, policies which pay up in this specific situation get (value of a life - 10,000 dollars) more than policies which don't pay in this specific situation, which just die.

Why is this called Parfit's Hitchhiker? Who is the Parfit it is referring to? Where was this scenario first written up? (I'm trying to dig up the original reference.)

2tslarm
It's based on a scenario described by Derek Parfit in Reasons and Persons. I don't have the book handy so I'm relying on a random pdf here, but I think this is an accurate quote from the original:

Not for Mormons. They don't believe in an omnipresent God.

2the gears to ascension
ok, fair, that is true, but I interpreted the core of the query to be > 1/ when someone says they "believe in God" does this mean something like "I assign a ≥ 50% probability to there being an [examples of attributes they may give] intelligence?"  the thing I was affirming is that yes, they really would verbally assign a high probability to that claim in any tractable way of querying their probabilities. I don't know how many actually perceptually assign a high probability, in the sense of their perceptual system making direct predictions which are based on a high probability of the claim they would verbally endorse, but getting them to introspect enough to compare their perceptual anticipations to their verbal anticipations is generally extremely difficult.
joseph_c6452

Well, what are your actual steps? Or is this just advertisement?

-24George3d6

Do you still live in Utah?

Did your family cut you off?

Do you know about [r/exmormon](https://old.reddit.com/r/exmormon/)?

8ErioirE
I do still live in Utah. I haven't told my family yet. One of my siblings already left the church and my family didn't cut them off, and I'm confident they wouldn't cut me off either.  On the other hand, my sibling leaving broke my mom's heart so I don't see any reason to do that any sooner than I have to. In this case, what she doesn't know can actually hurt her once she does. In her perspective she sees a personal failure on her part if her children lose faith, which is obviously irrational and unfair to both herself and us. r/exmormon is quite a bad environment IMHO. While there are some nice and reasonable folks, they seem to be either a minority or simply less vocal than those who are not. A significant portion of those who frequent exmo-specific groups (or at least post often) tend to be those who are angry and bitter. As far as I can tell some of them still blame the church for everything bad in their life even decades after leaving. Those with a more healthy outlook tend to move on and find better things to do. Back when I was a questioning-but-not-yet-disenfranchised member, encountering exmo groups was counter-productive because it only served to feed the confirmation bias of "wow, all these ex-mormons sure are miserable, just like I've been told!"

Maybe try controlling for age? I think young people are both less likely to have signed up for cryonics (because they have less money and are less likely to die) and also have higher probabilities of cryonics working for them (because cryonics will improve by the time they need it).

 

This graph seems to match the rise of the internet. Here's my alternate hypothesis: Most people are irrational, and now it's more reasonable to call them crazy/stupid/fools because they have much greater access to knowledge that they are refusing/unable to learn from. I think people are just about as empathetic as they used to be, but incorrect people are less reasonable in their beliefs.

1Kevin Dorst
Interesting!  A middle-ground hypothesis is that people are just as (un)reasonable as they've always been, but the internet has given people greater exposure to those who disagree with them.

The trick here is that both equations contain  which is the hardest to calculate, and that number drops out when we divide the equations.

 

You have a couple typos here. The first centered equation should not have a $P(\bar H H | X)$ but instead have $P(\bar H | X)$, and the inline expression should be $P(D | X)$, not $P(D | H)$.

3Gunnar_Zarncke
Also, the second $P(intact∣orc,X)$ should be $P(intact∣goblin,X)$  

A few things to note:

  1. GPT-4's release was delayed by ~8 months because they wanted to do safety testing before releasing it. If you take this into account your graph looks much less steep.
  2. The employees at OpenAI know about prediction markets.
  3. They also have incentives to manipulate them to look like GPT-5 will come out later than it actually will. They don't want to set off an AI arms race.
2Malte
"GPT-4's release was delayed by ~8 months because they wanted to do safety testing" I have heard this claim before (with 6 months).  This could be understood as "GPT-4 was ready to go 6 month earlier, they simply did a lot of testing to go the extra mile." Alternatively this is how long it took to make the foundational model useful, and while they did spend extra resources for red teaming etc. in parallel, this didn't come with a great cost of releasing it later.   Are we sure they didn't just count the time to RLHF it? Seems plausible to me that it always takes ~ 20% of dev time to RLHF a model. (epistemic status: spitballing)

I think most people view "All people are equal" as a pronouncement of a moral belief they hold, not as a statement of fact. When they say, "All people are equal", they mean they believe "all people should be treated equally", or "everyone should have to obey the same laws" or "everyone's needs have equal importance".

This moral pronouncement is also consistent with a utilitarian pronouncing "All people are equal to me", as in that all people's lives hold equal weight in his utility function.

1Jacob G-W
Thanks! I've updated my post: https://jacobgw.com/blog/observation/2023/08/21/truth.html
Answer by joseph_c11

I think the old meaning of "bigot" is very close to this. From the 1828 Websters Dictionary:

BIG'OT, noun

1. A person who is obstinately and unreasonably wedded to a particular religious creed, opinion, practice or ritual. The word is sometimes used in an enlarged sense, for a person who is illiberally attached to any opinion, or system of belief; as a bigot to the Mohammedan religion; a bigot to a form of government.

2. A venetian liquid measure containing the fourth part of the amphor, or half the boot.

How much more advantageous would this be than a "head only" option? To get to the brain, wouldn't you have to cut open the head anyways?

2Mati_Roy
All the advantages listed in the post are advantages compared to preserving the skull along with the brain. Advantages of leaving the brain in the skull: * Additional protection provided by the skull * Avoid delicate procedure of removing the brain The post links to this Isolation of the Brain for Human Cryopreservation. I was told by someone performing cryopreservations that you can remove the brain and just leave a scar at the top of the forehead hidden by hair.

I think it really depends on your reading speed. If you can read at 500 wpm, then it's probably faster for you to just read the book than search around for a podcast and then listen to said podcast. I do agree, though, that reading a summary or a blog about the topic is often a good replacement for reading an entire book.

1Adam Zerner
I'm having trouble seeing how that'd ever be the case. In my experience searching for a podcast rarely takes more than a few minutes, so let's ignore that part of the equation. If a book normally takes 10 hours to read, let's say you're a particularly fast reader and can read 5x as fast as the typical person (which I'm skeptical of). That'd mean it still takes 2 hours to read the book. Podcast episodes are usually about an hour. But if you're able to read 5x faster that probably implies that you're able to listen to the podcast at at least 2x speed if not 3x, in which case the podcast would only take 0.5 hours to go through, which is 4x faster than it'd take to read the book.
Answer by joseph_c22

I think robotics was (and still is) mostly bottlenecked on the algorithms side of things. It's not too expensive to build a robot, and the software is good enough that a hobbyist could hack something together easily enough in a day or two. The issue is that it's really hard to make a robot do what you want it to do. Even if you have a robot that can stand up, run around, and do back flips, how do you make it go rescue people from burning buildings? Most of the tasks robots could be useful for are messy, complicated things, and robots don't yet know how to ... (read more)

1ZZZZZZ
Link to video to try for myself?
3DirectedEvolution
When you tried it out, did you go as far as to hook yourself up to a brain scanner and match the flashes to the frequency and phase of your (alpha?) waves? My memory of the paper was that getting the phase and frequency right made a big difference for the effect. Importantly, that paper also studied the benefits of entrainment matched to phase and frequency of brainwaves for learning a very specific type of dynamic visual recognition task. My take on the paper was that it seemed unlikely to extrapolate to learning generally (i.e. it wasn't "strobe yourself and learn anything 3x faster"). By contrast, the body of research here finds modest effects, but on a wide range of cognitive performance tasks and with a clear mechanistic hypothesis to explain why this might be happening. I think employing powerful medical lasers, as in the original studies, is far more plausible and tractable as a general learning-enhancement strategy than the strobe light brainwave-matching. But the effects sizes are small enough and the risks uncertain enough that it doesn't seem worth the investment to me.

I noticed that you listed "Salamander" as rationalist/rationalist adjacent fiction. I've never heard of it before, and Google doesn't seem to know either. What is this?

3gjm
I'm guessing it's this. Its author (a libertarian economist) sometimes comments on, and is sometimes mentioned in, Scott's blog posts. (There's a David Friedman with an LW account but whether or not it's the same person it doesn't seem like he's ever posted anything.)

Lying is a social lubricant. The classic defence of lying here- if someone asks you: "Does my bum look too big in this dress?", you don't want to be honest and respond: "Yes, you look like a whale who has swallowed another, much larger whale."

That's not being honest--that's just being mean. If you really want to present an uncharitable view of honesty, maybe at least make the statements you claim to be honest actually true? For example, the response "No, it's your fat that does it," is also rather unkind but has the advantage of maybe being true.

2Dzoldzaya
Good point- I should have stressed that the overwhelming and undeniable visual effect from the bum in the dress was that of a whale swallowing another larger whale, such that saying anything else would be dishonest in spirit.
joseph_cΩ010

Did you and your friend only communicate via text messages/email? I think that would make a better comparison to asking ChatGPT help than having your friend in the same room as you give instructions based off of what they see.

6Alex Flint
I asked a group of friends for "someone to help me with an AI experiment" and then I gave this particular friend the context that I wanted her help guiding me through a task via text message and that she should be in front of her phone in some room that was not the kitchen. If you look at how ChatGPT responds, it seems to be really struggling to "get" what's happening in the kitchen -- it never really comes to the point of giving specific instructions, and especially never comes to the point of having any sense of the "situation" in the kitchen -- e.g. whether the milk is currently in the suacepan or not. In contrast, my human friend did "get" this in quite a visceral way (it seems to me). I don't have the sense that this was due to out-of-band context but I'd be interested to retry the experiment with more carefully controlled context.
Answer by joseph_c10

I can't really think of a word that describes this. Maybe "dogmatic", "fanatic", "blind faith", or "convicted"?

1metachirality
That's not really specific enough. I would describe it as someone being really angry about something, contingent on a certain belief being true, but then when you ask them why they believe that belief, its very weak evidence or something that is the opposite of an open and shut case or something that could vary depending on context and so on and so forth.

You should probably also put up a sign/sticky note saying "free books" so people know they're free :)

4ChristianKl
If someone just finds a book lying in a corner there's added excitement for finding it that you don't have with a "free books" sign. 

The current premise is that, by locally monitoring factors, such as the MAC and IP address a user is connected to, we can prevent others signing onto the same device. Essentially, one account may be accessed via multiple devices, however, only one account may be accessed per device. In theory, this should minimise the incentive to create multiple accounts, as there is presently no explicit way to circumvent the issue.

Why can't someone spoof their MAC/IP address? Or even easier, buy two devices?

Could you please elaborate? Why is it bad to publicly specify these things?

Answer by joseph_c10

Is there a reason you want to take classes instead of self-study? If you're interested in self-studying, MIT OpenCourseWare has a lot of useful classes. I'd also check out https://www.cs.cmu.edu/~10715-f18/.

Answer by joseph_c10

Hi! I'm a current MIT student. Here's how it works at MIT. Feel free to reply back for more information:

MIT is great in terms of classes. Getting out of prereqs is pretty easy. You just talk to the professor and get permission to take their class. I've done this in two classes so far (this is just my first semester here!) and they approved me without a problem. I also took many concurrent enrollment classes in high school at a local university and the process was much the same. My experience has been that professors are very willing to let ambitious studen... (read more)

[anonymous]108

Weak downvoted. The links above aren't paywalled, and our homepage states that there's a subscription to join as a full member. A substantial portion of our members (about 20%) asked us to waive their subscription and we have a long standing policy of granting these requests with no questions asked. (While this isn't mentioned on the site, it's the first thing you see upon joining the Discord server, which doesn't require you to pay us anything.)

9David Youssef
Uh the Guild does ask for money but we are completely free as long as you send us even one emeial. the money is nice to continue the work of expanding but that is NOT why we do this. Several of our students pay nothing at all Source: Me, I am a Member of the Guild Council

Sorry, I meant . And yes, that should eliminate the term that causes the incorrect initialization to decay. Doesn't that cause the learning to be in the correct direction from the start?

2Adam Jermyn
I don't think so? I think that just means you keep the incorrect initialization around while also learning the correct direction.

Have you experimented with subtracting  from the loss? It seems to me that doing so would get rid of the second term and allow the model to learn the correct vectors from the beginning.

1Adam Jermyn
That's not a scalar, do you mean the trace of that? If so, doesn't that just eliminate the term that causes the incorrect initialization to decay?

I thought masters' theses were supposed to be about new research (and maybe bachelor theses too?). Is this not the case?

4Richard_Kennaway
That would depend on the university's rules and customs. I think a general rule of thumb might be that a bachelor's thesis may contain new research, a master's thesis should, and a doctoral thesis must.

Is this serious? I find it somewhat ironic that your deontology is completely closed-minded on its belief about narrow-mindedness.

1LVSN
I don't think being somewhat ironic would be slightly worse than being completely insensitive to unfamiliar considerations.

Fact check: Mormons don't go on missionaries until they are at least 18 for men and 19 for women.

Missionaries can be single men between the ages of 18 and 25, single women over the age of 19 or retired couples. Missionaries work with a companion of the same gender during their mission, with the exception of couples, who work with their spouse. Single men serve missions for two years and single women serve missions for 18 months.

See https://news-pg.churchofjesuschrist.org/topic/missionary-program.

Also, ever since the most recent transfer of power, Mormons h... (read more)

1[anonymous]
Oh I'm well versed in the horrors of contemporary religious tradition and ritual. The idea was always to mimic the good without the bad.  The tradition itself would never be good, it would be the end to which it serves that makes it good. This ideally keeps traditions in check. 

Your outline has a lot of beliefs you expect your students to walk away with, but basically zero skills. If I was one of your prospective students, this would look a lot more like cult indoctrination than a genuine course where I would learn something.

What skills do you hope your students walk away with? Do you hope that they'll know how to avoid overfitting models? That they'll know how to detect trojaned networks? That they'll be able to find circuits in large language models? I'd recommend figuring this out first, and then working backwards to figure ou... (read more)

1Tapatakt
Thanks for your answer! This is about... I wouldn't say "beliefs" - I will make a lot of caveats like "we are not sure", "there are some smart people who disagree", "this is an arguments against this view", etc. (mental note: do it MORE, thank you for your observation) - but about "motivation" and "discourse". Not about technical skills, that's true. I have a feeling that there is an attractor "I am AI-researcher and ML is AWESOME, and I will try to make it even more AWESOME, and yes, there are this safety folks and I know some of their memes and may be they have some legitimate concerns, but we will solve it later and everything will be OK". And I think that when someone learns some ML-related technical skills before basic AI Safety concepts and discourse, it's very easy for them to get into this attractor. And from this point it's pretty hard to return back. So I want to create something like a vaccine against this attractor. Technical skills are neccesary, but for most of them there are already good courses, textbooks and such. The skills I saw no texbooks for are "to understand AIsafetyspeak" and "to see why alignment-related problem X is hard and why obvious solutions may not work". Because of the previously mentioned attractor I think it's better to teach this skills before technical skills. I make an assumption that average 15-16-year-olds in my target audience know how to program at least a little bit (In Russia basic programming in theory is in the mandatory school program. I don't know about US), but don't know calculus (but I think smart school student can easily understand a concept of a derivative without strict mathematical definition).

No, I don't. The resources I saw on a quick Google search were rather poor as well.

Have you heard about pseudoentropy? The pseudoentropy of a distribution is equal to the highest entropy among all computationally indistinguishable distributions. I think this might be similar to what you're looking for.

1Adam Shai
No I haven't! That sounds very interesting, I'll definitely take a look, thanks. Do you have a particular introduction to it?

Also by buying off or convincing those who think they have concentrated benefits that they are wrong and should stand down, as even they get more benefit from ending the diffuse costs.

This really doesn't seem like a good way to get politics done. Is this even legal? And if it is, do you really think it makes the government better to have people effectively bribing politicians?

jaspax101

Those benefitting are usually not politicians, they're commercial interests who make money from the status quo. They will oppose efforts that cause them to lose money even if the change is a net good overall, but you can quiet them down by giving them a bunch of money. Typically doing so is still a net good, because the cost of buying off the opposition is (usually) less than the value gained by the rest of society.

Perhaps the verb "buy off" is not the best one here, but I'm not sure what else you'd use. If you're morally offended by the idea of offering payments to lessen the sting for people who suffer a concrete downside from your policies then, uh, don't go into politics I guess.

7ryan_b
This sentence refers to interest groups, not to politicians or officials. It refers to unions, who want bargaining power, and union members, who want stable jobs and good pay. Or to businesses with a captive market, like American shipbuilders and dredge operators. These groups think they are getting a payoff from status quo, and it is one they want to keep. The solution is therefore to match the payoff under the new proposal, or persuade them (of the truth that) they are not actually getting the payoff they think.

What Marc Andreessen has been reading. I am envious of those who get to read this many books, let alone Tyler Cowen levels of reading books. No idea how to make the time for it.

Have you considered reading Twitter less and replacing that with books?

Could I point out that avoiding head injuries might not be the only reason you wouldn't want your children to play football? You might also not want your child to adopt the culture that a lot of high school football teams have (partying, not caring about school, self-centered), which can happen quite easily if they're around football kids 3 hours/day 6 days/week.

15hout
Fair enough. Certainly schools have that, other schools have different cultures as well (but I'd guess the average is closer to your thought). Likely a different vibe from track/cross-country and marching band for sure. (Which isn't too say you don't have people that crossover/differences).
Answer by joseph_c80

I don't think so. I've also done the foobar challenge in the last year or so, and got nada from them.

I remember reading about a startup that is basically using LLMs to let you navigate through websites quicker. I'll edit this comment if I remember what it is.

9gwern
Adept.

I know you're joking, but I'd like to clarify that Jesus actually said "Let he who is without sin cast the first stone," in case some future archeologist who doesn't know anything about 21st century religions uncovers this article. Nukes didn't exist in the first century A.D.

Answer by joseph_c20

With Obsidian, I think you can get the Excalidraw plugin to draw images, though it's not inline (it opens a new pane).

You can also use Numba to speed up loops.  It's still slower than C, but it's much better than plain Python code, and it's really easy to implement (just import  numba and put a @numba.njit() before your function).

Why did you decide to only use rotation matrices instead of any invertible matrix?  If you're trying to find a new basis to work in, wouldn't any invertible matrix work just as well?

I agree with your fundamental claim that there are lots of top tier students going to non-top schools, but I think you focused too much on SAT scores and GPA.  Right now, there are so many kids getting top scores (about 5,500 students every year get a 36 on the ACT, and about 4500 students get a at least a 1570 on the SAT), test scores just aren't enough to determine who gets in.  Instead, admissions officers use a "holistic" approach, which seems rather noisy, but does factor in other real accomplishments, like getting to the IMO or starting a m... (read more)

Load More