All of JoshuaFox's Comments + Replies

> maybe Gary Marcus-esque analysis of the pattern of LLM mistakes?

That is good. Can you recommend one?

Thank you. 

> It's unclear if this implies fundamental differences in how they work versus different specializations.

Correct. That article  argues that LLMs are more powerful than humans in this skill, but not that they have different (implicit) goal functions or that their cognitive architecture is deeply different from the human.


> Like, obviously it's gonna be alien in some ways and human-like in other ways. Right
It has been said that since LLMs predict human output, they will, if sufficiently improved, be quite human-- that they will behave in a quite human way.  

> Can you say more about what you mean by "Where can I find a post
As part of a counterargument to that, we could find evidence that their logical structure is quite different from humans. I'd like to see such a write-up. 

> Surely you would agree that if we were to do a cluster analysis of the cogn... (read more)

2Daniel Kokotajlo
OK, thanks. Your answer to my first question isn't really an answer -- "they will, if sufficiently improved, be quite human--they will behave in a quite human way." What counts as "quite human?" Also are we just talking about their external behavior now? I thought we were talking about their internal cognition. You agree about the cluster analysis thing though -- so maybe that's a way to be more precise about this. The claim you are hoping to see argued for is "If we magically had access to the cognition of all  current humans and LLMs, with mechinterp tools etc. to automatically understand and categorize it, and we did a cluster analysis of the whole human+llm population, we'd find that there are two distinct clusters: the human cluster and the llm cluster." Is that right? If so then here's how I'd make the argument. I'd enumerate a bunch of differences between LLMs and humans, differences like "LLMs don't have bodily senses" and "LLMs experience way more text over the course of their training than humans experience in their lifetimes" and "LLMs have way fewer parameters" and "LLMs internal learning rule is SGD whereas humans use hebbian learning or whatever" and so forth, and then for each difference say "this seems like the sort of thing that might systematically affect what kind of cognition happens, to an extent greater than typical intra-human differences like skin color, culture-of-childhood, language-raised-with, etc." Then add it all up and be like "even if we are wrong about a bunch of these claims it still seems like overall the cluster analysis is gonna keep humans and LLMs apart instead of mingling them together. Like what the hell else could it do? Divide everyone up by language maybe, and have primarily-English LLMs in the same cluster as humans raised speaking English, and then nonenglish speakers and nonenglish LLMs in the other cluster? That's probably my best guess as to how else the cluster analysis could shake out, and it doesn't seem very p

Thank you. But being manipulative, silly, sycophantic, or nasty is pretty human. I am looking for hints of a fundamentally different cognitive architecture

Yes, I agree with your point about most journalists. Still, I think well enough of the professors and AI developers that I mentioned to imagine that they would have a more positive attitude.

He is also maybe just somewhat annoying to work with and interact with

I have heard that elsewhere as well. Still, I don't really see that myself, whether in his public posting or in my limited interactions with him. He can be rough and on rare occasion has said things that could be considered personally disrespectful, but I didn't think that people were that delicate.

...  advocating for a total stop... which might make AI lab... less likely to pay attention to him. 

True, but I had thought better of those people. I would have thought that they cou... (read more)

2Ilio
You may wish to update on this. I’ve only exchange a few words with one of the name, but that was enough to make clear he doesn’t bother being respectful. That may work in some non delicate research environment I don’t want to know about, but most bright academic I know like to have fun at work, and would leave any non delicate work environment (unless they make their personal duty to clean the place).
JoshuaFox2514

At the time of Hofstadter's Singularity Summit talk , I wondered why he wasn't "getting with the program", and it became clear he was a mysterian:  He believed -- without being a dualist --  that some things, like the mind, are ultimately, basically, essentially, impossible to understand or describe.

This 2023 interview shows that the new generation of AI has done more than chagne his mind about the potential of AI: it has struck at the core of his mysterianism

the human mind is not so mysterious and complex and impenetrably complex as I imagined i

... (read more)
5Mo Putera
Gwern's comment makes it clear to me that Hofstadter has never been a mysterian. 

He was only a de facto mysterian: thought mind is so complicated that it may as well be mysterious (but ofc he believed it's ultimately just physics). This position is updateable, and he clearly updated.

Correct. That is why the original Turing Test is a sufficient-but-not-necessary test: It is meant to identify an AI that is definitively above human level.

What about a Turing Test variant in which such inquiries are banned?

That would be possible. Plenty of people don't know much about this topic. If you had such a judge, do you think actually doing a  Turing Test (or some variant) for ChatGPT is a good ideaa

1Yonatan Cale
"Doing a Turing test" is a solution to something. What's the problem you're trying to solve?

Nice! I am surprised we don't hear more about attempts at a Turing Test, even if it is not quite there yet.

That looks pretty close to the level of passing a Turing Test to me. So is there a way of trying a full Turing Test, or something like it, perhaps building on the direction you show here?

4Rafael Harth
It shouldn't. We know GPT is good at writing stories, so (for GPT), the hard part of the Turing Test is responding to the questions you choose. If you let GPT itself invent the questions, you're not measuring the hard part. Like, "write an essay about someone doing the Turing Test" is much easier for GPT than doing a Turing Test.

 Do you think there is a place for a Turing-like test that  determines how close to human intelligence it is, even if it has not reached that level?

2Rafael Harth
Probably, but I think figuring out exactly what you are measuring/trying to determine is a big part of the problem. GPT doesn't think like humans, so it's unclear what it means for it to be close. In some absolute sense, the "intelligence" space has as many axes as there are problems on which you can measure performance.

ChatGPT isn't at that level.

That could well be. Do you think there is a place for a partial Turing Test as in the Loebner Prize -- to determine how close to human intelligence it is, even if it has not reached that level?

1Vitor
I definitely think so. The Turing test is a very hard target to hit, and we don't really have a good idea how to measure IQ, knowledge, human-likeness, etc. I notice a lot of confusion, anthropomorphizing, bad analogies, etc in public discourse right now. To me it feels like the conversation is at a level where we need more precise measures that are human and machine compatible. Benchmarks based on specific tasks (as found in AI papers) don't cut it. (ep status: speculative) Potentially, AI safety folks are better positioned to work on these foundational issues than traditional academics, who are very focused on capabilities and applications right now.

I thought there was a great shortage of cadavers. How did they manage to get them for a non-medical school, indeed for use by non-students? Also, I am quite impressed that any course, particularly in the Bay Area, is $60 or free.

8Professora Em
There is a great abundance of cadavers, nor shortage at all.  The U.S. and India are the largest suppliers to the world's medical schools and research labs.  The U.S. obtains cadavers from willed body programs as well as unclaimed bodies.  The UCSF Willed Body Program will not allow their willed bodies to be used in any sort of program that does not ultimately benefit human anatomy education or research.  I run the lab that Alok is referencing where we create prosections for teaching anatomy.  Despite his casual remarks of wonder and curiosity, Alok has an extraordinary gift for focusing, working hard and accomplishing tasks.  His work in my lab is extremely valuable, and he sometimes brings a friend or three which makes my work as a 75 year old anatomy professor so much better and easier. These cadavers are used by anatomy professors to teach anatomy and physiology to pre-med and other allied health students in both the undergraduate and also post-baccalaureate courses (UC Berkeley Ext. and Merritt College).   
8DirectedEvolution
It sounds like a "good" cadaver is young, not too obese or too evidently diseased. And apparently although it's illegal to buy and sell organs for transplant (one of my obsessions, AMA), it's legal to sell whole cadavers. But considering that doctors in particular (FAR in excess of the public) consider any form of financial incentive associated with body parts "repugnant," I wonder if medical schools may not choose to purchase cadavers? If so, this would then mean that there's a more adequate supply of sold or "low quality" cadavers, but a shortage of donated "high-quality" cadavers, and so abundance can coexist with shortage depending on how one is willing to source them. I have to stress that I'm not confident this is true -- it's just something to look into if you're curious.

Nice! Is there also a list of AI-safety corporations and non-profits, with a short assessment of each where  feasible: goals, techniques, leaders, number of employees, liveness, progress to date?

3plex
There certainly should be! This is going to be one of the most important living documents. We're working on that, using Stampy and borrowing from this post, volunteers very welcome.
2Jozdien
I'm not sure if there's a live list, but there is this list of orgs, their research, other activities, finances, etc. that Larks has been putting out for the last couple years.
JoshuaFox*340

I organized that, so let me say that:

  • That online meetup, or the invitation to Vassar, was not officially affiliated to or endorsed by SSC. Any responsibility for inviting him is mine.
  • I have  conversed with him a few times, as follows:
  • I met him in Israel around 2010. He was quite interesting, though he did try to get me to withdraw my retirement savings to invest with him. He was somewhat persuasive. During our time in conversation, he made some offensive statements, but I am perhaps less touchy about such things than the younger generation.
  • In 2012, he
... (read more)

It seems to me that despite organizing multiple SSC events you had no knowledge that Vassar was banned from SSC events. Neither had anyone reading the event anouncement to the extend that they would tell you that Vassar was banned before the event happened.

To me that suggests that there's a problem of not sharing information about who's banned to those organizing meetups in an effective way, so that a ban has the consequence one would expect it to have.

Thank you. Can you point me to a page on FLI's latest grants? What I found was from a few years back. Is there another organizations whose grants are worthy of attention?

2Charlie Steiner
Right on time, turns out there's more grants - but now I'm not sure if these are academic-style or not (I guess we might see the recipients later). https://futureoflife.org/fli-announces-grants-program-for-existential-risk-reduction/?fbclid=IwAR3_pMQ0tDd_EOg_RShlLY8i71nGFliu0YH8kzbc7fClACEgxIo2uK6gPW8&cn-reloaded=1
9Charlie Steiner
I actually haven't heard anything out of them in the last few years either. My knowledge of grantmaking organizations is limited - I think similar organizations like Berkeley Existential Risk Initiative, or the Long-Term Future Fund, tend to be less about academic grantmaking and more about funding individuals and existing organizations (not that this isn't also valuable).

Thank you. Can you link to some of the better publications by Wentworth, Turner, and yourself? I've found mentions of each of you online but I'm not finding a canonical source for the recommended items.

Sure.

For a bit more funding information:

... (read more)

Thank you! That is  valuable. I'd love to get also educated opinions on the quality of the research of some of these, with a focus on foundational  or engineering research aimed at superhuman-AGI XRIsk (done mostly, I think, in MIRI, FHI, and by  Christiano), but that article is great.

Thank you.  Also Sloman's  The structure of the space of possible minds. All these are worthwhile articles, but I'd like to highlight Goertzel's discussion of kinds of embodiment.

Israel: "A monthly payment...Assistance finding a place to live...Free Hebrew classes...Probably quite a bit more." There is a basket of benefits: The point is to help in settling into the country.

In total, these are not a lot of money, but do ease the process of absorption.

1Josh Jacobson
Thanks for that link! I do think the lack of taxation can be quite a lot of money in certain circumstances.

Israel: "Waived taxes for 10 years!... (These may not be fully waived; maybe they’re just reduced. ...)." Reduced is a better way to say it. Various taxes, including income and property tax, are reduced. 

The 10-year period you are thinking of is on income generated investments. from outside Israel. If you are American, however, that does not much help as the US government will then grab the tax (as there is then no "double taxation" issue for those 10 years).

1Josh Jacobson
No double taxation issue then would be quite the detriment to the appeal of Israel from a taxation perspective. Do you happen to have a source or more info about that? Thanks for noting this information with more granularity than I provided / had.

"[p]rograms in...Israel require your physical presence in the country." You are expected to be moving here, but preliminary registration steps (which that sentence is mostly talking about) are usually done from outside Israel.

1Josh Jacobson
Thanks for noting this. FWIW, I think that sentence was talking about the entire process, not just preliminary steps.

Where did you see that "Israel... tax[es] citizens living anywhere in the world"? I've never heard of that.  Various sources state that  it  only the United States and Eritrea.

1Josh Jacobson
I'm not sure; it does look like this was inaccurate! Thanks for correcting it.

Delegation of authority -- "management by objectives" -- is the newer style of command since World War II.  The lower-level commanders are given goals but are allowed to choose how to achieve them. This was only enabled by improved communication and control measures.

5ChristianKl
In Germany that would be Auftragstaktik which is older then WWII (the German Wikipedia gives 1866).

Thank you, organizers! Amazing virtual space and  event! I hope that something similar can be done often.

3nick lacombe
Thank you very much to the organizers and everyone that was present! ❤️ I had a nice time and I loved the online platform/system I would not have been able to meet some of the people I met if it weren't for the online event
Answer by JoshuaFox20

If you have goals that justify working hard, like the satisfaction of a job well done; or hopes for a professor position; then do  that. 

If your goals justify not working hard, like enjoying your doctoral stipend while gliding through and  then moving to some other profession, then do that.

New event post with the new online-events feature.

New post for the new online-event feature

New posting of this using the new online-events feature.

Thank you for the correction. When I joined one Gather Town event it was so hard to use that I can't say much about how it worked.

(This is the duplicate of a regular post as LessWrong does not support non-geographical events.)

to always have a set topic and if possible also presentation prepared.

These talks all start with a speaker. Do you mean we should have a set topic for the videochat rooms afterwards? It seems that the main event should provide a topic of conversation, though I admit that even then, breaking the ice is often difficult.

> Are you tracking where attendees come from? (% email, % type of Facebook group,  
Not precisely, but the  email list (opt-in and specific to this meetup series)  seems the best. Back when SlateStarCodex was live, the link th... (read more)

I like Icebreaker, but I noticed

I only participated in one Icebreaker. It would be good if someone who organized one of these wants to write up tips.

 

gather town? This simulates what you want pretty well, and deals with the issues stemming from 100 people in the same call trying to talk together.

Gather Town and Mozilla Hubs have scalability problems, and with Gather Town I kept getting stuck. But more importantly, I believe in seeing people's faces to built social contact. Having multiple videochat rooms solves the scale problem.

1adamShimi
I'm confused by your last sentence: you can see the video for other people in gather town. It's just that you control the radius at which you interact with people, and you have a physical position.

As LessWrong does not support non-geographical events, this event post is also a regular post.

As LessWrong does not support non-geographical events, this post is also an event post.

This duplicates a  regular LW post (since lesswrong.com does not support non-geographical meetups).

This duplicates an event post (since lesswrong.com does not support non-geographical meetups).

This is a duplicate of the ordinary post, as lesswrong.com does not yet support non-geographical events.

This is a duplicate of the event posting, as lesswrong.com does not yet support non-geographical events.

I was thinking you should do a game like Hofstadter describes in  "The Tale  of Happiton" and the Platonia dilemma. Avoid destruction by  cooperation, even if it is  without coordination.

In coordination with the LW admins, this post also appears as an event-post, as LW does not support non-geographical events.

In coordination with the LW admins, this event-post also appears as a regular-post, as LW does not support non-geographical events.

(After discussion with the LW admins, this regular post presents this event-post, because events cannot be online/non-geographic.)

(After discussion with the LW admins, this also appears as a post, because events cannot be online/non-geographic.)

(This Event also appears, in coordination with LW admins, as a Post since online nongeographical Events are not supported.)

(This Post also appears, in coordination with LW admins, as an Event, since online nongeographical Events are not supported.)

Load More