All of Brendon_Wong's Comments + Replies

This is an interesting point. I also feel like the governance model of the org and culture of mission alignment with increasing safety is important, in addition to the exact nature of the business and business model at the time the startup is founded. Looking at your examples, perhaps by “business model” you are referring both to what brings money in but also the overall governance/decision-making model of the organization?

Great article! Just reached out. A couple ideas I want to mention are working on safer models directly (example: https://www.lesswrong.com/posts/JviYwAk5AfBR7HhEn/how-to-control-an-llm-s-behavior-why-my-p-doom-went-down-1), which for smaller models might not be cost prohibitive to make progress on. There’s also building safety-related cognitive architecture components that have commercial uses. For example, world model work (example: https://www.lesswrong.com/posts/nqFS7h8BE6ucTtpoL/let-s-buy-out-cyc-for-use-in-agi-interpretability-systems) or memory syst... (read more)

I appreciate your thoughtful response! Apologies, in my sleep deprived state, I appear to have hallucinated some challenges I thought appeared in the article. Please disregard everything below "I think some of the downsides mentioned here are easily or realistically surpassable..." except for my point on "many-dimensional labeling."

To elaborate, what I was attempting to reference was QNRs which IIRC are just human-interpretable, graph-like embeddings. This could potentially automate the entire labeling flow and solve the "can categories/labels adequately express everything?" problem.

This approach is alignment by bootstrapping. To use it you need some agent able to tag all the text in the training set, with many different categories.

Pre GPT4, how could you do this?

Well, humans created all of the training data on our own, so it should be possible to add the necessary structured data to that! There are large scale crowdsourced efforts like Wikipedia. Extending Wikipedia, and a section of the internet, with enhancements like associating structured data with unstructured data, plus a reputation-weighted voting system to judge contributions... (read more)

This is a fantastic article! It's great to see that there's work going on in this space, and I like that the approach is described in very easy to follow and practical terms.

I've been working on a very expansive approach/design for AI safety called safety-first cognitive architectures, which is vaguely like a language model agent designed from the ground up with safety in mind, except extensible to both present-day and future AI designs, and with a very sophisticated (yet achievable, and scalable from easy to hard) safety- and performance-minded architectu... (read more)

One possibility that I find plausible as a path to AGI is if we design something like a Language Model Cognitive Architecture (LMCA) along the lines of AutoGPT, and require that its world model actually be some explicit combination of human natural language, mathematical equations, and executable code that might be fairly interpretable to humans. Then the only potions of its world model that are very hard to inspect are those embedded in the LLM component.

Cool! I am working on something that is fairly similar (with a bunch of additional safety consideratio... (read more)

Yep, I agree that there's a significant chance/risk that alternative AI approaches that aren't as safe as LMAs are developed, and are more effective than LMAs when run in a standalone manner. I think that SCAs can still be useful in those scenarios though, definitely from a safety perspective, and less clear from a performance perspective.

For example, those models could still do itemized, sandboxed, and heavily reviewed bits of cognition inside an architecture, even though that's not necessary for them to achieve what the architecture working towards. Also... (read more)

I implied the whole spectrum of "LLM alignment", which I think is better to count as a single "avenue of research" because critiques and feedback in "LMA production time" could as well be applied during pre-training and fine-tuning phases of training (constitutional AI style).

If I'm understanding correctly, is your point here that you view LLM alignment and LMA alignment as the same? If so, this might be a matter of semantics, but I disagree; I feel like the distinction is similar to ensuring that the people that comprise the government is good (the LLMs i... (read more)

Do you have a source for "Large labs (OpenAI and Anthropic, at least) are pouring at least tens of millions of dollars into this avenue of research?" I think a lot of the current work pertains to LMA alignment, like RLHF, but isn't LMA alignment per say (I'd make a distinction between aligning the black box models that compose the LMA versus the LMA itself).

Have you seen Seth Herd's work and the work it references (particularly natural language alignment)? Drexler also has an updated proposal called Open Agencies, which seems to be an updated version of his original CAIS research. It seems like Davidad is working on a complex implementation of open agencies. I will likely work on a significantly simpler implementation. I don't think any of these designs explicitly propose capping LLMs though, given that they're non-agentic, transient, etc. by design and thus seem far less risky than agentic models. The propos... (read more)

Have you read Eric Drexler's work on open agencies and applying open agencies to present-day LLMs? Open agencies seem like progress towards a safer design for current and future cognitive architectures. Drexler's design touches on some of the aspects you mention in the post, like:

The system can be coded to both check itself against its goals, and invite human inspection if it judges that it is considering plans or actions that may either violate its ethical goals, change its goals, or remove it from human control.

My experience on Upwork is actually the same as yours! In our tests of the platform, it appears to be very difficult to find jobs due to the intense competition. I was unpleasantly surprised at first when I saw how difficult it was to earn money on Upwork as a new user. However, that was the whole point of the initial tests we did, so we expanded and have still been expanding the program to encompass other forms of virtual work that pay reliably and still have room to grow. Upwork will be a minor or non-existent part of our program.

If my program was just on Upwork, then I would be inclined to side with your analysis. Thankfully, it's not.

I think I understand the point: hypothetically, this program would take work away from people more in need, possibly even making the world worse off because of that. But if I magically made half of the virtual workforce disappear, then the half of the people that were removed would be really poor and the other half would be twice as rich. But is that creating more good? No, because the richer half would not need the money as much as the poorer half. If I added more people who were earning less money before being added then I am creating a net good, and tha... (read more)

Well the total pool of work available for everyone is imperceptibly decreased in the short run, not aversely affecting anyone to any significant degree, while giving more of the poor who really need the money work opportunities... Is employing several dozen more people a small net good? I guess it's a matter of opinion.

We are continuing our search for similar projects, thank you for your suggestion. I hope that we have not missed any pitfalls, but like Strangeattractor wrote, we are indeed doing tests of the concept in various stages of development, and this project is kind of a pilot in and of itself, so hopefully we can catch anything we might have missed.

'In a charitable way" meaning good for the people. Just because there are for-profit companies out there doing this doesn't mean they are doing what is best for the people, they are distributing wealth, but also keeping a lot of it for themselves. A charitable venture would give most of the profits to the people involved, and this project also involves providing many things to people like internet and computer access, training, opportunities, something a lot of freelancers have to acquire for themselves in developing countries. It is very difficult fo... (read more)

Thank you, that is one of the markets we are looking to branch out to.

I did not throw every detail into the video and fundraiser/my post in LessWrong, that is correct... I do think I described the jist of it. I explicitly stated that funds will go towards providing computer and internet access, training given by staff, and opportunities that staff have to find. As implied, expenses will go towards computer acquisition, internet, and helping staff implement various facets of the project. I could have explained each and every detail, but it would be too long for the target audience to read. The campaign is not noticeably more ... (read more)

MTurk employs a lot of people in developed countries. I have read they are starting to reject Indian based workers because of poor work quality. I can find employment for people who can provide a similarly high standard of work relative to workers in more developed countries, but who need the income more. Member participants would otherwise have had difficulties joining, say, MTurk because of a lack of computers, internet access, proper guidance, training... I don't think there are any companies helping freelancers find work because it's not very profitable, and yet there is a great need to reach people who are not working to their potential.

I'm having trouble seeing how a for-profit corporation would create more good and be a more effective structure in this case. A non-profit organization can operate without income tax and attract donations which can be tax-deductible to donors. A for-profit organization could get investment capital, but I think it's highly unlikely I would be able to find any interested investors, and it otherwise performs worse compared to a non-profit with the same business model.

The way I see it, making the project a nonprofit allows it to better compete with for-profit companies because of tax-advantages. It can also get donations. A for-profit corporation has the advantage of attracting investments from people hoping to make a profit, but I am quite sure that I would not be able to attract large sums of investment capital. That pretty much gives starting this program as a nonprofit the only logical choice.

Regarding your point about re-compensation, I don't think I cannot extract the value, it will just be difficult to pay myself... (read more)

Belizeans would probably be competing with wealthier people for work because their high level of English mastery allows them to compete for more advanced positions. The websites I mentioned have many workers from more developed countries. For example, half of MTurk's users are from the United States.

Many people in developing countries do not have access to the technology needed to participate in virtual employment, so we will provide computer and internet access. We will be doing marketing in a way, yes, although it is guidance and training as well. In the future, we will move on from guiding people through using third party systems to directly selling virtual employment services, which should be much more profitable.

Thank you for your suggestions. I have in fact surveyed people and organizations in Belize. The general consensus is that there are a lot of people who are unemployed or working for very low wages, and getting higher paying employment would improve their standard of living. You mentioned a small scale pilot, we have actually run many such pilots, which is how we found that it would be possible to help people earn around $3 USD an hour. We are currently working on remote testing of our program before actually sending staff to Belize.

There is definitely no prominent implementation of this concept and its related variations. Many nonprofits offer job training and give people computer and internet access, but starting what is essentially a virtual employment company to help people is not something I have heard about before, hence this program. It is possible that this idea was not implemented before in a charitable way because people start virtual employment companies for for-profit purposes, and those companies are very successful. As to the idea of connecting the impoverished with virtual employment services, it is possible many people are not aware of virtual employment services and thus have not implemented the idea.

The venture could be profitable, yes. Would it generate massive amounts of income? That is also possible. I did not consider a for-profit version of the idea because the project itself was supposed to be charitable in nature. I am considering starting a for-profit branch of this idea, and would be open to hearing other people's ideas and motivations. Is your motivation and other's in getting involved in a for-profit implementation of this idea to earn money?

To elaborate more on profits, the initial implementation of this idea might not be incredibly profit... (read more)

Thanks for your question! This particular project is charitable in nature and would probably require funding to get off the ground and expand more rapidly. Since it is not expected to attract for-profit support, especially because it would probably not be a particularly profitable venture, most funding would probably come from people/organizations with non-profit motives. People/organizations with non-profit motives generally only donate to nonprofits, which have a better public image and are more trusted to pursue altruistic goals like donors expect. We c... (read more)

Thank you for your help! I have edited my post with additional information. My audience is a general youth audience, think of promoting content to an entire high school, with "average teenagers" and people that might be more interested in content. Of course, some people will be more interested than others, so a wide variety of recommendations for different interest groups is better. I'm primarily looking for books that promote ethical/altruistic behavior, I'm not sure if any of your beforementioned recommendations do so.

Thanks for your recommendations! I've edited my post to make it more specific, I am looking for content that promotes altruistic behavior, either fiction or nonfiction.

Answered all questions, I hope I helped!

I'm very curious to see how the monetary reward works out.

Thank you very much. How sure are you that top colleges will trash unschooling applications?

Since you asked me about the most effective technique I've used since this point, I started using the pomodoro technique with Beeminder. I have experienced a very dramatic increase in productivity. Thank you very much!

I don't think I've accomplished enough at this point in my life to go to college immediately.

If I unschooled, I would be engaged in many other activities with social support like internships, classes, and possibly even school extracurricular activities.

Historically, I have had no problem getting things done with no social support.

It's true, my parents kind of forced me into it...

You are right, I should try to free up more time while taking the conventional and safer route.

My problem is that the akrasia seems to be partially caused by staying in a highly structured environment. I don't have much trouble doing things I believe are beneficial towards my goals.

I currently believe if I pursued option 2 I could get into a top college just like I would have done if I stayed in high school but more useful things would get done.

If this belief is false, then my akrasia would be slightly reduced.

Yes, but with all those activities I listed above I have minimal free time during the school year. I do have much more time during the summer though.

I believe the pomodoro technique had me accomplishing many tasks for one day, then it failed. It failed because I failed to start using the pomodoro method itself, I just procrastinated on starting it. I also got distracted while working. I either stopped working and never got on track again, or I forgot about the rules about distraction (record it, apply the 3 steps) and wasted a lot of time. Over time I just forgot about it. Thanks for reminding me, I'll give it another go because it was so close to working and I can try different motivational techniques... (read more)

Unfortunately, Akrasia still strikes in the morning and I don't always have the motivation or energy to finish everything. But that's only with minimal sleep due to procrastination the night before.

I have to see if I can get my parents to agree...

So I leave high school... and then what comes next on the path to world optimization? :)

Huh. Ask Eliezer about that :)

I think that's the best guaranteed way to improve the world. There is almost no uncertainty. But I'd rather not subject myself to decades of monotonous work, especially since there are so many other organizations and individuals who could create an impact thousands or even millions of times more than mine.

I was thinking more along the lines of actually working at a nonprofit, starting businesses to raise money, something at least a little higher impact then earn several hundred thousand and donate it.

With those updated plans, which of my three options (or neither of them) are the best?

Thanks for your helpful replies!

I've actially spend years trying to fix this problem, with little success. I've tried multiple books, read about every productivity system ever invented, tried thousands of articles over hundreds of websites... No luck.

Recently, I've been looking into Akrasia on Less Wrong because I thought the suggestions might finally have an impact. I memorized The Motivation Hacker (along with all the techniques shared in the Procrastination Equation). I've also tried PJ Eby's materials. I found that unfortunately it had no impact on my... (read more)

Heh. I wake up at 7:00 AM, attend a full 7 periods with all the Advanced Standing classes I can, and leave at 3:15 PM. From there, I go to cross country, taking my time until 6:00 PM at the earliest. Then I eat dinner, shower... Then it's 8:00 PM already and all my homework is there waiting. Then there are family activities, chores, distractions, and other projects I need to do thrown in. Did I mention I have a serious akrasia problem? Then I sleep at like 12:00 AM... Not that much time if you ask me.

College does sounds pretty useful, so I guess the question is whether I should leave high school, unschool for 3 years, then reapply to college. If that does not significantly reduce my college admissions potential it would seem like it is the most strategic thing to do.

Carefully putting off assignments sounds like a potential solution, but usually assignments are assigned one day and due the very next. I have around 4 hours to get an average of 2 hours of homework done on school nights. But I fail to shift into homework mode which caused me to write this article at 10:00 PM last night, get 5 hours of sleep, then wake up early to finish studying because it seems easier in the morning.

Thanks for the clarification.

I'll focus on resources rather than topics, and collect crowd opinion on resources.

I'd call it success. Really, I am more afraid of the opposite situation: too few people caring enough to comment; because then I wouldn't know what to do. If there are too many comments, you could for example collect the resources and make a poll. Or just start another discussion a month later, where the first comment would contain the poll about the resources recommended in the previous discussion. Or anything else. The big problem is IMHO if

Thanks for suggesting concrete actions, I'll go ahead and post it ASAP.

Questions before I start (thanks in advance)!

1. What's better, recommending a resource to improve something or recommending a specific topic to improve with resource suggestions as reply's? Ex. Watch The Blueprint Decoded to learn PUA vs improve PUA and add resources as replies.

2. What do you mean by collecting data? Do you mean collecting the self-improvement resource suggestions themselves, or opinions/ratings/votes on the suggestions?

3. What if there are too many comments on the disc

It seems like people find discussions more rewarding than posting to a wiki. There could be weekly discussions on the many aspects of self improvement, and then those ideas could be posted on a wiki for organization and further updates.

Do you think using a separate wiki is a good idea? It seems like the LW wiki is not being used for collecting self-improvement articles, and a new wiki with a separate purpose, community, and article format might be better. After all, the current wiki is organized only for rationality articles, and changing the layout and article format might cause some conflict and confusion.