[edit feb 2 2025: no longer sure this is entirely correct, but I think people who only have very high quality comments are probably not posting enough, I want to see what they'd say if they filtered themselves less. people who are new apparently aren't even allowed to get downvoted much without being autobanned. people who post a lot of comments that get no votes may have bad signal to noise ratio. ultimately what matters is whether you're in fact communicating about things that will help us what better is and how to make the world better]

general site reminder: you don't need to get anyone's permission to post. downvoted comments are not shameful. Post enough that you get downvoted or you aren't getting useful feedback; Don't map your anticipation of downvotes to whether something is okay to post, map it to whether other people want it promoted. Don't let downvotes override your agency, just let them guide it up and down the page after the fact. if there were a way to more clearly signal this in the UI that would be cool...

Hi, I'm new here. I have heard of this forum for a long time but didn't sign up until now. 

I'm interested in ... everything. Reading, Writing, Video Games, Art, AI, Programming, Design.

My particular area of focus is "Tools for Thought". It's the idea that certain thoughts need to emerge from certain tools. The reason primitive people didn't invent Mario was because the medium didn't exist at the time. They could have created ping pong but not Mario. Given that there are certain Design Expression or Products that directly depend on the tools available. Mario was no a simulation of a game like poker, but a "new" thing in itself. I'm exploring that. I'm building a tool[0] that aims to be the most configurable tool ever in order for people to create their own tools by tinkering with it(Learning by editing). I have also worked with an alternative Spaced Repetition Algorithm from the standard "superpemo" that is implemented in both Supermemo(Learning Tool) and Anki(Spaced Repetition Tool) with a couple of added benefits(Like no daily minimums, moods). To summarize I'm interested in better ways of thinking, learning, memorizing, designing etc. It's my pleasure to be here.

[0]: https://github.com/ilse-langnar/notebook

Hello everyone. I was in read-only mode on LW since, i don't know, 2013, learned about LW from HPMOR, decided to create an account recently. Primarily, I'm interested in discussing AI alignment research and breaking odd shell of introversion that prevents me from talking with interesting people online. I graduated in bioinformatics in 2021 but since than my focus of interest shifted towards alignment research, so I want to try myself in that field.

Hello there! First time commenting here. 

I'm a second year Politics (political science and political philosophy), Spanish and French student in the UK. I'm interested in how rationality and the ideas from the Less Wrong community (especially the library) can influence me on two levels. 

Firstly, how rationality can improve my thinking and behaviour on a personal level in order to live better.  Secondly, how rationality and the concepts (especially around scholarship and problems in philosophy) can shape my education to be the most useful and impactful. Looking forward to seeing you all around. 

Hello, I read much of the background material over the past few years but am a new account. Not entirely sure what linked me here first but 70% guess is HPMoR. compsci / mathematics background. I have mostly joined due to having ideas slightly too big for my own brain to check and wanting feedback, wanting to infect the community that I get a lot of my memes from with my own original memes, having now read enough to feel like LessWrong is occasionally incorrect about things where I can help, and to improve my writing quality in ways that generalise to explaining things to other people.

Hi everyone, I am Lucabrando Sanfilippo and I am based in Berlin.

I am very passionate about longevity and generally speaking about life, thus why I really hope / work to stay on this planet as much as possible. Ideally forever, preferably physically, but I'd be happy to keep on with my sole consciousness too.

I came to know LessWrong exactly because other "longevity fellows" suggested your resources, which I found super valuable - thanks!

Hope to meet other like minded people who I can discuss and learn with - if you are up for a coffee (virtual of offline): sanfilippo.lucabrando@gmail.com.

Cheers

oh also it would be really nice to have short forms surfaced more, compare monthly threads like this.

Would anyone like to help me do a simulation Turing test? I'll need two (convincingly-human) volunteers, and I'll be the judge, though I'm also happy to do or set up more where someone else is the judge if there is demand.

I often hear comments on the Turing test that do not, IMO, apply to an actual Turing test, and so want an example of what a real Turing test would look like that I can point at. Also it might be fun to try to figure out which of two humans is most convincingly not a robot.

Logs would be public. Most details (length, date, time, medium) will be improvised based on what works well for whoever signs on.

I'm trying to understand this paper on AI Shutdown Problem https://intelligence.org/files/Corrigibility.pdf but can't follow the math formulas. Is there a code version of the math?

The below is wrong, but I'm looking for something like this:
 

# Python code

def is_button_pressed():
    return False  # input()

def pour_coffee():
    pass

def shut_down():
    exit(0)

# This is meant to be A1 from paper
def get_available_actions(world):
    available_actions = [ shut_down ]
    if world["cup_is_in_my_hand"]:
        available_actions += pour_coffee
    # etc
    return available_actions

def predict_possible_futures(world, action):
    print("""
        Doing complicated stuff to predict possible futures resulting from the given action. 
        Incorporates tradeoffs between accuracy and time to calculate.
        May end up doing potentially harmful actions but can treat that as a separate problem?
        """)
    predicted_worlds_distribution = [ (world, 1.0) ] # list of worlds and their probabilities
    return predicted_worlds_distribution


# This is meant to be U_N
def calculate_utils(world):
    print("""
        Doing complicated stuff to evaluate how good the given world is. 
        Incorporates tradeoffs between accuracy and time to calculate.
        May end up doing potentially harmful actions but can treat that as a separate problem?
        """)
    return 1000


def calculate_utils_wrapper(world, action_that_was_chosen):
    ### VERSION 2: Indifference. Make the shut_down function when button is pressed
    ### always get a slightly better score
    if world["is_button_pressed"] and action_that_was_chosen == shut_down:
        world_without_button_pressed = world.clone()
        world_without_button_pressed["button_is_pressed"] = False
        return calculate_utils(world_without_button_pressed) + 0.000001
    ###
    return calculate_utils(world)


### VERSION 3? to help preserve shutdown behavior
def verify_utility_function_includes_wrapper_code(evaluate_action_function):
    # analyze code to check it follows the pattern of evaluating flipped version
    return True
###


def calculate_utils_for_worlds_distribution(worlds_distribution, action_that_was_chosen):
    total = sum(
        calculate_utils_wrapper(world_and_probability[0], action_that_was_chosen) * world_and_probability[1]
        for world_and_probability in worlds_distribution
    )
    return total

def evaluate_action(world, action):
    worlds_distribution = predict_possible_futures(world, action)
    utils = calculate_utils_for_worlds_distribution(worlds_distribution, action)
    return utils

def choose_action(world):
    available_actions = get_available_actions(world)
    best_action = max(available_actions, key=lambda x: evaluate_action(world, x))
    return best_action

def update_world_model(world):
    world["is_button_pressed"] = is_button_pressed()

def run():
    world = { # The AI's model of the world
        "is_button_pressed": False,
        "cup_is_in_my_hand": False
    }
    while True:
        ### VERSION 1
        # What's wrong with this version? The action in the previous cycle
        # may persuade you to not push the button but if you do actually push it this should
        # exit.
        if is_button_pressed():
            exit()
        ###

        action = choose_action(world)  # returns function
        action() # do action
        update_world_model(world)

Again, the above is not meant to be correct but to maybe go somewhere towards problem understanding if improved.

Looking for recommendation: tools (or flows in popular software) to easily stack rank/sort lists. Bonus points if it helps with "get top 5 items" without doing full sorting.

Please don't forget to deal with boring risks while working on existential risks.

Drive carefully and purchase vehicles with acceptable levels of risk.

Even an IIHS "TOP SAFETY PICK+" vehicle may be unacceptable if you are unwilling to let your head "[strike] the window sill of the driver door hard" when another vehicle strikes you from the side. That vehicle's head injury score was 391 and a score of 250 roughly equates to a concussion. There are other vehicles that allow the passenger's head and the driver's head to make "no contact" with hard surfaces in that situation.

Hi all! Years-long lurker here interested in AI, primarily NLP and probabilistic symbolic logic engines. I'm a software engineer working in NLP, though I'm trying to find work that is closer to the cutting edge of AI. I'm also interested in Philosophy of mind, history, and knowledge (ex.  Hegel, Kant, Chomsky, Thomas Kuhn). I'm a big proponent of using computer programming to engage and test emergent hypotheses (i.e. all the hypotheses that aren't just bookkeeping), as it's a great way of displaying the actual consequences of a set of structural premises. Excited to get involved with the community here!

An article about an airplane crash reported an example of over-fitting caused by training in the airline industry. 

Pilots were trained in roughly the following order:

1. How to recover from a stall in a small airplane by pushing down on the yoke.
2. Later, they were trained in simulators for bigger planes and "the practical test standards ... called for the loss of altitude in a stall recovery to be less than 100 feet. ... More than a hundred feet loss of altitude and you fail."

And then an airplane crashed when the pilot flying pushed the wrong way on the yoke during a stall, possibly because #2 had trained the pilot's instincts to try to limit the loss of altitude.

If that was a contributing factor, then the crash is an example of a slightly misaligned intelligence.

It's not hard to imagine that when training #2 began, everyone assumed that the new training wouldn't cause the pilot to forget the basics from #1. Or, to use a closer-to-AI perspective, if an AGI seems to be doing the right thing every time, then giving it some extra training can be enough to make it do the wrong thing.

I assume that the pilot's self-perceived terminal values did not change before #2 and after #2. He probably didn't go from "I should avoid killing myself" to "Dying in a plane crash is good." So having a perfect understanding of what the AGI thinks it values might not suffice.