Electing a Speaker does you no actual good if they can't, in office, maintain the confidence of a majority of the House, and assemble that majority into a coalition to pass legislation. If they were elected without genuine majority support they would be ineffective and potentially quickly removed by a vote to vacate.

So while the current mess is embarrassing and annoying, it's mostly a result of the fragmented factions and there not being a majority legislative coalition, moreso than the particular mechanics of how you hold a Speakership election.

I've thought about this for a bit, and I think that the constitution imposes many constraints on the shape and constituting elements of the House that aren't anywhere close to optimal, and the best thing would be to try to apply lots and lots of mechanism design and political science but only to the House (which is supposed to catch the passions of the people and temper them into something that might include more reflection).

A really bad outcome would be to make a change using some keyword from election theory poorly, and then have it fail, and then cause there to be a lot of "no true X" debates for the rest of history.

You don't want to say that the failure of "X applied to the House" was the fault of X instead of some other nearby problem that no one wanted to talk about because it seemed even more stupid and sad than the stupid sadness of status quo House Speaker elections.

So the best I can come up with for the House given time constraints (that I think would cause the House to be the "part of the US government that wasn't a dumpster fire of bad design") would require a constitutional amendment to actually happen:

<proposal>

A) The full proposal envisions there being initial chaos after the proposal is adopted, such that a really high quality algorithms for Speaker selection becomes critical for success rather than just "a neat little idea". Also, we intentionally buffer the rest of the government from the predicted chaos while "something like real democracy, but for the internet era" emerges from the "very new game with very new rules". The federal government will take over running the elections for the House. Not the Senate, not the President, and not any state elections. There have to be two separate systems because the changes I'm proposing will cause lots of shaking and there has to be a backup in place. The systems I'm proposing might not even have the same sets of voters if some states have different franchise and voter registration processes and laws. Some people might be able to vote in the "federal house elections" but not state or "old federal" elections and that's just how it is intended to work. The point here is partly to detect if these diverge or not (and if they diverge which is better). 

Can states grant voting rights to AIs? That's an open question! Voters in both system will have a state party registration and a federal party registration and everyone in the US who is either kind of voting citizen (or both kinds) will have a constitutional right to be in different parties on different levels. The House's initial partisan chaos (like in the plan I'm proposing the Senate Republican Party and the House Republican Party wouldn't even be a single legal entity even if they both use the word "Republican" in their name, and will only align if that's what the people in the two things strongly desire and work to make real) and that will almost certainly make it much much harder to "validly or sanely use FPTP" to pick a Speaker... so...

A1) The election for the Speaker will internally occur within the house using secret ballot ranked pairs (but with anti-cheating measures from cryptography so that if cheating happens in the counting then any member of the House will be able detect "that cheating occurred" and release their data to prove it). Part of the goal here is that House Reps will be F2F familiar to many voters, and so many voters can believe "that Rep is honest, and saw cryptographic math, that says the Speaker is really the speaker" and then they will know who the valid Speaker is by that method (like part of the goal is to make legitimacy destroying misinformation very hard to pull off in the near future where AI powered disinformation attacks attempt to destroy all democracies by this method). 

If a circle in the voting shows up (that is, if there is no Condorcet Winner for Speaker at first) and if the Ranked Pairs resolution for that produces a tie (it could happen) then re-run the Speaker election over and over until it goes away, like how the Pope election runs until they agree based on pure tiredness (or being spoken to by the Holy Spirit or whatever it is that causes people to vote better the second time). The plan is to have every election always produce a sort of a Prime Minister who represents the entire country in a central way. The hope is that after several election cycles things settle down, and the Senate and the Presidency start to become somewhat vestigial and embarrassing, compared to the high quality centrist common sense that is served up regularly by the Speaker over and over and over. 

If the experiment goes well, we hope for an eventual second constitutional amendment to clean things up and make the US a proper well designed Parliamentary government with the Presidency and Senate becoming more symbolic, like the British House of Lords or the British Monarch.

A2) We don't know what parties will even exist in advance. Thus the Speaker needs personal power, not just "the loyalty of their party". They get some power to control how the votes go, like Speakers have traditionally had, but now added to the constitution explicitly. The federal parties still have some power... they get to generate a default preference ballot for the voters in that party to start out with. Its a UI thing, but UIs actually matter.

B) Super districts will be formed by tiling the country with a number of "baby" house districts that is divisible by 5, and then merging groups of 5 such baby districts into super districts (even across state lines if necessary (so Wyoming is just gonna be one big baby district every time for a while)). State governments (where they have latitude) set the baby district shapes and the federal level chooses how to merge them. Then the US federal election system will run IRV proportionally representative elections within each super district to select 5 house reps from each super district.

C) The House is supposed to act very very quickly. It was given a 2 year cycle before telegrams existed and it is supposed to be "the institution that absorbs and handles the passions of the masses of voters who maybe should change their minds about stuff sometimes". It is totally failing to do this these days. There is too much happening too fast. To increase the speed at which things operate (and to fix the problem where elections can leave the House itself ungovernable sometimes (and how can something that can't govern itself hope to effectively govern anything else!)) we add "no confidence" judgements, that can be applied to the House such that its elections can happen on closer to "an as-needed to deal-with-the-Singularity" sort of timescale... so... much much faster... gated mostly by something like "the speed at which humans can handle a changing political zeitgeist in the age of modern media"...

C1) A "top-down no confidence" can be initiated by a majority role call vote of the Senate, first giving the warning, then waiting 3 months, and then the Senate can hold a 2/3s private ballot vote to agree to go through with it, and then the President has 3 days to either veto (restarting the clock such that the Senate can try again with a secret ballot in 3 months) or pass it. If the Senate has a majority persistently voting in their real names (but getting vetoed by the President or the 2/3s vote) then the third such vote (taking 2 months and 6 days to occur on the schedule where the 51% votes instantly and the 67% and President drag their feet) shall also be a way to trigger a "top-down no confidence" vote. It is good form to call these Bertolt Brecht elections.  If the Senate causes a top-down snap election, they can redo the federal portion of the districting (change which baby districts merge into which super district) as part of the reboot, in the hopes of getting a nearly completely new cast of characters in the House. The House would obviously still be representative (maybe too representative of an insane electorate?)... but the Senate can hope for "new specific persons raised up by The People".

C2) The Speaker gains the constitutional power to call an "internal no confidence" election. In games of Chicken vs the entire rest of the House, the Speaker should hopefully just win and have the entire House swerve. However, they have to try to rule the House for the first 2 months after the election and then they have to give a "7 day warning" in advance of the failure being legible and decisive. Part of the fear is that AI systems might attack the minds of the voters to intentionally cause the elections to crash over and over, if the minds of the voters actually start to matter to the real shape of the government. The 2 month thing puts a circuit breaker in that loop. So the Speaker can decide and make their threat unilaterally that the House deserves "no confidence" after 2 months from an election and ultimately and internally decide 7 days later about whether to kick off the next election. Then a snap election would happen as fast as pragmatically possible, probably using the internet and open source polling software that the NSA (and all the crazy programmers around the world looking at the code) say can't be hacked?

C3) If a "bottom-up no confidence" has been indicated by a (i) majority of voters overall expressing "no confidence" specifically in their own rep using the federal election system's real time monitoring processes, and (ii) a majority of reps have lost the confidence of the specific people they are supposed to represent, then a snap election shall occur as fast as pragmatically possible. The software for soliciting info from the voters would be part of the voting system, and also open source, and should be audited by the NSA and so on. Each voter, running a voting client, should get a digital receipt that tells them EXACTLY who their ballot caused them to be represented by. They should also know how far down that person was down in their list of preferences from the top to the bottom. They are not allowed to call no confidence on who they ended up with as their rep for at least 2 months (just like how the Speaker can't do that). Also the people who do this have to do it in two motions, first "warning" their candidate, second "following through" at least 7 days later.

C4) Default elections using the federal election system will happen for the House at the same time as the President and/or the Senate are holding their elections using the state election system but only if there hasn't been a "no confidence" snap election in the last 6 months. No convened elected House shall go longer, without any election, than 30(=6+24) months. Note that since the federal election system will be open source, it should be quite easy for the states to copypasta it, if they want (with any tweaks, if they want). The voters will get to see for themselves which layer of government is the bigger shitshow, in a head-to-head competition, and judge accordingly.

D) There will be a local town hall style system inside each superdistrict, with federal funding to rent the physical venue in a stadium or an auditorium or a conference center or whatever, and federal internet hosting for the video and transcripts from the proceedings, where the "popular also rans" from each superdistrict get privileges to ask questions in hearings with the superdistrict winners when the winners are visiting home from DC. These events will occur 1 month after every election, and also whenever a no confidence warning as been issued by the Senate or the Speaker, and 7 days before a Default Election. Basically: there will be debates both before and after elections and the people who ask questions won't be plants. Voters, in their final election "receipt" will see the "also ran representatives" and part of the goal here is to get people to see the ideological diversity of their own neighbors, and learn alternative new names they could be higher on their lists next time, to show a lot more ideological diversity at both the local and federal level, so the voters can change their mind if they become embarrassed of what is said by the people who nominally represent them. Also, voters can just "fire and forget" on their "no confidence" status updates, by proxying their "no confidence" to any single one of these "also ran reps" that they ranked higher than whoever is actually currently representing them.

Thus, each "also ran" will have some real power, connected to a real voice, and be able to credibly threaten all five of the winners from a superdistrict with "no confidence" to some degree or another, if they get a lot of disgruntled voters to proxy their confidence to that "also ran". Hopefully this lets the each voters have TWO people to complain to about the House, and let them not be constantly be obsessed with politics in real time forever, because that would be very exhausting and a terrible waste of brain power.

(There's a lurking implication here where reps who were elected and who were also the first choice of a lot of voters will get "confidence vs no confidence" directly by those first choice voters, who will not be allowed to proxy their "no confidence", because those voters won't have anyone that they ranked higher on their ballot than who they ended up being represented by! Either these voters will have to watch their representative more carefully all by themselves, or else those elected people will be predictably more secure as their unproxied supporters get distracted and don't register "no confidence" for stuff that they just never observed or heard about. This was an unintended design outcome, but on reflection I think I endorse it as a sort of circuit breaker that makes really good representatives very safe and really bad voters particularly clear targets for appeals to change their mind by their fellow voters.)

What you WISH would happen is that everyone (from the voters up to the Speaker) would just universally derive common sense morally good government policy from first principles to the best of their ability... and then elections would basically just amount to picking the wisest person around who is willing to perform altruistic government service in a fair way to cheaply produce public goods and cheaply mitigate the negative externalities, that naturally arise when free people exercise their freedom to exchange within locally competitive and efficient markets, in obviously good and fair ways.

</proposal>

I fear that my proposal will cause a lot of churn and drama at first, and seem to be broken, and to be a source of constitutional crises for... maybe 1-6 years? It might seem a bit like a civil war between the Republicrats and the New System, except fought with words and voting? The House might well reboot every 6 months for a while, until the first wave of Senate elections occurred.

But after 12 years (time enough for the Senate to reboot twice) I'd expect the House to become quite boring and very very very reasonable and prudent seeming to nearly everyone, such that the US could (and would want to) switch to a fully Parliamentary system within 18 years and think "what took us so long to do this obviously sensible thing?"

One thing to remember is that Rulers Who Rule A Long Time Are Generally Less Aligned With The People. 

I think most people haven't internalized the logic of such processes, and somehow have invented some kind of bullshit cope such that they can imagine that having the same representatives and elected officials for long stretches of time (with children of famous politicians being elected based on name recognition) is somehow "good", instead of a really really terrible sign. Then many of the people who don't believe this are in favor of (and sometimes even pass) term limit laws instead of designing elections with high turnover based on minor dissatisfactions, which is the opposite of the right move. Term limits REMOVE voter influence (again, like so many other things) rather than enabling voters to have more influence to truly pick who they truly think (1) is wise and (2) has their interests at heart.

My proposal treats "lots of people cycling through the House very fast for very short stints based on actual voting that solicits many bits of information from actual voters on low latency cycles" as a valid and good thing, and potentially just a "necessary cost of doing business" in the course of trying to literally just have the best possible (representative) government that can be had.

If ANYONE survives that kind of tumult, you would expect them to be shockingly benevolent and skilled rulers. You wouldn't want people so exquisitely selected from huge numbers by thorough sifting to then get "termed out"! That would be a tragedy!

In the ideal case, the US House would eventually have sufficient global centrality (because the US government is kind of the imperial government of the world?), and sufficient recognized wisdom (because this proposal makes it stop being a dumpster fire?), that eventually lots of countries would simply want to join the US, and get to help select the membership of our House, which could become the de facto and eventually de jure world government.

The really hard thing is how to reconcile this vision with individual rights. Most Americans don't actually understand social contract theory anymore, and can't derive rights from first principles... so the proposed House, if it were really properly representative, might be even more hostile to the Bill Of Rights than it already is, which would set them very strongly against the SCOTUS and I don't know what the resolution of that process would look like in the end :-(

My hope is that the (1) fast cycling, and (2) "most central wins" dynamics of the new electoral scheme...

...would cause "reasonableness" to become prestigious again?

And then maybe a generation of reasonable humans would come along and stop voting against individual rights so much? Maybe? Hopefully?

If you think voters are just completely stupid and evil, then I could see how that would be a coherent and reasonable reason to be against my proposal... but then for such people I'd wonder why you aren't already organizing a coup of all existing governments (except the authoritarian governments that are really great at respecting individual rights... except I think there is no such thing as a current or past example of a real government that is both authoritarian and also individual-rights-respecting).

It is precisely from sloshing back and forth between these alternatives ("actually good" vs "actually democratic") that causes me to try to "steelman the idea of representative government" with this proposal.

Granting that the existing government is neither competent nor honest nor benevolent, maybe the problem is that "true democracy has never actually been tried" and so maybe we should actually try "true democracy" before we overthrow the existing shambolic horror?

However, this full extended vision aims to imagine (1) how a good House could actually work, and (2) how the voters could learn to stop being hostile to freedom and individual rights, and (3) how other countries wanted to get in on the deal... and if it hits all of its various aims at the same time then it might give humanity "world peace" for free, as a side effect? <3

You gotta have hope, right? :-)

You gotta say what might actually work in Heaven BEFORE you start compromising with the Devil, right? :-)

There are still some compromises with the Devil in my plan, but the only devils I'm trying to compromise with here are the voters themselves.

It seems to me that the Speaker of the House has a lot of power and it not really necessary to have a position with that much power.

There should be processes whereby every law that would have a majority behind it can be brought to a vote even if the Speaker of the House doesn't like the law.