You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

DanArmak comments on The Web Browser is Not Your Client (But You Don't Need To Know That) - Less Wrong Discussion

22 Post author: Error 22 April 2016 12:12AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (47)

You are viewing a single comment's thread. Show more comments above.

Comment author: DanArmak 23 April 2016 01:22:57AM *  7 points [-]

I completely agree with everything you've said in the first half of your post. But I strongly disagree that NNTP is a good choice for a backend standard. (At this point you can say that you'll argue your case in a future post instead of replying to this comment.)

The NNTP model differs from modern forums in a crucial respect: it is a distributed system. (I use this in the sense of 'decentralized'.) More precisely, it is a AP system: it doesn't provide consistency in synchronizing messages between servers (and it makes messages immutable, so it gets the worst of both worlds really). This directly leads to all the problems we'd have in using NNTP for a forum, such as no true editing or deleting of messages. Because a message is not tied to a domain (or a server), and is not referenced but copied to other servers, authentication (proving you own an identity and wrote a post) and authorization (e.g. mod powers) become nontrivial. Messages don't have globally unique IDs, or even just global IDs. Implementing something like karma becomes an interesting computer science exercise involving decentralized consensus algorithms, rather than a trivial feature of a centralized database. And so on.

But we don't need to deal with the problems of distributed systems, because web forums aren't distributed! What we want is a standard that will model the way forums already work, plus or minus some optional or disputed extensions. Making NNTP resemble a forum would require adding so many things on top that there's no point in using NNTP in the first place: it just doesn't fit the model we want.

A good forum model would tie users and messages to a particular server. It would make messages mutable (or perhaps expose an immutable history, but make the 'current' reference mutable, like git does). It would at least provide a substrate for mutable metadata that karma-like systems could use, even if these systems were specified as optional extensions to the standard. It would allow for some standardized message metadata (e.g. Content-Encoding and Content-Type equivalents). It would pretty much look like what you'd get if you designed the API of a generalized forum, talking json over http, while trying not imagine the clientside UI.

There's probably an existing standard or three like this somewhere in the dustbin of history.

NNTP also has a lot of more minor ugliness that I'd be happy to argue against. It's one of the http/mime/email family of headers-body encodings, which is well known for producing fragile implementations (quick, recite the header folding rules!) and are all subtly different from one another to make sure everyone's sufficiently confused. It relies on sometimes complex session state instead of simple separate requests. There's a bunch of optional features (many of historical interest), but at the same time the protocol is extremely underspecified (count how many times it says you SHOULD but not MUST do something, and MAY do quite the opposite instead). Any client-server pair written from scratch inevitably ends up speaking a highly restricted dialect, which doesn't match that of any other client or server.

Given all of this, the only possible value of using NNTP is the existing software that already implements it. But there's no implementation of an NNTP client in Javascript (unless you want to use emscripten), if only because Javascript in a browser can't open a raw TCP socket, so until the recent advent of websocket-to-tcp proxies, nobody could write one. And implementing a new HTTP-based server to a new (very simple) standard, basically just CRUD on a simple schema, is much easier than writing an NNTP JS client - IF you're willing to not make a distributed system.

A final note: one may well argue that we do want a distributed, decentralized system with immutable messages (or immutable old-message-versions), because such systems are inherently better. And in an ideal world I'd agree. But they're also far, far harder to get right, and the almost inevitable tradeoffs are hard to sell to users. I'm not convinced we need to solve the much harder distributed version of the problem here. (Also, many decentralization features can be added in a secondary layer on top of a centralized system if the core is well designed.)

Comment author: DanArmak 23 April 2016 01:36:36AM 2 points [-]

It would make messages mutable (or perhaps expose an immutable history, but make the 'current' reference mutable, like git does)

As an aside, git is about as good a fit as NNTP (which is to say, neither is really all that good in my opinion).

Git has immutable messages, but it also has mutable references (branches) for edits, and the references can be deleted for retractions. It has a tree structure for comments. It has pseudonymous authentication (sign your commits). It has plenty of room for data and metadata (e.g. specify a standard mapping filenames to headers). It can run over HTTP and has existing servers and clients including Javascript ones. It can be deployed in a centralized model (everyone pushes to the same server) but others can mirror your server using the same protocol, and there are RSS and email gateways available. Messages (commits) have globally unique IDs, allowing for incoming links. It makes your server state trivial to backup and to mirror. I could go on.

In fact, someone has already thought of this and wrote a similar system, called GitRap! (I didn't know about it before I checked just now.) It doesn't do exactly what I described, and it's tied to github right now, but you can view it as a POC.

To be clear: I am 95% not serious about this proposal. Solving the vastly simpler centralized problem is probably better.

Comment author: Error 28 April 2016 04:21:23PM 0 points [-]

I think that's a terrible idea and it is awesome that it exists. :-P