Comment Permalink

matto4y30

When I first go into software, it was a lot of fun to try out as many things as possible during each project. This meant that my tiny blog was based on Django (with its dozens of dependencies), Gunicorn, and a full-fledged database server (postgres). But the cost to maintain (update, service, re-tweak) all that was too damn high! So over a series of moves, my blog is a bunch of static HTML files served by nginx; these files are compiled by a python script that probably gets 0.5 commits per year on average.

That site and a few others I host on the same Ubuntu server get backed up using a simple cronjob that fires off a bash script. In case the server disappears, which has happened to me once, I have another cronjob, this one running on a raspberry Pi at home, that uses SSH to download the backups. Recently, anything I've added to the Pi, I've coded up in Ansible first. It's a little extra cost up front, but then I can reproduce the setup by running one command.

Apart from actual dependencies, I think it's important to have the least number of "mental" dependencies. For example, working with files instead of a database not only cuts out the need for maintaining a database, but also for thinking in database terms -- ie. no need to re-learn/google for obscure admin commands that you run once a year etc.

Reply

See in context

38 Designing Low Upkeep Software

by jefftk

25th Sep 2021

jefftk

2 min read

10

38

It's common for me to get excited about a personal project, put a bunch of time into it, and then lose interest. As much as possible, I want these projects to continue working; upkeep isn't fun. What does designing for minimum maintenance look like?

The biggest piece is minimizing your dependencies, and limiting them to ones that value backwards compatibility. Most of my projects are on the web, which means I have an unavoidable dependency on browsers, but this is fine because browsers strongly prioritize keeping existing websites working.

On the server I run Ubuntu LTS until it's near the end of its supported lifetime. Every two years I run do-release-upgrade and move to the next one. This cost is shared over many jefftk.com projects, so it isn't too bad.

Many projects can just be hand-written HTML/JS/CSS. No build system, no libraries, just speaking the browser's native languages. These don't need any technical maintenance; they just keep working.

One step from there are projects where I need a little bit of server-side support. For example, I write my blog posts as plain HTML, but I need headers, footers, indexes, and RSS. Each time I change my site I run a script which creates/updates all these additional files. If I want to change something about the presentation of my site then I need to dig into code, and very occasionally something will break when updating the site, but just displaying my existing stuff doesn't require any maintenance.

Then we get to the tricky stuff: projects which need real time custom code. Things like Bucket Brigade and blog comment inclusion. How I approach this:

Code in python, with no dependencies beyond the standard library. As with browsers, python rarely drops support for widely used functionality. Moving to python3 was a bit annoying, but I waited a long time and then used 2to3. Avoiding dependencies made this easy.
Store data on the file system, including for caching. It's not as elegant as a proper database, but it doesn't require an additional process or dependency. Debugging and backup are very simple, because everything is visible.
Implement as much as possible on the client.
Leave out features if they can't be implemented simply, or would require more than minimal server-side processing.

For example, I taught my comment requester to fetch comments from the EA Forum and LessWrong two years ago, and I don't expect to have to think about that code again until there are breaking changes to the upstream API.

This is not how I approach design at work, or necessarily how I would recommend doing so. Having a budget where initial creation is essentially free (fun!) while maintenance is extremely expensive (drugery!) is a dramatic exaggeration for most software development. But this process works very well for my hobby code.

Comment via: facebook

Software ToolsPractical

Personal Blog

38

New Comment

10 comments, sorted by

top scoring

Click to highlight new comments since: Today at 7:50 PM

[-]clone of saturn4y80

I would suggest Common Lisp for this purpose--it hasn't had a significant compatibility breaking change for over 30 years.

Reply

[-]DanB4y60

Having a budget where initial creation is essentially free (fun!) while maintenance is extremely expensive (drugery!) is a dramatic exaggeration for most software development.

My feeling is that most software development has exactly the same cost parameters; the difference is just that BigTech companies have so much money they are capable of paying thousands of engineers handsome salaries, to do the endless drudgery required to keep the tech stacks working.

The SQLite devs pledge to support the product until 2050.

Reply

[-]CraigMichael4y30

I do tend to use SQLite when a flat file would work.

Reply

[-]jefftk4y20

most software development has exactly the same cost parameters; the difference is just that BigTech companies have so much money they are capable of paying thousands of engineers handsome salaries, to do the endless drudgery required to keep the tech stacks working

The payback is also very different at tech companies, or in any professional environment. I make something because I'm excited about it, and the payback is some combination of getting to use it and being glad other people can use it. When a company makes something, the payback is typically that people pay for it, or perhaps use it while looking at ads. This dramatically changes the incentives around adding features and generally changing it: you need to keep improving the product to compete with others that people might use instead. Once you need to keep the product live, in the sense that there are always multiple engineers spun up on it, the cost of switching to updated versions of dependencies is a small portion of the overall cost. And designing for minimal upkeep makes it harder to add new features.

Reply

[-]matto4y30

When I first go into software, it was a lot of fun to try out as many things as possible during each project. This meant that my tiny blog was based on Django (with its dozens of dependencies), Gunicorn, and a full-fledged database server (postgres). But the cost to maintain (update, service, re-tweak) all that was too damn high! So over a series of moves, my blog is a bunch of static HTML files served by nginx; these files are compiled by a python script that probably gets 0.5 commits per year on average.

That site and a few others I host on the same Ubuntu server get backed up using a simple cronjob that fires off a bash script. In case the server disappears, which has happened to me once, I have another cronjob, this one running on a raspberry Pi at home, that uses SSH to download the backups. Recently, anything I've added to the Pi, I've coded up in Ansible first. It's a little extra cost up front, but then I can reproduce the setup by running one command.

Apart from actual dependencies, I think it's important to have the least number of "mental" dependencies. For example, working with files instead of a database not only cuts out the need for maintaining a database, but also for thinking in database terms -- ie. no need to re-learn/google for obscure admin commands that you run once a year etc.

Reply

[-]DPiepgrass4y20

Huh, if I thought LessWrong were a water cooler for software developers, I might have posted my Top 11 principles & practices of programming here.

Low upkeep software is valuable, but so is rapid application development. I aim for both at once. In web development, I have found React + MobX + holders + TypeScript to be an effective combination, and probably Vue 3 and Svelte are too. The latter libraries will probably evolve faster (and TypeScript hasn't stopped doing breaking changes, but it's a price worth paying)... but we can expect they will use semantic versioning, and there is no obligation to upgrade. So, does the choice of Vue 3 increase upkeep if Vue 4 exists and you don't upgrade? Hmm.

If I were doing blog-like software, I might well stick to plain HTML+CSS and minimal JS, though. React is just what I use for "apps".

Reply

[-]jefftk4y70

This is a good description of the approach that I don't like for this kind of development ;)

If you don't keep your dependencies up to date, you are likely to have unpatched well-known security vulnerabilities. So you need to stay on a supported version. Vue 3 will work until it goes end of life, at which point you are stuck choosing between moving to a later version or taking down your project.

Reply