Update:

A month later and I'm still using it instead of my physical screens.  It must be okay.

I've switched back to Linux and am using a combination of dummy HDMI dongles and a USB-C dock to give me three screens.  It works okay, but is annoying.  There's some work going on to make virtual screens work better but it's spread throughout the display stack so don't expect it to happen any time soon (unless you're on Intel hardware, in which case you can create virtual screens fairly easily).

The annoyances with Windows were mainly Windows annoyances.  The ones that really got to me were:

• The WiFi Direct thing in Immersed is only available if your laptop is connected to a WiFi network.  I used a wired connection most of the time so this was a considerable annoyance.  It was usually easier to just open settings and start the Windows hotspot (which is what Immersed was doing under the covers anyway).
• The Windows WiFi hotspot occasionally just... switches off.  Not that occasionally, actually.  This happened at least once per day.
• To make my workflow work well, I switched to Windows 11 to get WSLg.  Windows 11 is... buggy (or was - big update released this week but I haven't tried it).  Didn't wake from sleep reliably, numerous lockups/freezes, the WSLg display server would occasionally freeze or crash with no very good way to recover it.
• Windows makes an utter dogs breakfast of remembering what screens are open and where they should be placed.  Several times a day I had to rearrange all the screens.

My Linux setup is pretty stable.  I still need to figure out how to start the agent as part of the display manager startup, so that I can put the headset on before I log in.  I don't think this is possible without making the agent a command-line process, though.

I've been using an Oculus Quest 2 in this exact setup for a few days.  I think a lot of what's said here is pretty accurate.  My list of pros:

• Anywhere.  I can put my laptop down anywhere, put a headset on and start working with four screens.  TBH I'm still sat at my desk most of the time but I'm considering whether I still need a desk.
• Why stop at three monitors?
• The Oculus headphones sound really nice.  How do they make sounds sound as though they're coming from behind you with two speakers that are both in front of your ears?  I don't understand this.

Lots of people mention the lack of distraction.  I work in a room on my own so it's not a big issue for me anyway.  When I have a distraction, it's my kids running in, so I feel kind of bad ignoring them because I've got a headset on.

Cons:

• Getting into and out of it is pretty clunky.  I write software so I want to use a keyboard (and occasionally mouse); there's no way I'm going to write code by pointing the Oculus controllers at a virtual keyboard.  To get into it, you start the agent, put your headset on, pick up the controllers, start the Immersed app, sort out the monitor layout, put your controllers down (make sure you cleared a decent space for this before you put the headset on) then find your keyboard.  A bit more thought going into how well it remembers screen layouts and a voice-activation way of launching apps on the Oculus would go a long way to making this better.
• You know how every time you plug in an external screen you have to tell Windows where it's physically placed?  That, multiplied by about a million times.  Each time you want to do this, you have to pick up the Oculus controllers.  The grid-snapping system for placing screens seems to have annoying quirks.
• WiFi.  My router is not in the same room.  This adds enough latency (and unpredictable latency) to make it unusable.  It has a WiFi direct thingy; for some reason, the option to use this is not always available in the agent software.  I don't know why.  I'll have to sort out another WiFi AP (been meaning to do this anyway to improve coverage) but you've just lost Pro #1.
• Linux.  I prefer a Linux desktop.  All of my software work is on Linux.  The agent claims to work on Linux but just... doesn't.  You run Wayland?  Sorry.  You have a 4k screen?  Sorry (actually this worked sometimes, which is nearly worse).  You want virtual desktops like the blurb says you can have on Linux?  Sorry.  You can display any physical monitors you have connected as desktops in VR, but you've just lost Pros #1 and #2.  I'm trying out working in WSL2 on Windows but it's not comfortable for me.
• Resolution.  There's a reason we have big, 4k screens.  The Oculus' resolution is hard to give a feel for, because it varies across the display surface.  The thing you're looking at has a much higher dot pitch than things on the periphery of your vision.  So the thing you're looking at is good enough, but you have to get used to turning your head constantly to look at things; glancing at a second screen out the side of your eye does not produce a good result.  This might be just something you get used to.