Glassys runs on a machine you own and you reach it from wherever you happen to be: a laptop on the same network, a phone through a tunnel. The link between the screen and the server is not reliable, which means the screen spends a good part of its life guessing. Last week I went through every place where it was presenting a guess as a fact.

The upgrade that said "done" while the old process kept running

The worst of them. On Linux the service was being brought up with enable --now, which does nothing at all to a unit that is already running. So an upgrade pulled, built, reported success, and left you talking to exactly the code you had been trying to replace.

It's enable plus restart now, and an upgrade started from the web app queues the restart so that the new gateway is the one that closes out the run. But the real fix is that the claim gets checked instead of assumed: health is probed on the bind address and port the configuration actually uses, not only what happens to be in the environment, and an upgrade fails unless the health endpoint reports the new commit. The health endpoint and the Updates tab both name the commit the running process is on, the tab flags a restart that hasn't happened yet, and it offers you a reload when the gateway you reconnect to turns out to be a different one.

Around that, I found a crop of smaller lies in the upgrade path.

  • An upgrade whose script died, never started, or never came back from its restart stayed recorded as "running" forever, and blocked every later upgrade as already in progress. It's an error with a reason now.
  • Install and upgrade run with a frozen lockfile. A lockfile rewritten mid-upgrade left the clone dirty, and the web app then refused every upgrade after it. A missing pnpm or git says which one it is, instead of "pnpm install failed."
  • The service points at a stable node path — Homebrew's opt directory, fnm's default alias, or an explicit override — rather than a versioned one that the next node release invalidates.
  • The unit's working directory is written unquoted, because systemd doesn't unquote path settings and a path with a space in it broke the unit outright.
  • On macOS, install waits for launchd to finish removing the old job and retries the bootstrap instead of failing on the spot, and the gateway trims its own logs past 10 MB, since launchd never rotates them.
  • The install script runs from a function called on its very last line, so a download that gets cut off runs nothing at all rather than half of itself.

The message shown as sent that wasn't

A send used to be drawn optimistically and then forgotten about. It stays pending now until the gateway echoes it back, and if no echo arrives within ten seconds, or the gateway refuses it outright, the text and its attachments go back into the composer with the reason. The old behaviour was the failure mode that annoys me most in other people's software: your message disappears and you have no way of telling whether it was delivered.

The socket underneath it is less credulous too. It notices a link that has gone quiet — no answer to two keepalives — and reconnects. Coming back to the tab, or back onto a network, either reconnects at once or proves the open socket is alive with a ping. The backoff got jitter, and only starts over once a connection has genuinely stayed up. Errors carry their HTTP status as well, so a 502 at boot tells you the server is unreachable instead of showing a login screen for a server that isn't answering.

The transcript that repeated itself, or cut off

On reconnect the gateway has to work out which events the client already has. It used to do that by comparing JSON strings against a second read of the transcript file, which missed an event persisted just before the snapshot — that's a token repeated on reconnect — and could swallow deltas that happened to be identical.

Every persisted event now carries a sequence number that increases within its thread, the snapshot says which one it ends at, and both the handshake and the client drop anything at or below it. The field is optional, so older clients are unaffected.

The deeper problem was the shape of the file. A streamed delta was one open-append-close and one line per token, so a long answer was thousands of lines, and the reconnect snapshot — which carries the last 800 events — showed you only the tail end of the answer you were in the middle of reading. Consecutive deltas merge into a single line now, written when another event arrives, after 100 ms, or before anything else reads the thread.

The screen also gets an event before the disk does, so a transcript write that fails is reported once rather than freezing the output you're watching. And only a write that really reached the disk clears that warning: a buffered delta was clearing it, so the next failure warned you all over again.

Green checks it hadn't earned

Three more claims I took away:

  • A tool still running when the run is cancelled or fails says Stopped, instead of the green Done check it never earned.
  • The "this run has gone quiet" warning was being saved into the transcript, so replaying the thread later warned you about a run that had been over for days. It isn't saved any more, it clears when the agent speaks again or the run ends, and a second silence gets a warning of its own.
  • A refused message — queue full, gateway busy, onboarding not finished — is told to the client that sent it, rather than saved as a failed run and pushed out to every device you own.

And the setup page that trusted anyone on loopback

This one belongs to the hardening pass I wrote about last week, and it's the same mistake in a different costume: an assumption taken for proof. First-run setup would claim the instance for any client arriving over loopback. Behind a reverse proxy on the same host — which is the arrangement the docs recommend — every visitor arrives over loopback.

So the gateway writes a one-time setup code on first start and logs it, the installer prints it along with a ready-made link, and setup requires that code unless the browser really is on the host itself. Wrong codes go through the login limiter. Its neighbour is a trust-proxy setting, so that behind a local proxy the login limits key on the forwarded address rather than on 127.0.0.1 — without it, four bad attempts from anyone at all locked the operator out of their own instance.

One plain bug, while I was in there

You couldn't send from a phone during a run. Stop replaced Send in the composer, and touch devices never send on Enter, so there was no way to queue a follow-up. Send comes back beside Stop as soon as there's text, and whether Enter sends is decided by the pointer rather than by the presence of a touchscreen, so touch laptops and tablets with a trackpad work again. In the same corner: a message starting with a slash only opens the template list while it actually names a template, so "/var/log/syslog is full" sends like any other sentence.

The thread through all of it is one rule worth writing on the wall: don't report a state you haven't confirmed. "I don't know yet" is cheap, and nobody minds reading it. "Done" when it isn't costs somebody an evening.

Glassys is public, and so is the week that produced all this.