im addicted to watching session replays (and you should be too)

I have issues with PostHog. Usually little things, but recently they completely switched up their navigation, broke the UX, and my muscle memory keeps fighting me as I relearn the damn app. Perhaps if they'd watched the session replay of me furiously looking for where 'activity' has gone, or 'session replays' and staring incredulously at the fucking empty input box that stares back at me when I load into the app — then perhaps they'd understand that this is a major regression in usability.

But they didn't, and that's ok(?) — they're clearly trying to switch up how their app works and go 'AI first' whatever that means these days. But I bring this up because the point I'm trying to make is that you (as a product person, engineer, prompter, whatever) think you know your product. You think you understand how it works, how it should be used, how others should interact with it. I was, by my own confident estimation, one of the 7 people in the world best placed to judge whether Querio was any good.

Sounds reasonable, but it's a steamy pile of bull—

the first hit

You start with a replay of a customer you know who's been really happy with the product. Watch it on a sleepy Sunday evening, G yourself up for the week — a little ego boost.

They're using the app, blasting through their queries, building up a report for the next day. Damn that report is getting long. Wow, he's been scrolling back up to the top for a while. Yeesh, still going. Hmm, yeah this is no good.

We'd shipped this clean look which did away with default browser scrollbars. Looks beautiful in Figma, dare I say even better on my laptop. But watching this session made me feel like a moron. This poor guy dragging his cursor down the edge of the report, feeling along the wall for a light switch in the dark. I pulled up Zed, ripped out the no-scrollbar class from the container and quietly shipped it.

Nobody ever knew, but the rush. Big fat dopamine hit. Found the issue, patched it, before a ticket gets opened or anyone says anything. Super satisfying.

But let's chill for a second, this container should've had a scrollbar from the get-go. I'd relied upon a stranger to show me a scrollbar was missing from the core feature in our app.

rock bottom

500 replays later, I'm still here — documenting my findings.

Every replay is the same, again and again. Someone walking into a room I built but never once walked into myself — never noticed the uneven floor, the weird slightly damp smell, I think there's a leak somewhere.

Meanwhile me looking in from the outside, god it looks beautiful. The walls are pristine — wow look at the skirting. It's perfect to me.

But after enough of these, I can't call it bad luck anymore. I can't file every twisted ankle under 'edge case'. And I can't write them off as just a dumb user.

Which ultimately leads me to the conclusion that I am just not a good judge of my own product. Not because I'm careless — because I can't be un-spoiled. I understood the app the second I touched it, and it has become part of my muscle memory. I can no longer look at it and see it new again. We've collectively been grading our own homework and giving it a resounding A* every time.

And you can't break this just by trying harder not to be blinded by your familiarity. It's your eyes that are broken. A bent ruler isn't going to start measuring straight just because you're really wishing it would.

the evidence

We recently opened up our application to let users sign up without needing to speak to us first and just get started with the product immediately (you can do that here btw).

This has been awesome, but it's also made this familiarity problem all the more obvious. Until now, we've been watching replays of people who have been using the product for a minute, who've spoken with us directly and been given the product tour. But these self serve users are looking at our app completely fresh — and the holes become painfully clear.

A good example: onboarding (self serve) a brand new user, first thirty seconds in the product. There's a placeholder workspace logo, with a button next to it saying 'upload'. I watch the user furiously clicking the placeholder directly, expecting to be able to upload their logo. Obviously. It's a big round logo-shaped target, of course that's the thing you reach for. But nothing happens, because the circle was a mere <div> and only the little upload button next to it invoked the file uploader.

The first thing a brand new user did in our product was reach for the most obvious thing on the screen and be told no. And most of the time, they don't go hunting for the real button afterwards — they just give up and move on, no logo uploaded.

A short while later, the user completes the onboarding and ends up on the landing of the Querio app! Fantastic, let's start with a query — test out the product. And so the user goes to send their first message, they type it out, hit enter.

Now they're just waiting.. one, two, three seconds. Is Querio dead? Cue the spam clicking of the send button. I knew we had a loading indicator, because I remember putting it there. I went and took a look and saw this beauty:

{isWaitingForResponse || isStreaming && (
  <LoadingIndicator ... />
)}

&& binds tighter than the || so it reads as isWaitingForResponse || (isStreaming && ..). So while you're waiting — the exact moment I would want the user to know something is happening — isWaitingForResponse is true, the || short-circuits, and the whole expression collapses to a bare true. React renders jack for a bare true. The indicator would only ever show once the response was already streaming in, reassuring you about a fear that had already passed.

don't tell me you'll just test it

I can already hear you saying 'that could never be me — I'm always testing' — buddy, I'm right there with you. I thought that was me..

You'll test it, get someone to click around — but they know exactly what they're looking to click, the build is running locally, everything is built around them to make sure it works.

Even interviewing customers is a slippery slope, they'll tell you the burning issues, the things that realistically you already know or genuinely are game-breaking. But they also often misremember, tell you a different version of events from what actually took place. It's really hard to remember the tiny details that just slightly piss you off — until it's a head-smash-against-your-keyboard level issue.

I'm one of the freaks that takes great pride in maintaining a list of these tiny 'niggles' I spot across apps — makes me feel better about this whole predicament I find myself in.

you

This brings me, finally, to you.

You think you see your product for what it really is, warts and all — but you only ever see the warts on the surface. It's the stuff underneath you should be worried about.

We're all contaminated by the complacency that allows us to be happy with what we're building. It's an armour — it stops you feeling the experiences real people are having daily.

So yes, I mean the title — I am addicted, and you really should be too. Not because watching is a nice habit, but because it's the only way you can make sure your users aren't developing Stockholm syndrome for your app.

the one that didn't get fixed before bedtime

A confession needs a resounding ending, the part where I somehow figure this all out and reveal to you my findings.

This loop was serving me pretty nicely — I was out here feeling like I was rewriting the future, on some time-bending shit. I'd end the day looking through replays, walking the key user flows, checking whether people even noticed the new things we'd just shipped, and patching little issues in a couple mins. I'd gotten comfortable.

But unfortunately I'm getting to the point where the loud issues I'm noticing are.. pretty chunky. Some fundamental architectural changes would be needed to fix these.

The big one right now: I'm watching users click 'new explore' and just wait.. for the page to come back. God I hate it so much.

We're using Next.js (for better or for worse) and this one isn't a typo or a missing class — it's RSC, baked deep into how the whole route loads. This post was actually going to be about how RSC has ruined UX and our beautifully fast page loads, but we're in the weeds migrating off Next as we speak, so I'll save that story for later.

But this ain't getting fixed before bedtime.

—msh