Product Thinking

The metric that drifted for a month before anyone looked

A number nobody watches daily moves a little every day until it is a crisis. A sentinel that knows what normal looks like flags the drift on day one, not in the quarterly review.

ASR

Apollo Space Research

Apollo Space

· 11 min read

A conversion rate slips from, say, four percent to three and a half. Then to three. Then to two and a half. Each step is small enough to blame on a quiet week. Nobody panics, because nobody is looking, not because they’re careless, but because nobody opens that dashboard on a Tuesday. The number lives on a slide that gets built four times a year. By the time it’s on the slide, the slope has been pointing down for a month, and the conversation isn’t “what changed” anymore. It’s “how did we miss this.”

That’s the failure this post is about. Not the drop. The month.

The most dangerous number in your company is the one that’s moving slowly in the wrong direction while nobody watches it. The crisis was never the value. It was the gap between the day it started drifting and the day someone finally looked.

The dashboard is a place numbers go to be ignored

Here’s the naive answer everyone reaches for first: build a dashboard. Put the metric on it. Make it big, make it red when it’s bad, and trust that someone will see.

It works for exactly as long as someone is staring at it. Which is to say, it works during the launch week when the dashboard is new and everyone’s proud of it, and then it quietly stops working forever. A dashboard is a thing you have to remember to open. The metrics that hurt you are precisely the ones you’ve stopped remembering to open, because they were fine for so long that checking them felt like a waste of a morning.

So the number sits there, available, technically visible, and functionally invisible. The dashboard did its job, it displayed the value. The value was correct. The chart was accurate. And the business still walked off the cliff, because displaying a number is not the same as noticing it changed. A dashboard answers a question. The whole problem is that nobody was asking.

Worse: the slow drift is the kind a glance can’t catch even when you do look. Four percent versus three-point-eight percent doesn’t read as alarming on a Tuesday. It reads as noise. The human eye is built to catch the cliff, not the slope. The drop that kills you is the one shallow enough to look like a normal bad day, repeated thirty times.

A threshold alert is a tripwire at the bottom of the canyon

The next instinct is smarter: don’t make a human watch the dashboard, make the system watch it. Set an alert. If the conversion rate drops below, say, two percent, fire a notification.

This is better, and it’s still wrong, and it’s worth being precise about why. A threshold is a single line drawn at a number you picked in advance. It catches the value once it crosses, and the entire danger of drift is that by the time it crosses your line, the damage is a month old. You set the tripwire at two percent. The metric spent four weeks sliding from four to two-point-one, costing you a little more each day, and the tripwire stayed silent the whole way down because two-point-one is not yet two. The alert fires at the bottom of the canyon, after the fall, to tell you that you have, in fact, fallen.

There’s a second failure baked into the fixed threshold, and it’s the one that quietly kills the whole approach. To avoid false alarms, you set the line conservatively, low enough that normal variation never trips it. But “low enough to never false-alarm on a bad Tuesday” is exactly “low enough that a slow bleed never trips it either.” You tuned the alert to ignore the thing it was supposed to catch. The tripwire that never cries wolf is the tripwire that never cries at all.

A threshold knows one number: the line. It doesn’t know what last month looked like. It doesn’t know that four-percent-trending-down is a story and four-percent-holding-steady is fine. It has no memory and no shape. It can tell you where the metric is. It has no idea where the metric is going, and going is the whole game.

A slow decline in a conversion metric over a month. A glance on any single day reads as normal noise. A fixed threshold alert stays silent until the value finally crosses the line, firing only after the loss has accumulated. A sentinel that learned the baseline flags the drift on day one, while the slope is still small.

Our way: a sentinel that knows what normal looks like

The fix isn’t a better dashboard or a lower threshold. It’s a different question. Stop asking “is the number bad yet” and start asking “is the number behaving differently than it usually does.” Those are not the same question, and the gap between them is the month you’ve been losing.

The key idea is simple. Watching a metric well means three things, in order, and a dashboard does none of them.

First, learn the baseline. Before you can notice a change, you have to know what unchanged looks like. Not a single number, a shape. The conversion rate has a normal weekday rhythm, a normal weekend dip, a normal range it wanders inside without anything being wrong. A sentinel watches long enough to learn that shape, so “normal” stops being a guess you hard-coded and becomes a thing the system actually knows. This is why a fixed threshold is so blunt: it replaced a rich, moving baseline with one flat line.

Second, watch the slope, not just the level. A point is not a story. The level, where the metric is today, tells you almost nothing on its own; three percent is a triumph for one business and a five-alarm fire for another. The signal lives in the change: this metric has stepped down a little, then a little more, then a little more, and the steps point the same direction four days running. A human reviewing a quarter of data can see that trend instantly. The trick is to have something reviewing it every day, so the trend is caught at four days, not at four weeks.

Third, and this is the part that separates a coworker from a smoke alarm, explain the drift, don’t just announce it. A bare alert (“conversion is down”) is the start of someone’s afternoon of digging, not the end of it. A useful sentinel arrives with the context already assembled: the metric started sliding around the day the checkout page changed; the decline is concentrated on mobile; here are the two things that moved at the same time. Not a diagnosis it swears by, a head start. The difference between “the number dropped” and “the number dropped, here’s where it started and what coincided with it” is the difference between a problem dumped on your desk and a problem half-solved before you saw it.

Put those together and you don’t have an alert. You have a watcher that learned your business’s normal, noticed the shape break early, and handed you the break with its likely cause attached, on day one of the drift, while it’s still a slope and not yet a crisis.

Why this only works if something is always watching

There’s a reason a dashboard can’t do this and a human checking weekly can’t either. The watching has to be continuous and it has to be cheap, and humans are neither.

A person can watch one number obsessively, or fifty numbers occasionally, never both. So we triage. We watch the three metrics that are currently on fire and ignore the forty that are merely fine, and the forty that are fine are exactly where the next slow drift is quietly starting. The metric that drifted for a month did so because it was fine when you last looked. Its good standing is what bought it the invisibility that let it rot. You don’t watch the healthy ones, which is precisely why the next sick one will be a healthy one you stopped watching.

A system doesn’t triage. It can hold the baseline for every metric at once, re-check every one of them every day, and stay just as alert on the boring number in month nine as it was on launch day. It never gets bored, never decides this quarter’s fine so next quarter’s probably fine too, never lets a metric earn its way off the watch list by behaving for a while. The boredom that makes humans stop watching is the exact failure a sentinel doesn’t have.

And it has to speak first. This is the thread running under all of it. The dashboard waits for you to open it. The threshold waits for the value to cross. Both are reactive, they sit quietly until you come to them or until the damage comes to you. A sentinel inverts that: it watches while you’re not, and it interrupts you the moment the shape breaks, with the break and its context in hand. The most dangerous number in your company is the one moving slowly in the wrong direction while nobody watches it, so the answer is something whose whole job is to watch it when no one will.

Two responses to a slowly drifting metric. On the left, the reactive path: the number lives on a dashboard nobody opens, a fixed threshold stays silent, and the drift surfaces in the quarterly review as a crisis. On the right, a sentinel learns the metric's baseline, re-checks it every day, catches the broken shape on day one, and sends a single message naming the drift and its likely cause while there is still time to act.

What a sentinel is actually made of

Strip away the word and a metric sentinel is four plain jobs, composed.

It reads the number on a schedule, from wherever the number actually lives, the database, the analytics tool, the billing system, so nobody has to remember to pull it. It remembers the history, because you cannot judge a change against a baseline you don’t hold; memory is what makes the difference between an alarm and a judgment. It compares today’s shape against the learned normal and flags the break, scoring the slope rather than the single point. And when it flags, it assembles the why, the other things that moved around the same time, so what lands on you is a lead, not a chore.

Read, remember, compare, explain. Separately, those are unremarkable. Composed into one watcher that runs every day across every metric and speaks up only when a shape genuinely breaks, they’re the thing a dashboard was always pretending to be and never was: not a place numbers go to be looked at, but something that does the looking for you.

This is the same move under every proactive system worth building. The work was never displaying the data. The data was always there. The work is the noticing, and noticing, done continuously across everything, at the moment the shape first bends, is not something you can ask a busy person to do. It’s something you have to build.

The turn: stop paying the cost of looking too late

Here’s the part that isn’t about metrics.

Every slow drift you’ve ever caught late had the same hidden cost, and it wasn’t the metric. It was the month. The thirty days the number bled before anyone looked are days you can’t get back, the customers who churned while the rate slid, the spend that crept while the efficiency slipped, the quiet erosion that was reversible on day three and a crisis by day thirty. You didn’t lose because the number got bad. You lost because the gap between got bad and got noticed was thirty days wide, and everything that happened in that gap was avoidable.

That gap is a choice, even when it doesn’t feel like one. It feels like diligence to review the metrics quarterly, to build the careful dashboard, to set the sensible alert. But diligence that catches the drift in the quarterly review is diligence that catches it a month too late, every time, by design. The most capable people in your company were never going to win this by watching harder. The number of things worth watching outgrew the number of mornings anyone has to watch them a long time ago.

The promise isn’t a prettier chart. It’s that the drift gets caught on the day it starts, by something that learned what your normal looks like and never stops watching it, so the only review that’s left is deciding what to do, not discovering, a month late, that there was something to do all along.


That’s part of what we’re building at Apollo Space, not another dashboard you have to remember to open, but a watcher that knows your company’s normal and tells you the moment the shape breaks. If you’ve ever found a problem in a quarterly review and felt the floor drop out, you already know the real cost was never the number. It was the month nobody was looking.

Apollo runs your company's repetitive ops so your team doesn't.

Join the waitlist for early access, founding-user pricing, and a front-row seat as we ship.

Join the waitlist