On this page
Key takeaways
- A shadow spreadsheet records what the official system fails to: exceptions, holds, and judgement calls.
- Columns are requirements; colour-coding is policy; comments are edge cases.
- Read the workbook before interviewing anyone, then ask about the parts that look strange.
- The goal is not to migrate the sheet. It is to make it unnecessary without losing what it knew.
Every operations team has one. A workbook that lives on someone's desktop, is emailed around on Fridays, and is quietly more accurate than the system of record. Leadership usually treats it as a problem. We treat it as the best requirements document in the building.
Why the workbook exists
A private spreadsheet is a rational response to friction. The official system was designed for the common case; the person at the desk handles the uncommon one forty times a week. So they built the thing that fits: a grid, a few filters, and some colour.
That makes the workbook an unusually honest artifact. Nobody built it to impress anyone. Every column exists because someone needed it twice.
How to read one
Columns are requirements
List the columns. For each, ask what decision it supports. A column called "Holding?" is a state. A column called "Ask Jonas" is a missing permission. A column of free-text notes is a workflow nobody has named yet.
Colour is policy
If rows turn red, there is a rule. Find it. It is usually a business policy that exists nowhere else — not in the system, not in a wiki, only in the person who set the conditional format.
Comments are edge cases
Cell comments and strike-through rows are where exceptions are recorded. They are the most valuable and least structured data you will find, and they are where a naive migration loses information.
Hidden tabs are integrations
A tab of pasted export data is an integration someone performs by hand. Count how often it is refreshed; that is your required sync frequency.
| In the workbook | What it usually means | What the product should do |
|---|---|---|
| A column of yes/no flags | A hidden state machine | Model the states explicitly |
| Conditional colour | An unwritten policy | Write the rule down and enforce it |
| Free-text notes | An unnamed workflow | Name it, or keep a note field on purpose |
| Pasted export tab | A manual integration | An adapter with a visible retry |
| Initials in a column | Ownership | Assignment with a required owner |
| A "do not touch" row | Fear of a broken formula | Remove the fear by removing the formula |
Every column exists because someone needed it twice. That is a better requirements process than any workshop.
A short discovery routine
- Ask for the workbook before the first meeting. Read it alone.
- Write down every question the columns raise. Do not ask them yet.
- Sit with the owner while they use it during a real shift. Watch, do not interview.
- Then ask about the parts that look strange. Strange is where the policy is.
- Write the non-goals: what the new system will explicitly not model.
Ask about the deletions
The most revealing question is: what did you stop tracking because it was too much effort? Those absent columns are usually the ones the business needs.
Migrate the sheet
- Reproduces its accidents
- Loses comments and colour rules
- Shames the person who built it
Read the sheet
- Extracts states, policy, and edge cases
- Names what the new system will not model
- Makes the owner the first user
What not to do
- Do not migrate the sheet as-is. You will reproduce its accidents.
- Do not shame the owner. They are the domain expert, and you need them to be the first user.
- Do not promise the sheet will be deleted on day one. It will be retired when it stops being needed, and not before.
Halcyon's version
Halcyon's night managers kept a private workbook because it was faster than the official system. It had a column for whether finance would care, a colour for "reopened", and initials for who had the problem. Those three things became the spine of Northframe's exception model: consequence, reopen reasons, and owners.
Within thirty days of go-live the workbook was retired. Nobody deleted it. It simply stopped being useful, which is the only retirement that lasts.
Questions to ask the workbook's owner
- Which column would you miss most if it disappeared?
- What do the colours mean, and who decided?
- What did you stop tracking because it was too much effort?
- Which tab do you refresh by hand, and how often?
- What would make you trust a new system enough to close this one?
A workbook audit template
Before designing anything, audit the workbook like a codebase. The audit produces a one-page inventory that everyone can argue about, which is far cheaper than arguing about a half-built product.
| Inventory item | What to record | Example |
|---|---|---|
| Sheets and tabs | Name, purpose, refresh method | "TMS_paste" — pasted export, refreshed by hand each morning |
| Columns | Name, type, who fills it, source | "Holding?" — yes/no, set by night lead |
| Formulas | What business rule each encodes | =IF(C2<TODAY()+4/24,"URGENT","") — the four-hour window rule |
| Conditional formatting | Rule and meaning | Row turns red when reopened twice |
| Comments and notes | Categories of edge case | "customer changed window after departure" |
| Macros or scripts | Automation and its owner | Email digest sent at 05:30 |
| Access | Who can view and edit | Three people edit; six view by email |
Formulas are business rules in disguise
A formula that turns a row red is a policy with no owner and no history. Extract every formula into a sentence — "an exception is urgent when the delivery window closes within four hours" — and ask the workbook's owner to confirm it. Those sentences become your ruleset.
From columns to schema: a worked example
Here is the translation for a simplified night-desk workbook. The point is not the exact schema; it is how each spreadsheet habit becomes an explicit, constrained fact.
| Workbook habit | Becomes |
|---|---|
| Column "Status" with free text | A state enum with an allowed set and defined transitions |
| Initials in a column | A required owner_id foreign key, nullable only in the unassigned state |
| Colour meaning "reopened" | A reopened_count and a required reopen_reason |
| Column "Will finance ask?" | A finance_impact boolean with a reason |
| Notes in a comment | A child table of exception_notes with author and time |
| A pasted export tab | An adapter that records source_ref and last_synced_at |
CREATE TYPE exception_state AS ENUM ('unassigned','assigned','waiting','approved','closed','reopened');
CREATE TABLE exceptions (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
source_ref text NOT NULL, -- load id in the TMS
state exception_state NOT NULL DEFAULT 'unassigned',
owner_id uuid REFERENCES staff(id),
finance_impact boolean NOT NULL DEFAULT false,
reopen_reason text,
reopened_count int NOT NULL DEFAULT 0,
created_at timestamptz NOT NULL DEFAULT now(),
CHECK (state = 'unassigned' OR owner_id IS NOT NULL),
CHECK (reopened_count = 0 OR reopen_reason IS NOT NULL)
);
CREATE TABLE exception_notes (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
exception_id uuid NOT NULL REFERENCES exceptions(id),
author_id uuid NOT NULL REFERENCES staff(id),
body text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now()
);The two CHECK constraints do more to protect the desk than any training session. An exception cannot be assigned without an owner, and it cannot be reopened without a reason. The database enforces the policy the workbook only implied [1].
Cutting over without a heroic weekend
- Do not delete the workbook. Archive it read-only, with a note explaining what replaced it and when.
- Import history selectively. Bring in open items and the last month of closed ones. Ten years of archived rows will import ten years of accidents.
- Keep a translation table. Map old free-text statuses to new states explicitly, and list the ones that did not map cleanly for a human to resolve.
Privacy, permissions, and the person who built it
Workbooks accumulate sensitive data: customer names, prices, staff notes. Auditing them means seeing that data. Agree in advance who will look at it, what will be copied out, and how it will be handled. Workbooks also carry a person: the one who built the formulas and knows why. Treat them as the domain expert and the first user, and give them credit in the launch note. The system works best when its most knowledgeable critic is its earliest champion.
Domain-driven design [2] frames this as building a shared model with the people who hold the knowledge. The spreadsheet is simply the fastest way to see their model before they explain it.
Closing
A shadow spreadsheet is not technical debt. It is a field study you get for free. Read it carefully, and the specification mostly writes itself.
References & further reading
- 1PostgreSQL Documentation: Constraints — PostgreSQL Global Development Group
- 2Domain-Driven Design: Tackling Complexity in the Heart of Software — Eric Evans, Addison-WesleyThe chapters on ubiquitous language are directly relevant to naming states.
- 3Bliki: Strangler Fig Application — Martin Fowler, martinfowler.com
The product behind this post
Northframe
The operations layer for teams who still run the night shift.
From $890 per month
About the author
Product behaviour described here reflects what is implemented and tested; anything else is marked as planned. Code samples are illustrative.
All writing