How to Measure Before and After Switching Your Company's Management Software
Paul Boards ·
To measure the before and after of switching management software, pick four to six metrics tied to the problem that prompted the switch, measure them for at least four weeks in the old system, record the cutover date, and measure the same numbers, the same way, for at least eight weeks afterward. Without a baseline, any improvement or decline is just opinion.
Companies usually switch systems out of frustration: lost tasks, rework, a report that eats the whole afternoon. Measuring turns that frustration into numbers and lets you answer honestly whether the change was worth the cost and effort.
Start with why you're switching
Write one sentence explaining why the company is changing systems. Examples: "we miss deadlines because tasks are scattered across email and spreadsheets" or "the monthly client report takes four hours to put together." Each sentence points to a metric. Measuring everything at once scatters attention and produces numbers nobody uses.
Metrics that usually make sense
Delivery
- On-time delivery rate: tasks delivered by the due date divided by the total number of tasks whose due date passed in the period.
- Cycle time: days between task creation and delivery, using the median.
- Stalled tasks: number of tasks with no status change for more than seven days, counted on the same day each week.
Quality
- Revisions per deliverable: number of revision rounds divided by number of deliverables.
- Client complaints or pushback: a simple monthly count.
Administrative effort
- Time to build a report: minutes spent preparing one client's report, timed.
- Tools in use: how many systems and spreadsheets the team has to open each week.
Adoption
- Active users: how many team members updated at least one task that week.
- Tasks created outside the system: requests that came in by message and never became a task.
Example of a well-defined metric
A vague definition like "on-time deliveries" leaves room for everyone to count differently. Compare it with a complete definition:
On-time delivery: tasks completed by 11:59 p.m. on the due date, divided by the total number of tasks whose due date passed in the period. Only client tasks count. Canceled and internal tasks are excluded. Tasks with no due date are left out of the calculation and counted separately.
With that definition, an accounting firm moving from spreadsheets to a system can compare March with June without arguing over what went into the count. The number of tasks without a due date also becomes a metric: if it drops after the switch, it's a sign the team started setting dates for work that used to float.
How to measure the "before"
The old system rarely hands you ready-made data. A few practical ways to build the baseline:
- Export whatever you can from the current system: creation, due, and completion dates.
- Take a manual sample when there's no export: pull 30 recent tasks and note the due date and delivery date.
- Time administrative tasks for four weeks, such as closing out reports.
- Write down the definition of each metric, including what's excluded.
The written definition is what guarantees a fair comparison. If you counted only tasks with due dates before and all tasks after, the number changes because of the rule, not the system.
Switching systems? Start on a free plan with a pilot team and measure your first numbers side by side. Create your pilot account
How to measure the "after"
Separate the ramp-up period
The first two to four weeks after cutover usually show worse numbers because the team is still learning. Record that period, but compare the baseline with the weeks that follow. That way the learning curve neither hides the result nor gets mistaken for it.
Keep the same yardstick
Use exactly the definitions from the before period. In a system that keeps status history, like Scrumboards, it's easier to calculate cycle time and stalled tasks because every change is dated. Task, time, and revision reports help you pull the numbers without building a spreadsheet.
Note what else changed
New hires, a big client leaving, a process change, the year-end season. These events affect the metrics and need to appear next to the numbers.
Comparison table template
| Metric | Definition | Before (4 weeks) | After (weeks 5 to 12) | Notes |
|---|---|---|---|---|
| On-time delivery | Delivered by due date / tasks past due date | fill in | fill in | Number of tasks in each period |
| Cycle time | Median in days | fill in | fill in | |
| Revisions per deliverable | Rounds / deliverables | fill in | fill in | |
| Report time | Minutes per client | fill in | fill in | How many reports were timed |
| Active users | Updated at least 1 task that week | fill in | fill in |
Always report the number of cases next to each rate. An on-time delivery rate based on 12 tasks is far less reliable than one based on 200.
How not to fool yourself
- Don't choose metrics after seeing the numbers. Define them before cutover.
- Don't add up overlapping savings. If the report got faster because hours are already logged, don't count the same savings twice.
- State what was excluded. Canceled tasks, clients who left, holiday weeks.
- Include the cost of switching: migration hours, training, and the difference in subscription fees.
Total cost of the change
The gain only makes sense compared with the cost. Record the hours spent on setup, data import, training, and adjustments, plus the new system's monthly fee and any tools you stopped paying for. If the new system brings tasks, calendar, CRM, and finance into one place, as Scrumboards does for companies in any industry, include the canceled subscriptions in the calculation.
When the numbers get worse
Not every switch improves every metric, and that's useful information too. If the on-time delivery rate dropped after ramp-up, first check whether the yardstick changed: the new system may be recording due dates that didn't exist before. Then look at adoption. A team where only half the people update their tasks produces incomplete numbers. If the yardstick is the same and adoption is high, the problem may be the configured process, such as too many statuses or poorly designed task templates. Adjust one thing at a time and measure again for a few weeks.
How to present the results
One page is enough: the reason for the switch, the metrics with definitions, before and after with case counts, the ramp-up period, outside events, and total cost. End with a decision: keep it, adjust the process, or reconsider the tool. Results without a decision become a forgotten slide deck.
Frequently asked questions
How long should you wait to evaluate the switch?
At least eight weeks after the ramp-up period, so you have enough data.
What if the old system has no data?
Take a manual sample of recent tasks and time your administrative routines before cutover.
How many metrics should you use?
Four to six, tied to the reason for the switch.
Should you migrate every team at once?
A pilot with one team reduces risk and produces comparable numbers before the full migration.
Does team satisfaction count?
Yes, as a supporting metric. Use one simple question, asked the same way before and after.
Build your baseline today
If the switch is still under evaluation, start measuring your current system this week. When you decide to try it out, starting a free pilot on Scrumboards doesn't require a credit card. Before that, check out the available modules and reports and the FAQ on migration and usage.