Business Software Evaluation Checklist
Most software selections go wrong before the first demo, by shortlisting products from different categories that solve different problems. This checklist fixes the order: establish the category, test enforcement rather than features, test the field and exit paths, then pilot narrowly and measure honestly.
Software demos are designed to show capability. They are not designed to show what happens when a certification expires, when there is no signal on site, or when you want your data back. Those are the three things that decide whether a platform works in practice, and none of them appears in a feature comparison. This checklist is deliberately short and deliberately awkward. Every item is something a vendor can demonstrate in a tenant during evaluation, and something you will regret not having asked afterwards. It applies to any operations-heavy business — contracting, manufacturing, distribution, laboratories, engineering services — not only inspection and testing.
What it covers
- Category test — instrument software, generic CMMS or EAM, owner-side integrity platform, or business management. Which problem are you actually buying?
- Enforcement test — expire a qualification in the demo tenant, then try to assign that person to a job. Does the system refuse, or merely warn?
- Calibration enforcement — take an instrument out of calibration and try to issue it. Same question.
- Point-in-time test — ask to see a record from a year ago and prove who was qualified and what was calibrated on that date, not today.
- Field test — put the mobile app in airplane mode, complete a full job with photographs, restore signal, and confirm nothing is lost or duplicated.
- Conflict test — edit the same record offline on two devices, then sync both. What happens?
- Client-format test — render the same dataset into two different client report formats without re-entering it.
- Exit test — request a complete data export with its schema, during evaluation rather than at renewal.
- Cost-visibility test — can you see margin on a live job, or only at final account?
- Configuration test — ask for one change to a workflow during the trial and see who has to do it, and how long it takes.
- Pilot definition — one team, one demanding client or contract, a fixed window, with the baseline measured before you start.
- Success criteria — written down in advance, in hours or errors rather than impressions.
How to use it
Before any demo: Answer the category question yourself. If you cannot state in one sentence which of the four categories you are buying, the shortlist will mix them and the demos will not be comparable. This is the single highest-value hour in the whole selection.
During each demo: Insist on running the enforcement and field tests in the tenant rather than being told about them. A vendor who will not let you expire a qualification and try to dispatch is telling you something.
Before signing: Run the exit test. How hard it is to get your data out during a sales process is the most reliable available signal of how hard it will be to leave.
During the pilot: Measure the same three things every week: how long a job takes from dispatch to invoice, how long it takes to assemble evidence for one client request, and how many compliance gaps reached a client. Everything else is opinion.
Frequently Asked Questions
Why does the category matter more than the vendor?
Because the gap between categories is structural rather than a matter of features. A maintenance system has no model for method-level personnel qualification; an instrument package has no model for a client contract. Choosing the wrong category costs the implementation plus twelve to eighteen months, and the second attempt is harder because the organisation has already lived through one failure. Choosing an average vendor in the right category is recoverable; choosing an excellent vendor in the wrong one is not.
What is the difference between enforcement and recording?
Recording means the system knows a qualification expired. Enforcement means the system makes it impossible to assign that person to work requiring it. Nearly every product records; far fewer enforce. Since the incident you are trying to prevent happens at the moment of assignment, recording without enforcement is documentation rather than control — and it will not stop the failure that costs you a mobilisation or a client audit.
How long should a pilot run?
Long enough to cover one full cycle of the work — for most service businesses that is around six weeks, which captures quoting, delivery, reporting and invoicing at least once. Shorter than that and you have tested the demo rather than the platform. Much longer and the pilot becomes the project, and nobody ever decides.
Who should be in the evaluation?
Whoever will actually use it daily, plus whoever will be blamed if an audit goes badly. Selections run entirely by management consistently pick tools the field will not use; selections run entirely by the field consistently miss the compliance and costing requirements. Both have to be in the room, and the least enthusiastic user should be the one who tests the mobile app.
See it running on your own workflow
Thirty minutes, your job types and your reporting formats, co-presented by an ASNT NDT Level III. Affordable, accessible, fully customizable — request a demo and a tailored quote.
Related: The four categories explained · Inspection management software · Business management platform · All free resources