K7 Insights
How to Evaluate AI Tools Before Your MSP Buys
Part of Grow Your MSP
Key takeaways
- Give the purchase a job: replace something with a measurable improvement, fill a real gap, or serve an opportunity a client has already raised.
- Agree on the evidence before you start a trial. Count the work your team still has to do, and put one person in charge of the decision and the renewal.
Before your MSP buys an AI tool, write down what the purchase is supposed to change. It should replace something with a measurable improvement, fill a real gap, or help you serve an opportunity a client has already raised. Then test that claim against your own work and count what it’ll cost to use the tool. That gives you a basis for comparing vendors, including the option to keep what you have. It’s the buying discipline that belongs in your plan for growing your MSP.
What qualifies an AI tool for your shortlist?
A tool qualifies when you can name the business reason to consider it. I use three starting points in my July 22, 2026 talk on buying under pressure: a better replacement, a real gap, or an opportunity clients are already asking about. Getting onto the shortlist earns the tool an evaluation.
| Reason to consider it | What you need to write down |
|---|---|
| Replace something | The tool you’re replacing, the improvement you expect, and when you’ll stop paying for the old one |
| Fill a gap | The job your current setup can’t do well enough and what that gap costs you |
| Serve a client opportunity | The client’s actual request and why this purchase would help you respond |
Be specific about that last one. A client asking, “What should we do about AI?” gives you a conversation to have. You’ll need to understand what they’re trying to accomplish before you can connect that interest to a software purchase.
And check what you already own. If there’s a capability in your current stack that nobody has tried, give someone the job of testing it. You may still buy the new tool. At least you’ll know what you’re paying extra to get. And if someone on your team would rather build it than buy it, answer five questions before anyone starts coding.
My broader AI for MSPs article deals with where automation belongs inside your shop. Here, we’re deciding whether a particular purchase deserves your money.
How do you handle a vendor pushing for a quick decision?
Make space between the pitch and your answer. Ask for the offer in writing, including its expiry date, and give yourself time to check the business case. A deadline can be real. You still get to decide whether the deal is worth making at that price.
I’m susceptible to the excitement too. I’ve had to build a pause into my own buying: a run on the trails, or 20 minutes where the adrenaline drains and I can hear myself think. I’ve woken up the next morning and decided I wasn’t going to buy the thing. I described that habit in the same talk, because I need the interruption myself.
So when a rep says you’re falling behind, ask them to show you where. What work in your business would change? Can they show it working on a task you recognize? What would make their product a poor fit for you?
You’re allowed to enjoy the demo. Bring the person who’ll use the tool into the next conversation, and let them ask the awkward questions. They have to live with the purchase after you’ve moved on to the next thing.
What should an AI trial prove before you buy?
A trial should prove the improvement that got the tool onto your shortlist. Write that down before you start, along with who’ll judge it and when you’ll make the decision. Otherwise, it’s easy to spend a month playing with the software and finish with “the team liked it.”
Say you’re considering a tool that helps prepare proposals. I’d pick a small set of ordinary proposals plus a few that needed extra work. Have your technical lead approve the material used in the trial. Record how long the current method takes to produce something you’d actually send.
Then use the same kind of work to test the new tool. Count the time spent preparing inputs and correcting the result. If it creates a draft in seconds but someone spends 40 minutes fixing it, those 40 minutes belong in the result. Have the person responsible for proposals judge whether the finished work is acceptable.
Set your pass condition before seeing the result. For a hypothetical trial, you might require the same quality with at least an hour less work per proposal. Choose the threshold from the cost you’re trying to justify. The one-hour threshold is just for this example.
Ask the vendor to show what happens when the task goes badly, too. Who helps your team? What support is included in the price you’ve been quoted? Put any work left with your people into the estimate.
Your technical lead still needs to assess the tool’s security and data handling for the intended use. This trial gives you evidence for the business decision; that technical review has its own work to do.
How do you count the full cost of the purchase?
Count the subscription at the usage you expect, the work needed to get started, and the work that continues after setup. Subtract spending you’ll actually stop. An old subscription that stays on the credit card is still a cost.
Here’s a hypothetical replacement decision. These numbers are assumptions for the example, not vendor prices or a client result:
| Item | Monthly amount |
|---|---|
| Current tool, which you can cancel when you switch | $400 |
| Proposed AI tool at expected usage | $700 |
| Additional subscription spend after cancellation | $300 |
| Staff time released, after checking outputs and maintaining the tool | 6 hours |
At an assumed loaded labor cost of $60 an hour, those six hours represent $360 of staff capacity. Loaded cost includes employment costs beyond the person’s wage. Use your own number.
That leaves $60 above the additional monthly subscription spend on paper. You still have setup and training to pay for, plus any period when both subscriptions run. A little less time saved could wipe out the case you made.
Also, your payroll may stay exactly the same. If the person uses those six hours to handle work you’d otherwise pay overtime for, you can track the cash effect. If they use the time to prepare better client meetings, decide how you’ll judge that benefit. Either can matter. Be clear about what you’re claiming.
For a tool bought to serve a client opportunity, use the same discipline with the revenue assumption. Identify what the client is prepared to buy before you treat that revenue as money available to cover the subscription.
If you’re staring at a quote and the case still feels thin, bring the purchasing decision to a call. We’ll pressure-test what you’re expecting the tool to change and whether you’ve counted the work it’ll create for your team.
What belongs on your AI vendor decision sheet?
Put the reason to buy and the evidence on one page, with the unresolved questions left visible. Use the same sheet for each vendor so you can compare them against the job you need done.
Here’s the sheet I’d use:
| Decision | What to record |
|---|---|
| Job for the purchase | The replacement, gap, or client opportunity you’re evaluating |
| Current result | What happens today and what it costs |
| Required result | What has to improve enough to justify buying |
| Trial evidence | What your team observed, including corrections and ongoing upkeep |
| Full cost | Expected usage, setup, training, support, and any overlap with the old tool |
| Open questions | What you still need from the vendor or your own reviewers before you can decide |
| Commitment and exit | The minimum term, renewal date, cancellation steps, and how you’d move your work out |
| Decision owner | The person who’ll recommend buying or declining, and the date they’ll do it |
Ask the vendor for written answers to the commercial questions. If they’re quoting a low starting price, have them price your expected usage as well. Have your team check the answers against the actual offer before you sign.
Leave room for keeping your current setup in the comparison. You need to know what improvement makes switching worth the trouble. If the trial can’t show that improvement, you’ve learned something useful while the commitment is still small.
Who owns the decision after the trial ends?
One person should own the recommendation and the follow-through. Give them access to the people who’ll use the tool and the numbers needed to evaluate it. The owner or budget holder can still approve the spending; the work of reaching a recommendation needs a name beside it.
If you buy, have that person record what was approved and put a review on the calendar before the cancellation deadline. Check whether the promised improvement showed up in normal use. If the case depended on replacing another tool, check that the old subscription actually ended.
If the evidence is incomplete, write down the missing answer and when you’ll get it. You can extend a test deliberately. You can also stop. The trial has done its job when it gives you a decision you can explain.
Take one AI quote you’re considering and write the job for that purchase at the top of a page. That’s the move to make first. A clear reason to buy gives your team something to test and gives you a way to hold the spending accountable inside your MSP growth plan.
- Should I buy the AI add-on from a vendor I already use?
- Put it through the same business test as any other purchase. Check whether your current subscription already covers the job, what the add-on costs at your expected usage, and how much work your team will have to do to use it. An existing relationship can make evaluation easier. You still need evidence that the purchase is worth making.
- What if I'm buying an AI tool to improve quality?
- Define the quality improvement you need and have the person responsible for that work judge the trial. For example, you could compare how often a proposal needs correction before it's ready to send. Record the current result, agree on an acceptable result, and keep the review method the same. You don't have to turn every benefit into a claim about hours saved.
- Can I buy an AI tool just to learn what's possible?
- Yes. Give yourself a learning budget and a date to review what you found. Keep that expense visible so it doesn't quietly become a company-wide subscription. If the experiment uncovers useful work, build a business case for that work before you expand the purchase.