Technology Evaluation Criteria Template
Three documents and three sheets, scoring each candidate on capability, then pricing what running the winner actually costs for two years.
Free download · No account needed
Every technology evaluation template scores capability and cost in the same weighted number, which hides the operating bill until after the choice gets made. That mechanism is not unique to software. NASA's own systems engineering handbook found that a program typically spends only about 15 percent of its life-cycle cost during design. That same design work commits roughly 75 percent of everything the program will ultimately cost, because the design decision determines what gets built, tested, and operated afterward.
This pack keeps the two numbers apart on purpose. A weighted comparison scores every candidate on capability only, cost excluded from the criteria entirely. Anything within ten points of the leader runs a timeboxed proof of concept against the same real workload, logging latency, the error rate at real burst load, and the engineer hours the trial consumes. Only then does an operational cost estimate turn those numbers into a multi-year bill, and the winner gets recorded the way a settled architecture decision is: one paragraph naming why every other option lost.
At Priorwood Analytics, replacing a job queue that was dropping jobs under burst load, the managed candidate won the weighted comparison at 85 out of 100, nine points ahead of a self-hosted broker at 76. The proof of concept and cost estimate reversed that: the managed option cost $62,360 to run for two years against the broker's $40,640, a premium the six capability criteria never asked about. Priorwood picked the broker. Once a technology is chosen, the solution architecture document covers what it slots into, and a migration plan covers replacing the old system.
What's in the pack
Weighted Comparison
Every candidate scored against criteria set before anyone had an opinion about which one should win, with cost deliberately left out of the weighting.
Proof of Concept Results
The same real workload run against every candidate still in contention, with latency, error rate at your actual burst level, and engineer hours logged as they happen.
Operational Cost Estimate
A multi-year cost per candidate built from the trial's real hours and a real hosting quote, not from a vendor's list price.
Evaluation Criteria
The method: why cost gets excluded from the weighted score, how the proof-of-concept protocol works, and what triggers recording a decision.
Findings
The weighted score, the trial results, and the cost estimate placed side by side, stating plainly whether the capability leader is also the cheapest to run.
Recommendation
The decision itself, written as a decision record: one winner, why each other candidate lost, and a review date.
How to use it
- 1
Open in River, or take it blank
Send the pack the candidates you're choosing between inside River, or download the three documents and three sheets and run the method yourself.
- 2
Score capability before cost exists
Set weighted criteria with no cost row, then score every candidate against them before a hosting quote or a vendor call changes anyone's opinion.
- 3
Run one proof of concept per contender
The same workload, the same trial length, against every candidate within ten points of the leader, with hours logged as they're spent rather than recalled later.
- 4
Price the winner, then record it
Turn the trial into a multi-year operating cost, compare it against the weighted score, and write the decision up as an ADR with a review date.
Frequently asked questions
Is this template free?
Yes, and the download needs no account and no card. Edit with AI is the optional half: it reads the candidates and workload you describe, helps set weighted criteria, and builds the cost estimate from what you report back from testing. Every other pack sits in the template library.
What format are the downloaded files?
Three documents as Word files and three sheets as CSVs, in one zip. The Weighted Comparison and Operational Cost Estimate sheets arrive with the Priorwood Analytics rows in place as a worked example, so the arithmetic is visible before you replace it with your own candidates.
What if we can't run a real proof of concept?
Score capability honestly with the weighted comparison and say so in Findings rather than inventing trial numbers. A cost estimate built on a vendor's quoted price alone is still worth recording, it just carries less confidence than one built on measured hours, and the Recommendation should say which kind it is.
Why exclude cost from the weighted score entirely?
Folding a dollar figure into one criterion among six lets a small price difference move the same number of points as a real capability gap. A reader of the final score can no longer tell which one actually moved it. Keeping them apart is what let Priorwood see that its highest scorer was also its most expensive candidate by a wide margin.
How is this different from an architecture decision record?
A decision record is the general-purpose format for recording any settled choice. This pack is the evaluation that runs before one exists: the scoring, the trial, and the cost model that the eventual record gets written from. Where the choice was argued out first, an RFC register is where that argument lives.
What happens after we pick a technology?
A technical design review catches problems in the design the winner slots into, before code depends on it. That review happens once, after the recommendation is recorded, not once per candidate during evaluation, so it never argues about a choice this pack already settled.
Find out if the capability leader is also the cheapest to run
Take the Word documents and CSV sheets blank, or open this exact pack in River and score your own candidates.
Edit with AI