Your buyers compare your product with a competitor’s before they sign, and your team needs to know how that comparison comes out. There are five common ways to benchmark a rival’s product: read its documentation and reviews, try it yourself, take it apart in a teardown, pay a benchmarking firm, or take its sales demo as a buyer.
Mystery Demo does the last of these for you. Its team books competitors’ real sales demos as a credible buyer, asks the questions your team needs answered, and records every call. You get the recordings and the analysis on a shared board: what each competitor’s team claims about its product, what it shows working, and what it steers around.
Each method produces a different kind of evidence, and none of them is the best one. They differ first in what they can verify about the rival’s product, and what they cannot see:
| Method | What it verifies | What it cannot see |
|---|---|---|
| Docs and reviews | What the rival publishes, and what customers felt after buying | Whether a feature works as documented, and how it is shown to a buyer |
| Trial | What the product does in your hands | How the rival’s team sets it up and shows it, and any product without a trial |
| Teardown | Features, setup, and limits, worked through one by one | What the rival’s team claims to a buyer |
| Firm | The methods it uses, in depth, measured the same way for every rival | What the rival’s team tells a buyer, unless its method includes the sales process |
| Mystery demo | What the rival’s team claims, shows working, and moves past with a credible buyer, all recorded | Anything not said or shown on those calls, and a promise stays a promise |
What Documentation and Reviews Can Tell You
Start with what the competitor publishes, because it is free and often more detailed than the homepage suggests. The help center, the documentation, and the release notes say what the product is supposed to do. Help content is written for customers who have already paid, so it tends to name the limits a homepage leaves out:
Review sites add what documentation cannot: how customers felt about the product after they bought it. That is evidence about the product in daily use, from the people who use it.
Both stop short of the product itself. Documentation says what is supposed to work, not whether it works that way today. A review describes one customer’s setup after the sale, which may look nothing like yours.
To see the product do what its documentation says, somebody has to use it.
What a Trial or a Teardown Shows You
A trial puts the product in your hands. You can run the workflow your customers care about, load the kind of data they would, and see where the product slows down or stops.
A teardown takes the same access further. For learning how a product behaves in use, nothing else here comes close, a mystery demo included. Someone works through the product systematically and writes down what they find:
The first catch is access. In one software review site’s check of 318 B2B SaaS products, mostly marketing and HR tools, 29% offered neither a free plan nor a trial, only a demo or a sales call. Another 35% offered only a trial that expires.
A trial also shows the product only as you configure it, with your own guesses and no help from the competitor. That is not how its team would set it up and present it to a buyer like yours. And a signup from your work email is one the competitor’s sales team can look up.
Both methods take your team’s time, and a teardown of several competitors takes a lot of it. So the next decision is whether to run them yourself or pay someone to.
What a Benchmarking Firm Adds
A benchmarking firm mostly changes who does the work, not the kind of evidence. It runs documentation analysis, hands-on testing, and teardowns with more depth than a busy team can spare. It also measures every competitor the same way, so the results line up.
That saves your team’s time and gives you a cleaner comparison, priced per engagement. The catch shows up when you track several rivals. Replying to a question about four competitors, one product marketer on Reddit wrote: “Agency deep-dives don’t scale to four competitors, because you’re paying for the same collection work four times over.”
The bigger question is what the firm’s method can see. Three questions settle it before you sign:
Documentation, trials, and teardowns, whoever runs them, never see the product the way your buyers meet it on a sales call.
What Only a Buyer in the Room Sees
Documentation, reviews, trials, and teardowns all test the product away from the people who sell it. Your buyers also meet it on a sales call, presented by the competitor’s own team. That team decides what to claim, what to show working, and what to describe on a slide and move past.
Those choices are evidence about the product, and they are the evidence your reps are up against. One product marketer on Reddit concluded that most of their competitor monitoring had been pointless: “Reps get beaten by whatever the competitor says about us in the room.”
Mystery Demo is built to be the buyer in that room, and it starts from your questions. A project runs in five stages:
The evaluation is real, because the team is evaluating software on your behalf. Nobody is impersonated, and the one thing a competitor never learns is who asked.
Everything lands on the board, a shared Notion workspace your team keeps, with a page for each competitor. Below is its Research Objectives page, which holds the project’s questions and method, taken from our public example board. The client and every competitor on that board are invented, and a real project’s board is built from the calls taken for that project.
On the invented board, stage 4 shows the method adapting to the competitor: where one offered a self-serve trial, the trial went on the record beside its sales call. The recordings are the raw material. Your team works from the analysis built on them.
What Your Team Gets From the Calls
Each competitor’s page holds its recordings, transcripts, emails, and collateral, with our analysis of what it said and showed. Findings & Analysis is the page that compares the competitors with each other. Its feature matrix gives every capability one of three marks:
A capability that was described but never shown gets none of the three. It stays unresolved until someone sees it work.
Read this comparison from Findings & Analysis on the invented example board for its columns, not its numbers. They set what each competitor claimed beside what its demo verified. Another column says how each demo showed it, from live on sample data to cited but never demoed head to head.
The column on how each claim was shown is the one the other methods cannot fill. A trial tells you what a feature does in your hands, but it cannot tell you that the competitor’s team mentioned the feature to a buyer and did not show it.
.png)
Here is the same idea for one competitor, as an invented example in plant nursery software. Each feature appears three ways: what the rep said on the call, what the screen showed, and what the competitor’s help center says. Every rival, line, and minute on it is made up.
Read the first feature across. The help center says the stock sync runs hourly, and a careful trial might have found that too. Only the call records the rep telling a buyer it happens in real time, and that is the version a buyer carries into a deal against you.
The other four methods leave that evidence out, so the choice comes down to what each method costs and how long it takes.
What Each Method Costs in Money and Time
Most of these methods are paid for in your team’s hours rather than an invoice, so their honest price depends on your scope:
| Method | Cost | Time to an answer |
|---|---|---|
| Docs and reviews | Your team’s time | Depends on how many competitors and features |
| Trial | Depends on the vendor’s access terms | Depends on access and scope |
| Teardown | Depends on who runs it | Depends on the scope |
| Firm | Priced per engagement | Set per engagement |
| Mystery demo | €499 per competitor | One to two weeks for one competitor, four to eight for a landscape |
Your own team could book the competitor demos too. It would cost your team’s time, a work email the rep can look up, and the risk of being recognized by a competitor that knows your company.
There are four reasons to hand the calls to us instead:
Name the competitor whose product claims your reps hear most, and we will talk it through on a short intro call. Then we take its demo as a buyer and bring back a competitor product comparison that marks each capability shipped, partial, or stub, from what its team showed.
Which Method Answers Which Question
The methods combine. A mystery demo does not replace a trial or a teardown, and a teardown cannot tell you what the rep says. The clearest picture comes from the published record, your own hands, and the competitor’s sales call, side by side.
Documentation and reviews answer what the product is published to do and how customers feel about it. A trial or a teardown answers what it does in your hands, and a firm gives you that depth across many competitors. A mystery demo answers how the competitor’s team presents it to a buyer, which is the version your reps meet in their deals.
Start with the claims your reps already hear. This week, list the things buyers say most often about a competitor’s product, and mark which method could check each one:
| What your reps might hear from buyers | The method that can check it | What to look for |
|---|---|---|
| “They integrate with the tool we already use” | Documentation, then a trial | Whether the sync runs both ways, and on which plan |
| “Their product is faster on large files” | A trial or a teardown | The same file, timed in both products |
| “Their rep showed us a feature you don’t have” | A trial for the feature, a mystery demo for the pitch | Whether it runs on screen or stays on a slide |
| “Their rep says your product can’t do this” | A mystery demo | Whether their team says it to a buyer, and in what words |
The last row is the one no desk research reaches. Tell us which competitor it is, and we will take its demo as the buyer. You get back what its team claims about its own product, and about yours.
.png)
