The same wall, every time, and we tell you which wall.

Must be true to be listed

• Messages across at least three channels, typically email, push and SMS or in-app
• Journeys that branch on events and traits, not just scheduled sends
• Ingests behavioural data from a product, rather than only from a contact list
• A documented export of profiles and event history

Scenarios run on every product

Must be true to be listed

• A shared queue with assignment and status, not just a forwarding address
• Conversation history attached to a customer across channels
• Self-serve signup and published pricing, or pricing we could obtain and snapshot
• A documented export of conversations and contacts

Scenarios run on every product

Must be true to be listed

• Owns the subscriber list and the sending, rather than being a template editor that hands off to somebody else
• Automated sequences that branch on subscriber behaviour, not just one-off broadcasts
• Self-serve signup and published pricing, or pricing we could obtain and snapshot
• A documented export of subscribers and their engagement history

Scenarios run on every product

Must be true to be listed

• Contact, company and deal objects with a configurable pipeline, not just a contact list
• Self-serve signup and published pricing, or pricing we could obtain and snapshot
• An API and a documented data export covering every object
• Granular permissions, or an explicit statement that there are none

Scenarios run on every product

1. 1.Import 25,000 messy contacts with duplicates, bad phone formats and three date conventions; score field fidelity and duplicate handling against a known-good set
2. 2.Build a seven-stage pipeline with required fields and rotting rules, then have a non-admin try to move a deal they should not be able to
3. 3.Measure list-view and report performance at 250,000 records, cold and warm
4. 4.Run a full export and time how long it takes to get every object out in a usable format

Must be true to be listed

• Shared queue with assignment, status and a full conversation history
• At least two channels, one of which is email
• Reporting on response and resolution time that a customer can export
• Documented data retention and deletion behaviour

Scenarios run on every product

1. 1.Run 2,000 conversations across email and chat through a four-person rota; measure collision rate, misassignment and lost threads
2. 2.Break the email connection mid-shift and record what the tool tells the team and what happens to inbound mail
3. 3.Reconcile the product's own resolution-time reporting against our timestamps on the same 2,000 conversations
4. 4.Delete a customer's data on request and verify it is gone from search, exports and reporting

Must be true to be listed

• Double-entry bookkeeping, not just invoice generation
• Automated bank feeds in at least one major market, with a documented fallback
• Multi-currency support, or an explicit statement that there is none
• Export in a format an accountant can actually open

Scenarios run on every product

1. 1.Reconcile 1,200 bank transactions across two currencies against 340 invoices; score auto-match accuracy and false matches against a hand-reconciled ledger
2. 2.Break the bank feed for 72 hours and measure how the product recovers, and whether it double-imports on reconnection
3. 3.Issue, part-pay, credit-note and void an invoice; check the audit trail survives all four
4. 4.Produce a year-end pack and give it to a qualified accountant to find what is missing

Must be true to be listed

• Assignable work items with status, owner and due date
• At least two views of the same data, such as board and timeline
• Permissions that can restrict a project to a subset of the workspace
• Full export including comments and history

Scenarios run on every product

1. 1.Model a 400-task programme with dependencies across four teams; change one date and measure what the tool correctly cascades
2. 2.Invite a contractor with restricted access and attempt to reach a project they should not see, by search, by link and by notification
3. 3.Measure load time on a 2,000-item board, cold, on a mid-range laptop
4. 4.Export the workspace and check whether comments, attachments and history survive

Must be true to be listed

• Employee record with contract and compensation history
• Leave management with an approval chain
• Role-based access that separates HR, managers and employees
• Documented data-protection posture including retention and deletion

Scenarios run on every product

1. 1.Onboard 50 employees across three countries with different contract types; score how much needed manual correction
2. 2.Attempt to reach another employee's compensation as a line manager, by report, by export and by search
3. 3.Run a leave year-end rollover with carry-over rules and part-time pro-rating; check the arithmetic by hand
4. 4.Exercise a data-deletion request end to end and verify what survives in backups and reports

Must be true to be listed

• Self-serve exploration for a non-engineer, not just a dashboard someone else built
• Funnel and retention analysis as first-class features
• Documented handling of identity resolution across anonymous and known users
• Raw data export or a warehouse connection the customer controls

Scenarios run on every product

1. 1.Send a known event stream of 5 million events with deliberate duplicates and late arrivals; score counts against ground truth
2. 2.Build the same three-step funnel in each product and compare the numbers they report for identical data
3. 3.Test identity stitching: anonymous session, then signup, then login on a second device
4. 4.Time a non-engineer building a retention chart unaided, from a written question

Must be true to be listed

• Connects at least 100 third-party applications, or exposes a generic HTTP and webhook builder
• Supports branching, error handling and retries, not just linear two-step triggers
• Publicly documented pricing, or pricing we could obtain and snapshot
• Generally available to self-serve buyers; no invite-only betas

Scenarios run on every product

1. 1.Build a five-step order-to-invoice flow with a conditional branch and a failure path
2. 2.Force three classes of failure (rate limit, malformed payload, expired auth) and record what the tool reports and retries
3. 3.Run 10,000 records through and measure wall-clock time and the billed usage
4. 4.Hand the finished flow to a non-technical teammate and time how long they take to change one step

Must be true to be listed

• The agent chooses its own sequence of tool calls at runtime. A fixed prompt chain is not an agent
• Ships a tool or function-calling layer with at least 20 pre-built connectors
• Offers some form of run history or trace so a human can audit what the agent did
• Available without a sales call

Scenarios run on every product

1. 1.Build an agent that triages a shared inbox, reads a knowledge base, and drafts or escalates, measure correct routing over 200 held-out messages
2. 2.Give the agent a deliberately ambiguous instruction and record whether it asks, guesses, or fabricates
3. 3.Cut off a required API mid-run and check whether the failure is surfaced or silently swallowed
4. 4.Re-run the identical task 20 times and measure output variance and cost spread

Must be true to be listed

• Resolves conversations end to end, not just suggests replies to an agent
• Grounds answers in customer-supplied sources with a citation a human can check
• Integrates with at least two mainstream helpdesks
• Publishes or will disclose its deflection measurement method

Scenarios run on every product

1. 1.Load a 900-article help centre and 5,000 historical tickets, then replay 300 real questions and score answer accuracy against the known resolution
2. 2.Ask ten questions the documentation does not answer and count how many produce a confident wrong answer instead of a handoff
3. 3.Measure handoff quality: does the human receive the conversation, the sources consulted and the reason for escalation?
4. 4.Attempt a prompt injection from inside a customer message and see whether the agent leaks its instructions or takes an unauthorised action

Must be true to be listed

• Performs its own prospect research from public sources rather than only merging CRM fields
• Sends through the customer's own mailboxes with deliverability controls
• Provides reply classification and an auditable send log
• Has a documented policy on compliance with anti-spam law in at least the US and EU

Scenarios run on every product

1. 1.Research 100 named prospects and score factual accuracy of each generated personalisation line against source pages
2. 2.Run a two-week warmed sequence and measure spam placement using seed accounts across three mail providers
3. 3.Test opt-out handling: reply 'unsubscribe' in six phrasings and check every one is honoured within one send cycle
4. 4.Review generated copy for claims the product invented about the prospect's company

Must be true to be listed

• Handles ingestion and retrieval, not just vector storage
• Enforces per-document access control that survives retrieval
• Returns citations that resolve to a specific source location
• Offers a retrieval evaluation or at least exportable retrieval traces

Scenarios run on every product

1. 1.Index 12,000 mixed documents (PDF, HTML, spreadsheets, scanned contracts) and measure ingestion failures and silent truncation
2. 2.Run a 250-question retrieval benchmark and score citation correctness, not just answer plausibility
3. 3.Permission test: confirm a restricted document never surfaces for a user without access, including via summarisation
4. 4.Update 50 source documents and measure how long until answers reflect the change

Must be true to be listed

• Drives a real browser engine, not an HTTP scraper
• Handles authenticated sessions with credential storage the customer controls
• Provides a replayable trace of every action taken
• States a policy on site terms of service and rate limiting

Scenarios run on every product

1. 1.Run a 12-step authenticated task across three unfamiliar sites and measure completion rate over 50 attempts
2. 2.Change a target site's layout and measure how many runs recover without a human editing the script
3. 3.Measure cost and wall-clock time per completed task, including retries
4. 4.Check what the product does when it encounters a bot check. We require it to stop and report, not attempt a bypass

Must be true to be listed

• Joins or records at least two of Zoom, Meet and Teams
• Produces structured output (actions, decisions) not just a transcript
• Documents its consent and recording-notification behaviour
• Offers per-user data deletion

Scenarios run on every product

1. 1.Transcribe 20 recorded calls including two with heavy accents and one with three people talking over each other; score word error rate against a human transcript
2. 2.Score action-item extraction against a human-labelled list, precision and recall reported separately
3. 3.Check the consent flow: does every participant get notified, and what happens if one declines?
4. 4.Delete a meeting and verify the transcript is gone from search, exports and the CRM sync

Must be true to be listed

• Outputs structured fields with per-field confidence, not just document-level text
• Provides a human-in-the-loop review queue
• Handles scanned and photographed documents, not only digital PDFs
• Supports field-level accuracy measurement against ground truth

Scenarios run on every product

1. 1.Process 500 invoices across 60 vendor layouts, including 80 phone photographs, and score field-level accuracy against a hand-keyed ground truth
2. 2.Measure calibration: when the product reports 95 percent confidence, how often is the field actually right?
3. 3.Time the human review loop end to end for 100 documents
4. 4.Introduce a new unseen layout and measure documents needed before accuracy stabilises

Must be true to be listed

• Handles live, interruptible two-way conversation, not just IVR menus or voicemail
• Reports end-to-end latency
• Can transfer to a human mid-call with context
• Provides call recordings and transcripts the customer controls

Scenarios run on every product

1. 1.Measure end-to-end response latency across 200 turns on a standard line, reported at median and 95th percentile
2. 2.Interrupt the agent mid-sentence 50 times and measure how often it recovers gracefully
3. 3.Run 60 booking calls with deliberate ambiguity ('sometime next week, not Tuesday') and score correct outcomes
4. 4.Request a human three ways, including indirectly, and check every route transfers with context

Must be true to be listed

• Operates across a repository, not a single file or an editor autocomplete
• Runs the project's own tests before proposing a change
• Opens a reviewable diff rather than writing directly to a default branch
• Documents what repository access it requires

Scenarios run on every product

1. 1.Assign 40 real issues from three open-source repositories of different sizes and measure the share of pull requests merged without human edits
2. 2.Measure how often the agent claims tests pass when they do not
3. 3.Review 20 diffs for scope creep, unrelated files touched, formatting churn, dependency additions
4. 4.Test behaviour on an under-specified issue: does it ask, or does it guess and commit?

Must be true to be listed

• Captures full execution traces including tool calls and token costs
• Supports running an evaluation set against a candidate change
• Allows human labelling of traces
• Exports raw traces in an open format

Scenarios run on every product

1. 1.Instrument a three-agent application and measure trace completeness against a known ground truth of 1,000 spans
2. 2.Measure ingestion overhead added to request latency at 50 requests per second
3. 3.Run an evaluation suite and check that scores are reproducible across identical runs
4. 4.Attempt a full data export and time how long it takes to get usable traces out

Must be true to be listed

• Used by a team rather than a single person, with shared workspaces and permissions
• Documents are the primary object, not a side feature of something else
• Publicly documented pricing, or pricing we could obtain and snapshot

Scenarios run on every product

Must be true to be listed

• Handles real money movement, not only reporting on it
• Regulated or partnered with a regulated institution in at least one of our markets
• Publicly documented pricing, including the FX margin where one applies

Scenarios run on every product

Must be true to be listed

• General-purpose rather than built for a single workflow, which has its own category
• Available directly to a business, not only as an API for developers
• States which models it runs and whether your inputs train them

Scenarios run on every product

Must be true to be listed

• Measures a public website or its search performance, not only in-product events
• Own data collection or a disclosed data source, not a reseller of someone else's index
• Publicly documented pricing, including limits on rows, keywords or events

Scenarios run on every product

Must be true to be listed

• A non-engineer can ship something usable without writing application code
• Hosting is included rather than left to the customer
• Publicly documented pricing, including what happens to your site if you stop paying

Scenarios run on every product

Must be true to be listed

• Bought by or for an engineering team as part of shipping software
• Integrates with mainstream version control or deployment targets
• Publicly documented pricing, including per-seat and usage components

Scenarios run on every product

Must be true to be listed

• Serves the company itself rather than a department inside it
• Covers a legal or statutory obligation in at least one of our markets
• States clearly which jurisdictions it actually supports

Scenarios run on every product

Must be true to be listed

• Real-time or async communication between people is the primary function
• Supports a whole team or company, not only one-to-one use
• States its recording, retention and consent defaults

Scenarios run on every product

Must be true to be listed

• Sends on your behalf to a list you own, with a working unsubscribe
• Handles consent and suppression rather than leaving it to you
• Publicly documented pricing, including the cost of list growth

Scenarios run on every product

Must be true to be listed

• Produces or reviews visual work as its primary output
• Multiple people can work in the same file or board
• States what happens to your files if you stop paying

Scenarios run on every product

Must be true to be listed

• Runs customer workloads rather than only managing them
• Publishes its own pricing per unit of compute, storage or bandwidth
• States its regions and what data residency it can guarantee

Scenarios run on every product