AI assistants and search decision
100 Questions
A useful single-run benchmark/report is realistic for a technical user to build and host, but the full commercial product experience (polish, multi-provider grounding nuances, and curated deliverables) is better obtained from the vendor.
Visit website↗Open-source builds that already do this
Every project below is open source and already does this job today. Fork one, self-host it, or take the parts you need — the build prompt further down assumes an empty file, and this is the shortcut past that. Licences differ; check the one on each card before you ship.
What a replacement has to do
- Run a frozen 25-question prompt set against four web-grounded LLM providers, collect each provider's answer and cited sources, compute visibility/prominence/coverage/citation metrics, store evidence, and generate a client-ready PDF/CSV with five prioritized actions.
What it still won’t have
- Polished, branded UI and PDFs that match the vendor's deliverable
- Any proprietary integrations or tuned prompts the vendor keeps private
- Built-in sample reports, guides, and marketing-ready artifacts
- Operational conveniences like prepaid credit handling and hosted payments
What remains hard
- Product polish and ongoing maintenance
First-year cost
Keep paying
Paying is—cheaper in year one.
On cash alone, building overtakes the subscription at 3 seats.
Money you would actually spend
Time you would spend
—
What you would spend
What we assumed
The verdict above measures whether you could build it. This one is only about money.
Runnable build prompt
Build a minimal self-hosted AI visibility benchmark using Node.js (Express) backend, Postgres for storage, and a small React UI. In scope: (1) implement a payment flow that mints one prepaid 'benchmark' credit (Stripe), (2) run a frozen set of 25 neutral discovery questions against four provider APIs (configurable API keys), (3) store raw answers and any returned citations with timestamps in Postgres, (4) compute metrics (visibility, prominence, share-of-voice, citation rate, coverage) and produce a downloadable PDF and CSV report that includes five prioritized action items (templated rules), (5) UI to start a benchmark, view evidence, and download exports. Out of scope: training models, large-scale crawling, enterprise SSO, or any closed-source provider integrations beyond standard HTTP APIs. Deliverables must include error handling for failed provider calls, retries with backoff, unit tests for core metric calculations, and an integration test that runs the full 25-question loop against mocked provider responses.
How we checked
How the score was reached
- Partly verdict base52
- An open-source build was found+5
- 3 cited sources+3
- Price verified on pricing page+3
- 3/3 assessment runs agreed+4
- Evidence score67
The base comes from the verdict. Everything under it is a check that either happened or did not, and each one is a fact frozen in this record rather than a judgement made at render time — so the same evidence always produces the same number.
How scoring works →Cited sources · 3
Every page the run actually retrieved.
- official product100 Questions — AI Visibility Audit & Benchmark Tool
- open sourcearc53/DocsGPT
- open sourceaingdesk/AingDesk
Integrity checks
What held up, and what did not.
