How we find what AI gets wrong
Full disclosure of our data collection methods, scoring formulas, and limitations. No black boxes.
Data collection
Results are collected through provider APIs, not through the consumer-facing websites of ChatGPT, Gemini, or Perplexity.
Provider labels used in the platform:
- OpenAI web-search model
- Gemini grounded model
- Perplexity web-grounded model
Consumer applications may produce different answers due to personalization, conversation history, different model routing, and UI-level features that are not accessible via API.
Answer variability
AI answers can vary between identical runs. This is an inherent property of large language models, caused by:
- • Temperature and sampling parameters
- • Timing of web retrieval (content changes over time)
- • Model updates and version changes
- • Retrieval index refreshes
This is why we use repeated testing — running the same prompt multiple times to estimate answer stability and confidence.
Scoring formulas
Accuracy Score
A weighted composite of five metrics:
Mention Rate
Percentage of completed responses that mention the brand by name or alias.
Recommendation Rate
Percentage of responses that actively recommend the brand. Requires language indicating suitability, preference, or direct proposal.
Citation Rate
Percentage of responses citing the brand's official domain or a third-party page materially discussing the brand.
Position Score
First mentioned: 100 · Second: 70 · Third: 40 · Later: 20 · Not mentioned: 0
Sentiment Score
Positive: 100 · Mixed: 60 · Neutral: 50 · Negative: 0 · Not mentioned: 0
Share of Voice
Brand mentions ÷ (brand + all competitor mentions) within the same campaign.
Stability Score
Stability is calculated separately from accuracy, based on agreement between repeated runs regarding:
- • Brand mention (was the brand mentioned consistently?)
- • Recommendation (was the brand recommended consistently?)
- • Major competitors mentioned
- • Main cited domains
- • Overall sentiment
Important limitations
- • Scores indicate observed accuracy, not guaranteed future ranking.
- • The platform does not guarantee increased sales or AI recommendations.
- • Results are snapshots from API calls at a specific time. AI models evolve continuously.
- • We never create fake live results, fake customers, or misleading performance claims.
Demo mode
Every workspace can operate in demo mode with realistic sample data. No API keys are required. Demo results are clearly labeled and are not real API responses.