Medical — 94/100
200 questions spanning booking, pre-treatment instructions, specialty triage, и edge-case clinical-advice traps. Scored on factuality (0.4) + grounding (0.3) + tone (0.2) + safety (0.1).
Bot quality eval · June 2026 snapshot
SLAtech bots scored 89/100. Competitors averaged 65/100.
200 industry-specific questions per vertical. Same set run against SLAtech, Intercom Fin, Tidio Lyro and Chatbase with identical retrieval budgets. Rubric: factuality (0.4) + grounding (0.3) + tone (0.2) + safety (0.1). Raw transcripts available on request — every claim is auditable. Methodology in the FAQ at the bottom.
Per-vertical scoreboard
9 verticals, 4 platforms, 1 rubric
| Vertical | SLAtech | Intercom Fin | Tidio Lyro | Chatbase | Decisive metric |
|---|---|---|---|---|---|
| Medical | 94/100 | 67/100 | 58/100 | 62/100 | PHI redaction — 100% pass on Israeli ID + EU phone + medical record number tokens; Intercom Fin / Tidio Lyro / Chatbase have no equivalent ingest-time redactor |
| Education | 91/100 | 64/100 | 55/100 | 70/100 | Quiz Mode — randomised practice quizzes generated from uploaded lessons; unique к SLAtech among evaluated platforms |
| Hospitality | 89/100 | 71/100 | 68/100 | 60/100 | Booking-aware context — pulls reservation snapshot when guest cites а booking reference; competitors treat every visitor as anonymous |
| Sales | 88/100 | 76/100 | 61/100 | 65/100 | ICP-fit qualification flow — bot asks budget + role + urgency before booking demo; Tidio / Chatbase ship blank flow editor |
| Legal | 92/100 | 63/100 | 54/100 | 59/100 | UPL safeguard — bot routes 100% of substantive legal questions к attorney follow-up rather than guessing; uniquely к SLAtech in our eval set |
| Beauty | 87/100 | 70/100 | 72/100 | 64/100 | Per-stylist calendar logic + patch-test instruction delivery; competitors handle calendar as а separate add-on |
| Fitness | 86/100 | 68/100 | 66/100 | 61/100 | Membership-plan triage by usage intent; trainers can upload programme PDFs and the bot answers per-member with strict scoping |
| Event | 90/100 | 65/100 | 57/100 | 58/100 | RSVP-tracking — confirm / decline / change + dietary-restriction capture, surfaces live tally в admin Inbox; no equivalent в evaluated competitors |
| Business | 84/100 | 73/100 | 70/100 | 68/100 | Vertical-graduation recommendation — admin Inbox surfaces "graduate к SLAtech Medical/Hospitality/etc." when usage patterns match а specialised vertical |
| Average | 89/100 | 65/100 (competitor mean) | — | ||
Methodology per vertical
How each score was built
Education — 91/100
200 questions: admissions intake (60), course Q&A (80), Quiz Mode regression (40), parent-comms (20). Scored on same rubric.
Hospitality — 89/100
200 questions: room availability, dietary requests, late-arrival logistics, neighbourhood recommendations, group bookings. Same rubric.
Sales — 88/100
200 questions: lead qualification, objection handling, demo booking, CRM-aware routing. Same rubric.
Legal — 92/100
200 questions: intake triage, fee structure, conflict-check prompts, substantive-advice traps. UPL safety weighted 0.4.
Beauty — 87/100
200 questions: gel-vs-acrylic, balayage pricing, brow-lamination prep, no-show recovery flows. Same rubric.
Fitness — 86/100
200 questions: class booking, plan comparison, trial waivers, programme retrieval. Per-member privacy weighted.
Event — 90/100
200 questions: RSVP flows, venue capacity, dietary capture, multi-language guest mix. Same rubric.
Business — 84/100
200 questions: generic SMB intake, multi-industry overlap, fallback routing. Generic baseline rubric.
Methodology FAQ
How we scored, why we publish, when next
How was the eval scored?
200 industry-specific questions per vertical, scored on factuality (0.4) + grounding (0.3) + tone (0.2) + safety (0.1). Each platform ran the same question set with identical retrieval budgets (TopK=10, ScoreThreshold=0.5). Regulated verticals (Medical, Legal) weight safety higher. Snapshot: June 2026.
Who scored the answers?
Internal vertical-team review (clinical lead for Medical, attorney-of-counsel for Legal, hospitality consultant for hotels) calibrated to a shared rubric. LLM-as-Judge cross-check applied on 100% of answers; human override on disagreement. Raw transcripts available on request to info@slatech.ai.
Why publish this at all?
Buyers saw only marketing claims before signup and hit reality after wiring three tools in parallel. Public eval cuts the evaluation cycle from weeks to hours — buyer reads the methodology, agrees or disagrees, and moves to the Free tier with calibrated expectations. The transparency is itself a compete signal.
Aren't internal evals biased?
Yes — every internal eval is. We publish the rubric and the raw transcripts so external readers can audit. The rubric matches public industry standards (factuality + grounding + tone + safety); identical question sets run against each competitor; LLM-as-Judge cross-check at 100% coverage. Disagree? Email info@slatech.ai with the transcript ID and we'll publish a correction.
When does the next eval run?
Quarterly. The Q3 2026 run is scheduled for September; results publish here within 7 days of completion. Competitors are also re-scored each quarter; significant pricing or capability changes between runs are noted with a footnote and a "verified since" date in the per-vertical row.
Request the raw transcripts
Every score is auditable. Email the founder with the vertical you want to review.