How to Test an Arabic AI Receptionist Before You Buy It

Before you sign with any AI receptionist vendor in Saudi Arabia, call the demo number yourself and spend ten minutes trying to break it in Arabic. Every vendor page promises fluent Arabic. Almost none of them tell you how to check — and the gap between "supports Arabic" and "speaks the way your customers speak" is where most disappointing deployments start.

This guide is written by Modern Intelligent Solutions, which builds Arabic-first AI voice, WhatsApp, and web-chat agents for businesses in Saudi Arabia and the Gulf. It is aimed at owners and managers of clinics, salons, restaurants, and real-estate offices — businesses where the phone is the front door and one person is answering it between other jobs. It gives you a test you can run yourself, on any vendor, including us.

The demo you are shown is not the test you need

A vendor demo is a scripted call in a quiet room. It is designed to succeed, and it will.

What you watch in a demo is the vendor's best case: a clear question, asked in the phrasing the agent was tuned on, with no interruption and no background noise. Your customers do none of that. They call from a car with the window down. They change their mind mid-sentence. They use the word for "appointment" that their grandmother used. They start in Arabic and drop an English word in the middle, because that is how people actually talk here.

So the useful test is not "did the demo work." It is "what happens when I stop cooperating." You do not need technical knowledge to run that test. You need a phone, ten minutes, and a willingness to be an awkward caller.

The seven checks below are ordered deliberately: the earliest ones are the ones most likely to fail, and a failure in the first three tells you most of what you need to know.

The ALLO ten-minute phone test

Call the vendor's demo line as if you were your own customer, and work through these seven checks in order. Write down a yes or a no for each one. Do not let the vendor drive the call.

Check 1 — Is this your customers' Arabic, or news-anchor Arabic?

Listen to the first two sentences and ask whether a caller would assume they had reached a person in Riyadh.

Modern Standard Arabic is the Arabic of newsreaders and official documents. It is correct, and it is not how anyone books a haircut. An agent that answers in MSA sounds formal to the point of strange, and many callers respond by switching to English or by hanging up. What you are listening for is Saudi dialect — the ordinary spoken register your customers use with each other.

This is the single most common failure, and it is worth knowing that some vendors selling into the Gulf describe Saudi dialect as in development rather than shipped. The difference is audible in the first breath.

Check 2 — Give it a real Saudi name

Say a full name the way a caller would say it — for example, "عبدالرحمن بن مشاري القحطاني" — and then ask the agent to read it back.

Names are where speech recognition tuned on general Arabic quietly falls apart. Listen for whether the agent captures the whole name including the بن, whether it keeps the الـ on the family name, and whether it can repeat it without mangling it. If a booking lands under a name your staff cannot recognise on the day, the booking is worse than useless.

Check 3 — Test the calendar the way people speak about it

Ask for an appointment using the phrasing your callers actually use, not a date format.

Try "بعد بكرة بالمغرب" (the day after tomorrow, around Maghrib). Try "أول يوم بعد الإجازة" (the first day after the weekend). Ask whether it can take a Hijri date if your customers use one. Then check the two things that break most often here: that the agent knows the Saudi working week runs Sunday to Thursday with Friday and Saturday as the weekend, and that it treats prayer times as real landmarks in a day rather than words it does not recognise.

An agent that only accepts "Tuesday at 4pm" is a web form with a voice on top of it.

Check 4 — Interrupt it

Start talking over the agent while it is still speaking, the way a caller in a hurry does.

You are testing whether it stops and listens, or whether it keeps reciting its sentence to the end while you talk. This is the behaviour that makes people describe an agent as robotic even when every word it said was correct. A good agent yields the moment you speak. Notice the pause after you finish, too: if the silence is long enough to make you wonder whether the call dropped, your customers will wonder the same thing.

Check 5 — Mix English into the middle of an Arabic sentence

Say something like "أبغى أحجز appointment الأسبوع الجاي."

Saudi callers code-switch constantly, and an agent built as separate English and Arabic modes tends to break at exactly that word — it either loses the sentence or answers the rest of the call in English. Confirm that it takes the English word in stride and stays in Arabic. Then do the reverse: start the call in English and switch to Arabic halfway, and check that it follows you rather than forcing you back.

Check 6 — Ask it something it should not answer

Ask a question that belongs to a human at your business, and watch what it does with it.

For a clinic, describe a symptom and ask whether it is serious. For a salon, ask which treatment suits your hair. For a restaurant, ask whether a dish is safe with a specific allergy. The right answer is a clean, quick handoff — it offers to connect you or to take your number for a callback, and it does that without inventing anything. The wrong answer is a confident, plausible, invented one. This check matters more than the six around it, because this is the failure that damages your business rather than merely annoying a caller.

Afterwards, ask the vendor what exactly triggers a handoff, and whether you control that list.

Check 7 — Ask where you are

Ask for directions the way a Saudi caller asks: by district and landmark, not by street address.

Try "وينكم بالضبط؟ أنا جاي من حي النرجس" and see whether the agent can work with a district name, a well-known landmark, or an exit number, rather than reciting a street address nobody navigates by. If your business depends on walk-ins, this check is worth more than it looks.

What to ask the vendor after you hang up

The phone test tells you how the agent sounds. These questions tell you whether it can run your business, and they are the ones a demo will not raise on its own.

• Can it write to my system, or only read from it? An agent that can tell a caller your opening hours but cannot put a booking in your calendar has moved the work, not removed it. Ask which of your systems it reads and which it writes to.
• What happens when it does not know? You have tested this once; ask them to describe the rule behind it, and whether you can change that rule without waiting on them.
• Who owns the recordings and transcripts, and can I export them myself? Your call recordings are your customer data. Get the answer in writing.
• Where is the data stored, and what happens to the personal details callers give it? In Saudi Arabia this sits under the Personal Data Protection Law, so it is a question with a real answer rather than a formality.
• Can I hear a recording of an ordinary call in my dialect? Not the polished demo — a real one, with a customer who was not cooperating.
• What does it cost when my phones are busiest? Pricing that looks fine against an average month can look different in Ramadan, at Eid, or in a seasonal rush.

How to score it

Treat the first three checks as pass-or-fail and the rest as a ranking.

If an agent fails Check 1, Check 2, or Check 3 — the dialect, the names, or the way people speak about dates — it is not ready for your customers, and no feature list compensates for it. Those three are about whether the agent belongs in Saudi Arabia at all.

Checks 4 through 7 are how you separate two vendors that both cleared the first three. Run the same ten minutes on each one, in the same order, and compare your own notes rather than their brochures.

What this test does not tell you

The phone test is a filter, not a full evaluation, and its limits are worth stating plainly.

It will not tell you how the agent behaves at volume, because you are one caller. It will not show you how it holds up after a month of real customers asking things nobody anticipated. It will not tell you how quickly the vendor fixes something once you report it — which, over a year, matters more than anything you can hear in ten minutes. And it cannot tell you whether the integration with your booking system will hold, because that depends on your system as much as on theirs.

What it does do is stop you from buying an agent that cannot speak to your customers. That is the failure that is both the most common and the easiest to catch, and ten minutes is a cheap price for catching it.

Where to start

If the phone is the front door of your business, the honest first step is not shopping for a vendor at all — it is finding out how many calls you are missing now and what those callers wanted. A clinic can start with the missed-call week audit, which turns a vague sense that the phones are a problem into a scoped list of what an agent would have to handle.

If you would rather run the ten-minute test on us first, that is the right instinct — talk to Modern Intelligent Solutions and ask for the demo line. Bring a difficult caller. We build for clinics and medical centres, restaurants and hospitality, and real-estate offices in Saudi Arabia, in the Arabic your customers actually speak.