E-commerce voice AI: build an order and delivery agent

Build a voice AI agent that takes e-commerce orders and schedules deliveries. Step-by-step guide using Pulse, Electron, Lightning v3.1, and Shopify APIs
A working voice AI stack for e-commerce order taking and delivery scheduling has six layers: a telephony channel, speech-to-text, a language model that can call tools, text-to-speech, a commerce or order management API, and a webhook loop that carries fulfillment changes back to the agent. This guide builds that stack end to end using Twilio or Vonage for calls, Smallest AI's Pulse, Electron and Lightning v3.1 for the voice pipeline, and the Shopify Admin API as the order system. The same pattern applies to WooCommerce or a custom OMS.
What you will build and who this guide is for
By the end, you will have a phone agent that can:
Capture and verify an order number.
Confirm or update the delivery address.
Offer two or three delivery windows and confirm one.
Write the schedule to your OMS or carrier system.
Send status updates when fulfillment events arrive.
Hand the call to a human, with context, when it should not continue.
This guide is written for e-commerce and contact center engineering leads, and for enterprise contact center managers assessing whether voice automation is feasible for their order and delivery queues.
Prerequisites
A phone number and voice channel (Twilio, Vonage, or similar) that can stream call audio and send status callbacks.
A Smallest AI API key for Pulse (STT), Electron (LLM), and Lightning v3.1 (TTS).
Access to your order data: a Shopify Admin API token, or credentials for your OMS and carrier APIs.
A public HTTPS endpoint for call-state and fulfillment webhooks.
A session store such as Redis or Postgres.
Node.js 18 or later.
The happy path and three failure paths
The happy path is: caller gives an order number, passes verification, requests a new window, picks one, confirms, and hears a summary. Design for these failure paths from the start:
Missing or invalid order number. The lookup returns nothing, or ASR confidence is low.
Unsupported delivery window. The requested date is a holiday, past the cutoff, or outside carrier capacity.
Backend error or conflict. The OMS returns a 409, a timeout, or a 5xx.
Choose the tools for each layer
Packaged Shopify apps such as Consio cover standard phone support. Its App Store listing shows a Free plan, $30/month for 100 minutes, and $60/month for 400 minutes, billed in USD. Teams that need custom delivery-window logic, their own OMS, or control over escalation usually build on a voice infrastructure layer instead. For a wider view of the options, see this comparison of voice agent platforms.
Layer | Role in this build | Tool |
|---|---|---|
Telephony | Inbound calls, media streaming, call status callbacks | |
STT | Streaming transcripts, timestamps, VAD events, PII/PCI redaction | Pulse (WS /waves/v1/stt/live) |
Dialogue and tools | Entity extraction, tool calling, JSON mode | Electron (/v1/chat/completions) |
TTS | Spoken confirmations and prompts | Lightning v3.1 (POST /waves/v1/tts, WSS /waves/v1/tts/live) |
Order system | Lookups, fulfillment, tracking | Shopify Admin API or your OMS |
Orchestration | Agent builder, versioning, deployment, call analytics, webhooks | Atoms |
Atoms is the voice agent platform from Smallest AI. It covers the agent builder with versioning, phone deployment, transcript and call monitoring, and webhooks. If you prefer not to run your own orchestration, you can configure the flows in this guide there and keep the backend tool endpoints the same.
Step 1: Map the architecture
Draw this before writing code. Every arrow is a place where something can fail.

Responsibilities:
ASR produces transcripts, timestamps, and confidence values.
Dialogue manager extracts entities and picks the next tool call.
Tool layer is the only code allowed to touch the OMS. It validates inputs and returns structured results.
Webhook handler verifies signatures, deduplicates events, and updates session state.
Escalation service transfers the call and passes the transcript plus extracted parameters to your CRM or ticketing system.
Checkpoint: you can name the owner of every box and what happens when it times out. Store a session record keyed by call ID (for Twilio, the CallSid) containing the verified order ID, extracted entities, and idempotency keys. You need it for retries and for out-of-order events.
If you want a lower-latency alternative to the STT, LLM, and TTS chain, Hydra is a full-duplex speech-to-speech model at WSS /waves/v1/s2s with built-in barge-in. The pipeline in this guide gives you explicit control over each tool call, which matters when a call writes to an order system. For background on how order status flows are wired, see this overview of order status voice flows.
Step 2: Write the conversation scripts
Write the scripts before the prompts. Each turn should do one job.
Script A: inbound order status and delivery change
Turn | Agent | Backend action |
|---|---|---|
1 | "Thanks for calling. Please say your order number, digits only." | None |
2 | "I heard 1 0 4 2. Is that correct?" | Read-back of the extracted ID |
3 | "For security, please tell me the ZIP code on the order." |
|
4 | "Your order ships to 18 Maple Street in Austin. It is scheduled for Monday, 1 PM to 5 PM. What would you like to do?" | Fetch fulfillment state |
5 | Caller: "Can I get it Tuesday morning?" |
|
6 | "I can offer Tuesday 10 AM to 12 PM, or Tuesday 2 PM to 4 PM. Which works?" | Max 3 options |
7 | "Tuesday between 10:00 AM and 12:00 PM, Central Time. Say 'confirm' to proceed." | None |
8 | "Done. You will get a text confirmation shortly. Anything else?" |
|
Script B: new phone order with a delivery window
Greet and ask for the customer's name and phone number (or match by caller ID).
Confirm each item and quantity, one item at a time. Read the full list back before moving on.
Collect the delivery address, then read it back in normalized form.
Offer two or three delivery windows and confirm one.
Do not take card numbers by voice. Send a secure payment link by SMS, or transfer to a PCI-compliant payment IVR.
Read back the order summary and ask for "confirm" before creating the order.
Order creation should happen only after the explicit confirmation turn. That single rule prevents most costly mistakes.
SSML for confirmations
Keep confirmations short and spell out the window. Use say-as for digits and explicit times.
Confirm SSML support for your chosen voice in the Waves API reference before relying on specific tags. The same wording works as plain text if a tag is ignored.
Step 3: Define entity extraction rules
Validate every entity in code, not only in the prompt.
Order ID. Normalize to the format your store uses. A Shopify order name such as #1042 is a different value from its internal numeric ID. Strip spaces, convert spoken digits ("one oh four two") to numerals, and always read back.
Verification. Require a second factor, such as the shipping ZIP or last name, before disclosing or changing anything. Allow two attempts, then escalate.
Address. Collect street, unit, city, state, and ZIP as separate fields. Validate against an address verification service. Read back the normalized result, not the raw transcript.
Delivery window. Resolve phrases like "Tuesday morning" to an absolute start and end in the customer's timezone. Store ISO 8601 values with an offset.
Delivery versus pickup. If the caller says "I will grab it," ask once: "Do you want delivery to your address, or pickup?" Never assume.
Pulse settings. Turn on PII/PCI redaction for stored transcripts. Use the timestamps and VAD events to detect when a caller is still speaking before the agent responds.
Checkpoint: a table of golden test inputs (spoken order IDs, messy addresses, relative dates) with expected normalized outputs. You will reuse it in testing.
Step 4: Stream call audio to Pulse
Pulse supports streaming over WS /waves/v1/stt/live and batch over POST /waves/v1/stt/, with the model chosen by the model query parameter. This Node.js sketch forwards audio from your telephony stream. Field names in the incoming JSON are illustrative, so map them to the response schema in the docs.
Send final transcripts to the dialogue manager together with their confidence values. Carry the confidence through the whole turn, because Step 5 depends on it.
Step 5: Call Electron with tools
Electron is OpenAI-compatible at /v1/chat/completions, with the client base URL set to https://api.smallest.ai/waves/v1. It supports tool calling, JSON mode, and streaming, with typical time-to-first-token under 300 ms and a 32K context window.
The system prompt should state three rules: ask for one thing per turn, never call schedule_delivery before an explicit "confirm," and call escalate_to_human when confidence is low or the request is out of policy.
A tool call produced by the model looks like this:
Use JSON mode (response_format: { type: "json_object" }) for any step where you need a structured extraction rather than a spoken reply.
Checkpoint: reject any tool call that fails schema validation or carries an ASR confidence below your threshold (start at 0.80 for order IDs and tune from call audits). Re-ask once, then escalate.
Step 6: Build the backend tool functions
Shopify does not provide a native customer-selected delivery window field. Store the window in your OMS or carrier system, and mirror it on the Shopify order with a metafield or note attribute so human agents can see it.
The Shopify concepts to know:
GET
/admin/api/{version}/orders/{order_id}/fulfillment_orders.jsonreturns thefulfillment_order_idvalues.POST
/admin/api/{version}/fulfillments.jsoncreates a fulfillment and attaches tracking.POST
/admin/api/{version}/fulfillments/{fulfillment_id}/update_tracking.jsonupdates tracking details.Shopify's
OrderFulfillmentStatusaggregates states from unfulfilled to fully fulfilled, including in progress and on hold. Read it before offering a change. A shipped order usually cannot take a new window.
Check the current Shopify Admin API reference for the API version you target, because Shopify is moving more fulfillment work to GraphQL.
The idempotency key matters. Callers repeat themselves and networks retry, so the same confirmation must never create two schedules.
Return the tool result to Electron as a tool message, then let the model phrase the response. Keep the final spoken confirmation templated so the time window is never paraphrased.
Step 7: Speak the confirmation with Lightning v3.1
Lightning v3.1 offers POST /waves/v1/tts for short prompts, plus streaming through POST /waves/v1/tts/live (SSE) and WSS /waves/v1/tts/live. Use the HTTP route for fixed confirmations that you can cache, and a streaming transport for dynamic replies where time to first audio matters.
If you plan to clone a brand voice, Lightning v3.1 supports cloning from 5 to 15 seconds of reference audio. Lightning v3.1 Pro does not offer cloning. Obtain the speaker's explicit consent before cloning any voice, and keep a record of it.
Step 8: Handle webhooks and out-of-order events
Two webhook streams arrive at your server: call status from telephony, and fulfillment events from Shopify or your carrier.
Telephony. Twilio sends only the final completed event if you set a StatusCallback URL but omit StatusCallbackEvent, so request the events you need. Payloads include CallSid, CallStatus, CallbackSource, SequenceNumber, and CallDuration (on completed). Callbacks can arrive out of order, so compare SequenceNumber before updating state.
Fulfillment. Subscribe through POST /admin/api/{version}/webhooks.json (for example, to fulfillments/create and fulfillments/update). Shopify signs each request with the X-Shopify-Hmac-Sha256 header, a base64 HMAC-SHA256 of the raw body.
Verify against the raw body, not parsed JSON, or valid signatures will fail. Acknowledge quickly and process through a queue, with a dead-letter queue for events that keep failing.
A worker then updates the session record and triggers a notification, such as an SMS saying "Your delivery window is confirmed for Tuesday, 10 AM to 12 PM." Because Lightning supports spoken updates, the same event can also drive an outbound voice message. See this guide to delivery notifications for that pattern.
Step 9: Build the escalation path
Escalate when any of these is true:
ASR confidence stays below threshold after one re-ask.
The order ID is ambiguous or verification fails twice.
The requested window is unavailable and the caller rejects the alternatives.
Address validation fails.
The request involves payment, refunds, or authorization.
The caller asks for a person, or the agent has misunderstood twice in a row.
Use a warm handoff. Say, "I am connecting you to a specialist who can see your order and what we discussed," then pass the summary so the caller never repeats it.
Step 10: Test, monitor, and check production readiness
Testing
Unit tests for entity validators, using the golden table from Step 3.
Integration tests for order lookups and scheduling, including 409 and timeout responses.
End-to-end call simulations with recorded audio in noisy conditions and varied accents.
Golden transcript tests for spoken order IDs and addresses.
Metrics to track
Answer rate and call completion rate.
Successful scheduling rate.
Average tool-call latency.
Resolution rate by intent (order status, scheduling, returns).
Escalation rate and escalation reasons.
Monitoring. Alert on webhook error rates, retry counts, dead-letter queue depth, and spikes in scheduling failures. Review a sample of transcripts weekly and adjust confidence thresholds, prompts, and tool descriptions based on what you find. Atoms provides call analytics and transcript review, which cover much of this without custom dashboards.
Production readiness checklist
Tool functions are idempotent.
Webhook signatures are verified on every endpoint.
Transcripts are redacted for PII before storage.
Rate limits and backoff are in place for Shopify and carrier APIs.
Session state survives a restart.
A human fallback exists for every automated path.
Apply compliance and UX rules before launch
Recording and cloning. Disclose call recording at the start of the call and obtain consent where required. Voice cloning requires explicit consent from the speaker.
Payments. Do not capture or store card data in the voice flow unless you operate a PCI DSS compliant payment architecture. Use a payment link or a compliant payment IVR.
Outbound reminders. Automated calls and texts about scheduling can fall under the TCPA. Businesses generally need prior express written consent for automated telemarketing messages, and must honor opt-outs promptly. An FCC change effective April 11, 2025 tightened revocation handling. Confirm your obligations with counsel, and build an opt-out path into every message.
Order-taking UX. Confirm the order number early, ask for the minimum entities, keep prompts short, and confirm before any write to the OMS.
Scheduling UX. Offer two or three windows, state the timezone, and handle "as soon as possible" by offering the earliest real window. Check holidays, and warn the caller when an address change could invalidate the delivery promise.
For more on the broader build, see this walkthrough on building voice agents for e-commerce.
Troubleshooting common problems
The agent keeps mishearing order numbers. Ask for digits only, read them back digit by digit, and accept DTMF keypad entry as a fallback.
Webhook signature checks fail. You are probably hashing parsed JSON. Hash the raw request body.
Callers get double confirmations. Check that the idempotency key includes the call ID, order ID, and window start.
Status events overwrite newer state. Compare SequenceNumber (telephony) or an event timestamp before applying updates.
The agent talks over the caller. Use Pulse VAD events to hold responses, or evaluate Hydra for built-in barge-in.
Scheduling succeeds in the OMS but the human agent sees nothing. Mirror the window to a Shopify metafield or note attribute after a successful write.
What you have built and where to go next
You now have a voice agent that verifies a caller, extracts order and delivery entities, proposes and confirms windows, writes the schedule with idempotent calls, reacts to fulfillment webhooks, and hands off to a human with full context. The pipeline uses Pulse for transcription, Electron for tool calling, and Lightning v3.1 for speech.
Sensible next steps:
Run the golden-transcript suite and a week of shadow calls before routing live traffic.
Add returns and exchange intents using the same tool pattern.
Move orchestration, versioning, and call monitoring into Atoms once flows stabilize.
Pilot Hydra on a low-risk intent such as order status to compare latency and caller experience.
Add outbound delivery reminders only after consent capture and opt-out handling are tested.
Frequently asked questions
How do I build a voice agent for orders and delivery?
Can this agent update delivery windows in my order system?
What should I do when the order number is missing or invalid?
How do I handle backend conflicts or 5xx errors during scheduling?
How long does it take to build a first working integration?
