An AI Avatar as Digital Sommelier

The Challenge

In fine wine retail, advice is the product. Anyone spending several hundred euro on a bottle wants to know why this one and not another. That advice is also the scarcest resource: a sommelier is there during opening hours, for one customer at a time.

The brief was to make that experience available digitally without cheapening it. A chat window would not have done it. It had to be a persona you look at and listen to, answering in German and genuinely knowing the catalogue.

Two conditions decided success or failure. The recommendations had to be real: a language model inventing plausible-sounding wines damages a brand that trades on connoisseurship. And the answer had to arrive quickly. With someone visibly waiting, the line between a conversation and a form sits at around three seconds.

Our Approach

The answer to the first condition is a RAG setup rather than a bare model. The entire catalogue sits as embeddings in PostgreSQL with pgvector; every question first retrieves the matching wines, and only that selection is handed to the model along with the question. What gets recommended is therefore only what is actually in stock, and the wine card appears directly beneath the answer.

The second condition is a question of the chain, not of a single component. From the microphone the input runs through Whisper for speech recognition, through the vector search, to Claude Sonnet for the answer in persona, through ElevenLabs for speech synthesis, and finally to the avatar with lip sync. Every one of those steps costs time, and only because each is optimised individually and streamed where possible does the answer begin under three seconds.

The persona itself is not a prompt but a piece of work in its own right: tone, stance and limits were developed together with the client experts until the avatar sounds like the house rather than like an assistant.

The Results

The prototype was built in ten working days and demonstrates the full chain, from spoken question to animated answer with a matching wine recommendation.

That fulfils its actual purpose: the setup transfers to other brands. What was built here for a wine merchant is designed as a template for further brand personas, because the expensive parts - vector search, latency chain, persona format - do not depend on wine but on the structure.