AI Virtual Humans: Turnkey Digital Human Development

Why Digital Human Looks Unnatural and Loses Sync

Desynchronization of articulation and facial expressions is almost never accidental.

Sound and facial animation are assembled by different processing branches, each with its own delay: synthesis outputs a finished utterance, while the animation layer receives control signals noticeably later.

A discrepancy of fractions of a second is read by the viewer as dubbing — they notice the mismatch before they realize what exactly is wrong.

On fast speech, artifacts appear as well. Lip poses cannot keep up with the pace, the mouth "sticks" on closed positions, and emotional facial expressions are suppressed so as not to conflict with articulation. Natural facial expressions suffer first: the face formally moves correctly, but turns into a mask.

A long session finishes off the result — without anchoring to a reference state, the character drifts, features change, the gaze stops holding contact, and behavior outside the script falls apart into jerks between pre-made blocks.

A separate cause is the disconnect between layers.

Response logic, articulation sync, and facial expressions live in different places and are coordinated "by eye": the text generator outputs a phrase, the animator adjusts it manually or by timing, and emotions are layered on top with a conditional switch. As soon as a line deviates from the script, the whole chain loses its footing.

We'll break down how this is resolved: where exactly rigid links are established between sound, face, and the meaning of the response, and why Digital Human after that maintains natural facial expressions both on fast speech and in a multi-hour dialogue.

What Changes After Implementation: Effect in Measurable Values

The effect of implementing Digital Human is visible not in a showcase video, but in the release pace and the amount of manual work that leaves the process. We record the values below before the start and measure them after launch — we use them to accept the result.

The base is the same for everyone: the character is assembled on the professional real-time platform Unreal Engine and a shared library of assets, so quality does not depend on who exactly assembles the next video.

• A new script — 1–2 days instead of a shooting week. Lines are assembled from ready-made blocks of facial expressions and intonations; the studio and actor are not involved in the cycle.
• Editing one line — 15–30 minutes. Only the changed fragment is reassembled; the rest of the video stays as is, and re-editing is not needed.
• A flow of 50–100 videos per month without quality degradation. Appearance, voice, and manner hold from release to release — the viewer does not see seams between shifts.
• A class of failures on long lines goes away. Lip desync and facial expression "sticking" are caught by checks during assembly, not after publication.
• The load on the team is one queue operator. Instead of the "scriptwriter — studio — editing" chain, there remain the text author and a support engineer.

These five values are the acceptance checklist: they are visible in the release log and in the edits for each version, not in a presentation.

Approaches to Creating Digital Human: Comparison of Methods

The choice of approach is determined by one question: should the digital human respond to the interlocutor here and now, or is pre-prepared material sufficient?

The answer determines the requirements for computing resources, the allowable response delay, and how edits are made after launch.

In practice, the options do not compete but are combined — hence the hybrid mode, where interactive dialogue is built on pre-recorded motion.

We do not tie the project to a single mode: first we calculate the real load and allowable delay, then we propose the option that will withstand it.

Development Stages: from Material Audit to Handover

The work proceeds in waves: first we remove uncertainty about materials, then assemble the character framework, and only then enable speech and facial expressions.

The order is fundamental — errors in the face scheme and voice data surface in live dialogue, where rework costs many times more than at the preparation stage. Below is the route that a Digital Human project goes through.

1. Audit of source materials. We review video, audio, photos, and texts: head angles, lighting stability, articulation clarity, audio track cleanliness, and usage rights. The output is a list of what is suitable for training and what needs additional shooting or re-recording.
2. Character scheme. We fix anthropometry and articulation zones: how speech sounds are linked to the position of the lips, jaw, and eyebrows. This is the framework onto which voice and facial expressions are then laid.
3. Voice and facial expression setup. We select timbre and pace, check the pronunciation of names, terms, and numerals, and synchronize the speech stream with facial movements. We separately test neutral facial expressions so that the character does not "drift" during pauses.
4. Test scenarios. We run dialogues, including inconvenient ones: interruption, a long question, an emotional line, and a change of wording on the fly. We record where facial expressions diverge from speech and where the response logic breaks.
5. Calibration. Based on the run protocols, we adjust synthesis and behavior parameters and repeat problematic scenarios until the discrepancies fall within an acceptable range.
6. Acceptance. We compare the result against agreed criteria: recognizability, intelligibility, appropriateness of reactions, and response time in dialogue.
7. Handover with training. We hand over configurations, scenario sets, and instructions, and show your team how to make edits and add lines without us.

Delivery Package: Models, Regulations, Documentation

Digital Human is not only an avatar and a voice, but a set of artifacts from which the character is assembled and which the client manages independently, without contacting us for every line edit.

We hand over everything in working form: what was configured at launch remains with your team along with the rules by which it can be changed safely.

• Character profile — a single source: appearance and its variants, voice settings, speech pace and pause length, a set of facial reactions and gestures. From the profile, the character is assembled across all channels, so an edit does not turn into reassembling every scenario from scratch.
• Library of scenarios and dialogue branches — lines, reactions to interruption, behavior when the interlocutor gives an incomplete answer. Each branch is tagged with which stage of the conversation it belongs to — you can see what can be changed without affecting the rest.
• Off-script behavior rules — what the character does with an unrecognized question, when addressing a forbidden topic, or when someone tries to take it out of role. A separate list describes situations where the conversation is handed over to a live employee, and how this happens.
• Stop list and content restrictions — topics, wording, and promises that the character does not utter under any circumstances, including attempts to persuade or rephrase the request.
• Update regulations — who approves a change to the profile or scenario, what checks are passed before publication, and how the previous version is restored if the new one behaves differently.
• Documentation — solution diagram, map of external systems connected to the character, data flow description, operating instructions, and analysis of typical failures with a procedure for action.
• Acceptance dialogue set — test conversations with expected character behavior, which your team uses to check each new version before release.
• Team training — practical sessions on managing the character, reviewing dialogue history, and adding new scenarios, with recordings and brief guides for each topic.

At the output, the client receives not a "black box" under our support, but a manageable product: the character can be developed by your own efforts, and we get involved where logic rework is needed.

Case: Removed Facial Expression Desync and Accelerated Video Release

Aurora Media produces a line of training videos with a digital presenter.

Before us, facial expressions were assembled blindly using phonetic labels: on a short phrase this worked, but in a long session of 8–10 minutes of continuous speech, articulation "drifted" by the middle.

Lip and sound desync reached a whole frame, the gaze lost fixation, and every third video went into complete re-recording.

What Was Changed

Speech was cut into phonetic segments and sets of visemes were tied to them with cross-checking against the audio track, and transitions between visemes were smoothed so that there were no jerks at consonant junctions.

We separately fixed the head pose and micro-expressions over long spans: the scene is calculated in blocks of 30–60 seconds, not as a whole, so the error does not accumulate from the beginning of the take to the end. Implementation proceeded in iterations — first we closed short scenes, then long sessions.

Metrics Before and After

Acceptance of the result was formalized into a checklist: lip closure on plosives, matching of stresses, gaze retention, and freeze-frames at disputed junctions.

The client checks the finished take against it and decides by regulation whether re-recording is needed — instead of an "by ear, like it or not" assessment.

What to Do If the Character Is Already Created in Another Tool?

Transferring Digital Human from another tool as a single file almost never works — and this is expected. Exchange formats (FBX, glTF) reliably output what relates to the character itself: geometry, skeleton, binding weights, textures, and material sets.

Everything that describes behavior — lighting, material layers, hair and cloth physics, facial expression settings — is tied to the environment, and during export it is either lost or arrives in a non-working state.

Therefore, we divide character migration into two streams. We import the model and skeleton and check them for integrity: bone hierarchy, binding weights, inverted normals, matching units of measurement and axes.

We rebuild the scene — lighting, camera angles, and material layers are restored for the target environment, because in transfer they will still produce a different result.

Quality verification is done against reference frames captured before migration.

We fix a set of states — neutral facial expressions, extreme positions of the eyebrows and lips, articulation on complex sounds, deformation in the shoulders and neck — and compare after transfer: discrepancies in facial geometry and in the behavior of materials under light are immediately visible.

The risks of transition are predictable: clipped facial expressions, broken weights, shifted proportions due to different units of measurement, and different color rendering. Part is treated by reconfiguration, part only by rebuilding the facial rig.

If there are too many losses, it is more honest to assemble the character anew while preserving the approved appearance: the result does not inherit the old limitations.

Technical Doubts Before Project Start

Before the start, almost every client team asks the same set of questions: what to do with their materials, how the character will behave on a non-standard line, how long a long session holds, and who owns the rights. Below are answers on the merits.

What will the character answer to a non-standard line?

Behavior is described by an answer policy: the character answers from the knowledge base, clarifies, or transfers the dialogue to an operator.

We separately fix topics where improvisation is unacceptable — deal terms, legal wording, industry recommendations — and there the character does not answer on its own.

How long does a session last without restart?

The character maintains continuous dialogue for a whole shift: the conversation context is preserved, and if the connection breaks, the session is raised automatically without losing the thread of the conversation.

Before handover, we check this with a load run — with a margin for duration and for the number of simultaneous interlocutors.

Who owns the rights to the image and content?

The source materials remain yours; the rights to the created image, lines, and code are transferred to the client along with the documentation.

The contract also fixes the scenario for changing the contractor: the work remains with you, and the character continues to work on your side.