What Data Should a Real-Time AI Avatar Provider Receive?

Decide what belongs in your SaaS application, what the avatar layer actually needs, and how to review the boundary before launch.

Spatius Team12 min read 分钟阅读
On this page

When a SaaS team adds an AI avatar, the easy mistake is to treat the avatar provider as if it needs the full context of the conversation: user records, permissions, retrieval results, tool outputs, prompts, and transcripts.

It usually does not.

The right boundary starts with the job of the layer you are adding. In Spatius, Motion Server receives avatar speech audio and returns real-time motion data. AvatarKit renders the avatar locally in the client. Your application, agent framework, or backend remains responsible for conversation logic, including ASR, LLM, TTS, turn-taking, permissions, knowledge, tools, and workflows. Spatius does not return a finished video. See the Spatius Developer Docs Map for the current product model.

That gives product and engineering teams a straightforward design principle: keep customer and business context inside the system that needs to reason over it; pass the avatar layer only what it needs to present the approved response.

A SaaS user reviewing an AI avatar beside consent and data-security panels.

Key takeaways

  • Start from the output the avatar layer needs, not from every field your agent already has.
  • In a Spatius flow, the core presentation input is avatar speech audio; the output is motion data for local AvatarKit rendering.
  • Keep identity, permissions, knowledge, prompts, tool calls, CRM data, and product analytics in your application unless a separately reviewed requirement says otherwise.
  • Treat a spoken reply as content that may contain sensitive information. Apply your product controls before it becomes avatar speech audio.
  • Document the purpose of every extra field, then review the chosen integration path with product, engineering, security, and legal stakeholders as appropriate.

Start with the system boundary, not a privacy slogan

“We do not send customer data” is rarely a useful design statement on its own. A spoken reply can itself contain account information, a troubleshooting instruction, or a summary of a customer record. The useful question is more concrete:

What data is necessary for this component to produce the experience it owns?

For a real-time avatar layer, that is different from what your agent needs to decide what to say. Your agent may need retrieval, customer state, role-based permissions, business rules, and tool results. The avatar layer does not need those inputs merely to animate a response that has already been approved by your application.

System boundary diagram showing a SaaS application retaining identity, permissions, context, knowledge, and tools while only approved avatar speech audio moves to Motion Server and returned motion data is rendered locally by AvatarKit.

This distinction is practical, not cosmetic. It lets your team preserve the controls it already operates:

  • Identity and access: Your product decides who the user is, what tenant they belong to, and what they may see or do.
  • Conversation and knowledge: Your agent decides which sources to use, what context is relevant, and how to form a response.
  • Actions and records: Your application owns tool calls, approvals, CRM updates, support cases, and audit trails.
  • Presentation: The avatar layer receives the speech it should present and produces the motion needed for rendering.

The result is a clearer architecture and a clearer review process. Adding an avatar does not require moving the intelligence of your product into the presentation layer.

What Spatius needs in the core motion path

For the standard Spatius model, the important input is the audio of the avatar’s spoken reply. Motion Server turns that audio into motion data; AvatarKit consumes the returned motion data and renders the avatar locally. The exact connection path depends on the integration you choose, but the product boundary remains the same. The integration guide describes Direct Mode, platform integrations, and Backend Mode.

Flow diagram showing an application producing an approved spoken reply, sending avatar speech audio to Spatius Motion Server, receiving motion data, and rendering the avatar locally with AvatarKit.

Data that normally stays in your application

The following information is usually part of your product or agent layer—not a default input to an avatar motion provider:

Data or capabilityWhy your application should own itWhat the avatar layer needs instead
User, account, and tenant recordsThese determine identity, entitlement, and business context.The final response that your application allows the avatar to say.
Roles and permissionsAuthorization decisions must remain with the system that enforces them.No raw role model is needed to animate speech.
System prompts, conversation history, and retrieval contextThese shape the answer and may contain internal or customer information.The approved speech audio generated from that context.
Knowledge-base documents and attachmentsYour agent needs them to reason; animation does not.No document payload by default.
Tool credentials, tool inputs, and tool resultsThese trigger or record business actions.A response after your application has completed any permitted action.
Product analytics and CRM eventsThese belong in your product’s measurement and operations stack.Only the integration telemetry you intentionally implement and review.

This is not a claim that any data is categorically forbidden. It is a starting point for minimization. If your architecture proposes an additional field, be able to explain its purpose, who can access it, how long it is needed, and why the avatar experience cannot work without it.

The spoken reply deserves its own review

It is tempting to call avatar speech “only audio.” In a business product, it can still contain meaningful content. The application should decide what may be spoken before TTS produces the avatar’s audio. That can include the same kinds of product controls you already use for text or voice responses:

  • applying the user’s current permissions before an answer is generated;
  • requiring confirmation before a reply describes or triggers a consequential action;
  • limiting the scope of a pilot to approved topics or workflows;
  • choosing how an answer is redacted, summarized, or escalated when the product cannot safely answer.

The key is ownership: the avatar layer presents an approved reply; it should not become the place where your product makes authorization or tool-use decisions.

Choose the path, then map the data flow

The same data-boundary principle applies across integration paths, but the transport and runtime ownership differ. Do not treat a single protocol as universal. Choose the path that matches the runtime your team already operates.

Direct Mode: client-to-Motion Server for existing speech audio

With Direct Mode, your backend issues a Session Token and AvatarKit on the client connects directly to Motion Server over WebSocket. The client sends avatar speech audio, receives motion data, and renders locally. The token endpoint is not the runtime relay; it does not proxy the motion connection or run ASR, LLM, or TTS.

Direct Mode data-path diagram showing a SaaS backend issuing a Session Token while client-side AvatarKit sends avatar speech audio to Motion Server, receives motion data, and renders the avatar locally.

Direct Mode is a good mental model for data minimization because the connection has a narrow purpose: establish an authorized session for the avatar motion path, send the avatar speech audio, and receive motion data. Your product can keep its agent, context, and workflow logic where it already lives.

Backend Mode: more runtime ownership, not a reason to widen the payload

In Backend Mode, your backend runs the Server SDK pipeline and chooses the downstream transport to clients. That may be appropriate when your architecture needs that runtime control. It does not change the core question: what does the motion layer require to turn speech into motion?

More control over transport is not a reason to relay every internal object, transcript, or business event through the avatar path. Keep the boundary explicit in the backend interface as well.

A five-question test for every extra field

When a proposed integration includes more than the spoken reply and the credentials needed to establish the selected path, review the field deliberately. A compact design review can prevent a vague “we may need it later” decision from becoming a permanent data dependency.

Five-step data review checklist for an avatar integration: name the purpose, remove unneeded fields, keep authorization in the app, test with a limited scope, and document the approved data flow.
  1. What experience function does this field support? Name the exact behavior it enables.
  2. Can the application perform that function before generating speech? If yes, keep the field in the product or agent layer.
  3. Does the field change authorization, retrieval, or a tool action? If yes, it belongs in your application boundary.
  4. Can a smaller value do the job? Prefer an approved spoken summary over a raw record, document, transcript, or credential.
  5. Has the path been reviewed and documented? Record the purpose, data owner, selected integration path, and the condition for removing the field later.

For an early product pilot, testing a limited workflow with non-production or carefully scoped data is often easier to review than beginning with your most complex or sensitive customer scenario. The right scope is a product decision; it should be made with the teams responsible for that workflow.

Design for a narrow presentation contract

An avatar integration works best when the contract is simple enough to describe in one sentence:

“Our application decides what the avatar may say. We send the resulting avatar speech audio to the motion layer and render the returned motion data in the client.”

That contract is also useful for implementation planning. It gives each team a clear responsibility:

TeamOwnsShould not assume the avatar layer owns
ProductUser moment, scope, wording, fallback, and next actionWhether an answer is helpful or appropriate for the workflow
AI / application engineeringASR, LLM, TTS, context, retrieval, tool calls, policy, and handoffThe decision to authorize an action or expose customer context
Client engineeringAvatarKit integration, UI controls, state handling, and local renderingThe source of truth for user permission or business logic
Security and legal reviewData handling decisions, vendor review, disclosure, and contractual requirementsA generalized marketing claim based only on an architecture diagram

No article can replace your own vendor, security, legal, or product review. Requirements vary by industry, customer agreement, and the data your application handles. The point of this pattern is to give that review a narrow, inspectable scope.

Common mistakes to avoid

Passing raw retrieval context “just in case”

Retrieval content belongs where answers are selected and grounded: your agent or backend. Passing raw documents to a presentation layer creates another path to review without improving the motion transformation itself.

Treating a session identifier as a product context object

An integration may use a session mechanism for authentication or correlation. That does not mean the avatar service needs a direct copy of your CRM profile, tenant record, or application-level permission model. Keep mappings and identifiers intentional and minimal.

Assuming text, voice, and avatar layers have the same data needs

They do not. A text surface may need only an approved response string. A TTS layer needs text or speech instructions. A real-time avatar motion layer needs avatar speech audio. Design each boundary for the job of that layer.

Making privacy promises from an incomplete implementation diagram

Architecture diagrams are useful, but they are not a substitute for the current terms, configuration, data-processing details, and internal review that apply to your deployment. Be precise about what the selected path does, and ask the right owners to approve anything beyond the documented product behavior.

Frequently asked questions

Does Spatius need our full conversation transcript?

Not for the core avatar motion function described in the Spatius docs. Motion Server receives avatar speech audio and returns motion data; AvatarKit renders the avatar locally. Your application or agent framework owns the conversation logic. See the Developer Docs Map.

Should we send our knowledge base, CRM records, or tool credentials to the avatar provider?

They are not default inputs to the motion path. Keep them in the application and agent layer that uses them to decide what the avatar may say or do. If a proposed integration requires additional data, document the specific purpose and review it with the appropriate product, security, and legal owners.

Does the avatar provider receive user microphone audio?

The documented Spatius motion path uses avatar speech audio as its input. User-input capture, ASR, LLM reasoning, and turn-taking are owned by your application, agent framework, or backend. Confirm the details of the integration path you select before implementation.

Does Spatius return a finished video file or video stream?

No. Motion Server returns motion data, and AvatarKit renders the avatar locally in the client. This distinction is central to the current product architecture.

What should we do before sending any additional data?

Write down the business purpose, confirm that the behavior cannot be performed within your application first, minimize the value, and have the required owners review the final implementation. Do not treat an unreviewed field as harmless merely because it is convenient to pass along.

Keep the avatar layer focused

The most durable avatar architecture is not the one that moves the most data. It is the one that keeps the customer-facing intelligence, controls, and records in the product that owns them—and sends the presentation layer the approved speech it needs to do its job.

If you are evaluating a real-time avatar layer for your SaaS product, request a Spatius demo to discuss an integration path that fits your existing application architecture.

Sources

Related Articles