Why the same spec produces wildly different outputs—and how we fixed it after 14 API failures.

The Setup: What We Were Building

We needed to generate AI Employee configurations for the Ema platform. These "personas" have two critical pieces:

  1. workflow_def — a JSON graph defining how the AI processes requests (triggers, routing, search, response generation)
  2. proto_config — a JSON object defining UI/voice/chat settings (widgets, voice characteristics, welcome messages)

Both must conform to an API contract that isn't fully documented. Deviations cause cryptic errors or silent failures.

We tried two approaches:

  • Compiler approach: Parse user intent → build typed spec → compile to JSON
  • LLM-orchestrated approach: Parse intent → create from template → merge configs → let the LLM handle transformations

This post documents what each produced, why they differed, and which problems appeared at each level.

The API Contract: What the Server Actually Expects

Before comparing outputs, here's what the Ema API requires (learned through failure):

Widget Format

// CORRECT: What the API accepts
{
  "name": "conversationSettings",
  "type": 39,
  "conversationSettings": {
    "welcomeMessage": "Hello, how can I help?",
    "identityAndPurpose": "You are a sales assistant..."
  }
}

// WRONG: What our compiler initially produced
{
  "widget_name": "conversationSettings",
  "widget_type_id": 39,
  "widget_config": {
    "welcomeMessage": "Hello, how can I help?"
  }
}

The config values must be nested under a key matching the widget's name field — not under a generic widget_config key.

Action Format

// CORRECT: Full action structure
{
  "name": "search_1",
  "action": {
    "name": { "namespaces": ["actions", "emainternal"], "name": "search" },
    "version": "v2"
  },
  "inputs": { ... },
  "displaySettings": {
    "displayName": "Knowledge Search",
    "description": "",
    "coordinates": { "x": 800, "y": 200 },
    "showConfig": 0
  },
  "typeArguments": {},
  "tools": [],
  "disableHumanInteraction": false
}

// WRONG: Missing required fields
{
  "name": "search_1",
  "action": {
    "name": { "namespaces": ["search", "emainternal"], "name": "search" }
  },
  "inputs": { ... }
}

Notice: namespaces must be ["actions", "emainternal"], not ["search", "emainternal"]. And version, displaySettings, typeArguments, tools are all required.

Approach 1: The Compiler

How It Works

flowchart LR
    A[User Input]:::primary --> B[parseNaturalLanguage]:::secondary --> C[WorkflowIntent]:::secondary --> D[intentToSpec]:::secondary --> E[WorkflowSpec]:::secondary --> F[compileWorkflow]:::accent --> G[JSON]:::accent

The compiler is deterministic. Given the same WorkflowSpec, it always produces the same workflow_def and proto_config.

The Code

// workflow-generator.ts (simplified)

function compileWorkflow(spec: WorkflowSpec): CompiledWorkflow {
  const actions = spec.nodes.map(node => buildAction(node));

  return {
    workflow_def: {
      workflowName: { name: { namespaces: ["ema", "workflows"], name: spec.name } },
      actions,
      enumTypes: [],
      results: buildResultMappings(spec.resultMappings),
    },
    proto_config: buildProtoConfig(spec),
  };
}

function buildAction(node: Node): Record<string, unknown> {
  const namespaces = ACTION_NAMESPACES[node.actionType];  // Static lookup
  const version = ACTION_VERSIONS[node.actionType];       // Static lookup

  return {
    name: node.id,
    action: { name: { namespaces, name: node.actionType }, version },
    inputs: buildInputs(node.inputs),
    displaySettings: { displayName: node.displayName, ... },
    typeArguments: {},
    tools: [],
  };
}

What It Produced (First Version)

For input: "Voice AI for sales development"

Generated proto_config.widgets:

[
  {
    "widget_name": "conversationSettings",
    "widget_type_id": 39,
    "widget_config": {}
  },
  {
    "widget_name": "voiceSettings",
    "widget_type_id": 38,
    "widget_config": {}
  }
]

Problems:

  1. Used widget_name instead of name
  2. Used widget_type_id instead of type
  3. Used generic widget_config instead of conversationSettings: {...}
  4. Config objects were empty — no welcome message, no identity

API Error:

Invalid persona config: Widget name is empty, persona_id: abc123

Why the Compiler Failed

The compiler had outdated assumptions about the API contract. It was written when the API accepted widget_name + widget_config. The API changed. The compiler didn't.

More fundamentally: compilers validate syntax, not semantics. The TypeScript compiler was happy. The output was "correct" TypeScript. But the API rejected it because the structure didn't match the runtime contract.

After Fixing the Compiler

We corrected the widget format:

function buildProtoConfig(spec: WorkflowSpec): Record<string, unknown> {
  const widgets = [];

  if (spec.personaType === "voice") {
    const vc = spec.voiceConfig ?? {};
    widgets.push({
      name: "conversationSettings", // Fixed: "name" not "widget_name"
      type: 39, // Fixed: "type" not "widget_type_id"
      conversationSettings: {
        // Fixed: config under matching key
        welcomeMessage: vc.welcomeMessage ?? `Hello, this is ${spec.name}.`,
        identityAndPurpose: vc.identityAndPurpose ?? spec.description,
        takeActionInstructions: vc.takeActionInstructions ?? "...",
        // ... full config
      },
    });
  }

  return { project_type: 5, widgets };
}

New output:

{
  "name": "conversationSettings",
  "type": 39,
  "conversationSettings": {
    "welcomeMessage": "Hello, this is Sales Assistant. How can I help you today?",
    "identityAndPurpose": "You are Sales Assistant. Voice AI for sales development."
  }
}

Widget config now passes validation.

Approach 1 Continued: The workflow_def Problem

Fixing proto_config wasn't enough. The workflow itself had issues.

What the Compiler Produced

{
  "workflowName": {
    "name": {
      "namespaces": ["ema", "workflows"],
      "name": "sales_assistant"
    }
  },
  "actions": [
    {
      "name": "voice_trigger",
      "action": {
        "name": {
          "namespaces": ["triggers", "emainternal"],
          "name": "voice_trigger"
        }
      },
      "inputs": {}
    },
    {
      "name": "search_1",
      "action": {
        "name": { "namespaces": ["search", "emainternal"], "name": "search" }
      },
      "inputs": {
        "query": {
          "actionOutput": {
            "actionName": "voice_trigger",
            "output": "user_query"
          }
        }
      }
    }
  ]
}

API Error

Internal Server Error

No detail. Just 500.

What Was Wrong

After comparing with a working workflow from the UI:

Field Compiler Output Expected
search namespace ["search", "emainternal"] ["actions", "emainternal"]
action.version (missing) "v2"
displaySettings (missing) { displayName: "", coordinates: {...} }
typeArguments (missing) {}
tools (missing) []
top-level enumTypes (missing) []
top-level namedResults (missing) {}

The compiler's buildAction() was missing 5 required fields. And the namespace mapping was wrong for search.

The Fix

const ACTION_NAMESPACES: Record<ActionType, string[]> = {
  search: ["actions", "emainternal"], // Fixed: was ["search", "emainternal"]
  respond_with_sources: ["actions", "emainternal"], // Fixed: was ["generation", "emainternal"]
  // ...
};

const ACTION_VERSIONS: Record<ActionType, string> = {
  search: "v2", // Added: version matters
  chat_trigger: "v1",
  // ...
};

function buildAction(node: Node): Record<string, unknown> {
  return {
    name: node.id,
    action: {
      name: {
        namespaces: ACTION_NAMESPACES[node.actionType],
        name: node.actionType,
      },
      version: ACTION_VERSIONS[node.actionType] ?? "v1", // Added
    },
    inputs: buildInputs(node.inputs),
    displaySettings: {
      // Added entire block
      displayName: node.displayName ?? "",
      description: node.description ?? "",
      coordinates: { x: 800, y: 200 },
      showConfig: 0,
    },
    typeArguments: {}, // Added
    tools: [], // Added
    disableHumanInteraction: node.disableHitl ?? false,
  };
}

Still Failed

Even with correct format, another error:

Workflow name does not match the persona's existing workflow

The compiler generated workflowName as sales_assistant. But when creating a new persona from template, the platform assigns its own workflow name. We were trying to overwrite it with a mismatched name.

This is where the compiler approach fundamentally broke down.

Approach 2: LLM-Orchestrated with Template Preservation

The Insight

The platform's templates already have valid workflow_def structures. Instead of generating a replacement, we should:

  1. Create persona from template (get valid workflow)
  2. Only update proto_config (settings, not structure)
  3. Customize workflow later via transformation, not replacement

The New Architecture

flowchart TB
    subgraph GREENFIELD["GREENFIELD (New Persona)"]
        G1[User Intent]:::primary --> G2[Create from Template]:::secondary
        G2 --> G3[Fetch Created Persona]:::secondary
        G3 --> G4["Merge proto_config"]:::secondary
        G4 --> G5["Update proto_config ONLY"]:::accent
    end
flowchart TB
    subgraph BROWNFIELD["BROWNFIELD (Existing Persona)"]
        B1[User Intent]:::primary --> B2[Fetch Persona]:::secondary
        B2 --> B3[Decompile workflow_def]:::secondary
        B3 --> B4["WorkflowSpec (typed)"]:::secondary
        B4 --> B5["LLM transforms spec"]:::agent
        B5 --> B6[Compile back to workflow_def]:::secondary
        B6 --> B7["Update persona"]:::accent
    end

The Code

// handlers-consolidated.ts - Greenfield flow

// Step 1: Create persona from template
const createResult = await client.createAiEmployee({
  name: personaName,
  description: args.description,
  template_id: templateId, // "00000000-0000-0000-0000-00000000001e" for Voice AI
});

const newPersonaId = createResult.persona_id;

// Step 2: Fetch to get template's valid structure
const newPersona = await client.getPersonaById(newPersonaId);
const existingProtoConfig = newPersona.proto_config ?? {};

// Step 3: Merge proto_config - template structure + generated values
const existingWidgets = existingProtoConfig.widgets ?? [];
const generatedWidgets = compiled.proto_config.widgets ?? [];

const widgetMap = new Map<string, Record<string, unknown>>();

// Start with template widgets (valid structure guaranteed)
for (const w of existingWidgets) {
  if (typeof w.name === "string" && w.name.trim().length > 0) {
    widgetMap.set(w.name, w);
  }
}

// Merge generated VALUES into template structure
for (const genWidget of generatedWidgets) {
  const widgetName = genWidget.name;
  const existing = widgetMap.get(widgetName);

  if (existing) {
    // Deep merge: keep structure, update values
    const merged = { ...existing };
    if (genWidget[widgetName]) {
      merged[widgetName] = {
        ...(existing[widgetName] || {}),
        ...genWidget[widgetName],
      };
    }
    widgetMap.set(widgetName, merged);
  } else {
    widgetMap.set(widgetName, genWidget);
  }
}

const mergedProtoConfig = {
  ...existingProtoConfig,
  widgets: Array.from(widgetMap.values()),
};

// Step 4: Update proto_config ONLY - don't touch workflow
await client.updateAiEmployee({
  persona_id: newPersonaId,
  proto_config: mergedProtoConfig,
  // NOTE: No workflow parameter - template workflow preserved
});

What This Produces

For the same input: "Voice AI for sales development"

Template's proto_config.widgets (after create):

[
  {
    "name": "conversationSettings",
    "type": 39,
    "conversationSettings": {
      "welcomeMessage": "Hello, I'm your AI assistant.",
      "identityAndPurpose": "You are an AI assistant.",
      "takeActionInstructions": "Help the user with their request.",
      "hangupInstructions": "End the call when the user's request is complete."
    }
  },
  {
    "name": "voiceSettings",
    "type": 38,
    "voiceSettings": {
      "languageHints": ["en-US"],
      "voiceModel": "default"
    }
  }
]

Generated values to merge:

{
  "conversationSettings": {
    "welcomeMessage": "Hello, this is Sales Assistant. How can I help you today?",
    "identityAndPurpose": "You are Sales Assistant, a Voice AI for sales development..."
  }
}

Final merged output:

[
  {
    "name": "conversationSettings",
    "type": 39,
    "conversationSettings": {
      "welcomeMessage": "Hello, this is Sales Assistant. How can I help you today?",
      "identityAndPurpose": "You are Sales Assistant, a Voice AI for sales development...",
      "takeActionInstructions": "Help the user with their request.",
      "hangupInstructions": "End the call when the user's request is complete."
    }
  },
  {
    "name": "voiceSettings",
    "type": 38,
    "voiceSettings": {
      "languageHints": ["en-US"],
      "voiceModel": "default"
    }
  }
]

Key difference: Template provided valid structure. We only updated specific values. All required fields preserved.

Comparison: What Each Approach Got Wrong

Issue Compiler Approach Template + Merge Approach
Widget field names Wrong (widget_name vs name) Inherited from template (correct)
Config nesting Wrong (widget_config vs [widgetName]) Inherited from template (correct)
Empty configs Yes — no defaults No — template has defaults
Missing action fields Yes — 5 missing fields N/A — don't replace workflow
Wrong namespaces Yes — ["search", ...] N/A — don't replace workflow
Workflow name mismatch Yes — generates new name N/A — keep template workflow
First deploy success rate 0% (14 failures) 100% (after architecture change)

The Brownfield Case: LLM-Native Transformation

For existing personas, we need to modify workflows. This is where LLMs shine.

The Schema

// workflow-transformer.ts

export const WORKFLOW_SCHEMA_FOR_LLM = `
# Ema Workflow Schema

## WorkflowSpec
interface WorkflowSpec {
  name: string;
  description: string;
  personaType: "voice" | "chat" | "dashboard";
  nodes: Node[];
  resultMappings: ResultMapping[];
}

## Node
interface Node {
  id: string;                      // Unique identifier
  actionType: ActionType;          // search, call_llm, categorizer, etc.
  displayName: string;
  inputs?: Record<string, InputBinding>;
  runIf?: RunIfCondition;          // Conditional execution
}

## InputBinding (How nodes connect)
interface InputBinding {
  type: "action_output" | "inline_string" | "widget_config" | "llm_inferred";
  actionName?: string;   // Source node
  output?: string;       // Source output
  value?: string;        // Inline value
}
`;

How It Works

// Decompile: workflow_def (JSON) → WorkflowSpec (typed)
function decompileWorkflow(workflowDef: Record<string, unknown>): WorkflowSpec {
  const actions = workflowDef.actions as Array<Record<string, unknown>>;

  return {
    name: extractWorkflowName(workflowDef),
    description: (workflowDef.description as string) || "",
    personaType: detectPersonaType(actions),
    nodes: actions.map((action) => decompileAction(action)),
    resultMappings: extractResultMappings(workflowDef.results),
  };
}

// LLM receives: current spec + user instruction + schema
// LLM outputs: transformed spec

// Compile: WorkflowSpec → workflow_def
// Uses the same compileWorkflow() from before, but now with correct format

Why This Works Better

  1. LLMs understand intent: "Add a search node before the response" → LLM knows to wire trigger.user_query → search.query → respond.search_results

  2. Schema constrains output: The LLM can only produce valid WorkflowSpec shapes

  3. Compile handles format: The fixed compiler turns the spec into API-compliant JSON

  4. Existing structure preserved: We decompile first, so all existing nodes/wiring remain

Results: The Numbers

Metric Pure Compiler Template + Merge
Greenfield deploy success 0% 100%
API validation errors 14 distinct types 0
Fields requiring manual fix 12 0
Time to first working deploy 3+ hours debugging Instant
Widget config completeness Empty Full (template + custom)

Lessons Learned

1. Compilers enforce their contract, not the target's

The TypeScript compiler validated our code structure. It couldn't validate that ["search", "emainternal"] should be ["actions", "emainternal"]. That's a runtime contract the compiler knows nothing about.

2. Templates are contracts made concrete

Instead of reverse-engineering the API contract by trial and error, we used templates as a source of truth. If the template works, and we only add/modify values without changing structure, our output will work.

3. Merge, don't replace

The instinct was: "Generate complete output, deploy it."

The better approach: "Get working structure, merge our changes into it."

4. LLMs excel at transformation, not generation from scratch

Generating valid workflow_def from nothing required knowing every undocumented field. Transforming an existing valid workflow_def? The LLM just needs to understand the changes.

5. Validate at the boundary

We added widget validation before API calls:

function validateWidgetsForApi(widgets: Array<Record<string, unknown>>): {
  valid: boolean;
  errors: string[];
} {
  const errors: string[] = [];
  for (const w of widgets) {
    if (typeof w.name !== "string" || w.name.trim().length === 0) {
      errors.push(`Widget missing valid name: ${JSON.stringify(w)}`);
    }
    if (typeof w.type !== "number") {
      errors.push(`Widget missing type ID: ${w.name}`);
    }
  }
  return { valid: errors.length === 0, errors };
}

Catch problems before the API call. Much better than "Internal Server Error".

The Architecture That Worked

flowchart LR
    subgraph GREENFIELD["GREENFIELD"]
        G1[Intent]:::primary --> G2[Template]:::secondary --> G3[Fetch]:::secondary --> G4[Merge Config]:::secondary --> G5[Update Config]:::accent
    end

Template provides valid workflow — don't touch it

flowchart LR
    subgraph BROWNFIELD["BROWNFIELD"]
        B1[Intent]:::primary --> B2[Fetch]:::secondary --> B3[Decompile]:::secondary --> B4[LLM Transform]:::agent --> B5[Compile]:::secondary --> B6[Update]:::accent
    end

Existing workflow is source of truth — transform it

The compiler still exists. It handles WorkflowSpec → workflow_def conversion. But it no longer runs in isolation. It's part of a pipeline where valid structure comes from templates/existing data, and the compiler's job is format conversion — not structure invention.

Try This

If you're integrating with an API that has undocumented structural requirements:

  1. Get a working example — from the UI, from templates, from anywhere
  2. Diff your output against the working example — field by field
  3. Prefer modification over generation — transform existing valid structures
  4. Validate at boundaries — catch format errors before the API call
  5. Let LLMs handle intent — they're good at "add X to Y" transformations
  6. Let compilers handle correctness — after the structure is valid

The combination of template-based scaffolding + LLM transformation + compiler validation caught everything that each approach alone would have missed.

This post is part of a series on AI-assisted development workflows. See From Compiler Workflows to LLM Workflows for the conceptual foundation, and The MCP Mental Model for knowledge-first development patterns.