TL;DR: Duet is a library for building UIs where humans and AI edit the same state. You define a schema (Zod for TypeScript, typed fields for Swift), and both human UI interactions and LLM JSON Patches validate against it. Available for TypeScript (Zustand + Zod) and Swift.
Table of Contents
Over the break, I had a chance to build a couple of interesting iOS apps, which I’ll share later. But while building those apps, I realized I was often trying to extract out the state or the current view of the application over to the LLM and ended up building a simple representation of the current state as JSON for the LLM to understand and react to.
With that insight, I extracted it out as an independent library, which will allow me to keep abstractions cleaner and maybe help somebody out there who’s solving similar problems.
GitHub: duet-kit (TypeScript) · duet-kit-swift (Swift)
There are going to be multiple parts to this post because I started out building it for Swift, but since I primarily build for the web, TypeScript support was important :). And while Duet solves the same problem in both languages, the implementation style and approaches are very different, and I’ll explain those in this post.
#The Problem
Over the last few years, we’ve seen Claude and ChatGPT become so essential to our workflows. Pretty much every problem I think of, I start with a chat interface, ask questions, and iterate on my thinking. But then I end up going to a structured view of data on web or mobile. Let’s say that’s some legacy software running for a bank, or some long form I have to fill out to help with ski resort booking details, or sometimes it’s a procurement flow. In those tools, I miss the fluidity of sharing my thoughts and having it extract out what I meant and get what it needed.
Today's Reality
───────────────────────────────────────────────────────────
Chat Interface Structured UI
┌─────────────────┐ ┌─────────────────┐
│ "Change my │ │ Name: [_______] │
│ budget to │ ??? │ Budget: [5000] │
│ $8000" │ ──────► │ Date: [01/15] │
│ │ │ │
│ Fluid, natural │ │ Precise, but │
│ but imprecise │ │ no AI help │
└─────────────────┘ └─────────────────┘
↓ ↓
Great for thinking Great for precision
Bad for structure Bad for bulk edits
As a solution to that, many are trying completely different interfaces, like replacing the UI with a chat UI, which I think is very limiting too. People do need control and the level of specificity for certain values to be updated. And a chat-only UI takes away a lot of control of how UI can be used by humans to do point edits.
#Introducing Duet
Duet allows humans to use UI the way we are used to, while allowing LLMs to read/write too. This is not novel, and a lot of others do it already. Duet just makes it easy to build such applications.
#Duet’s Mental Model
┌─────────────────────────────────────────────────────────────┐
│ SCHEMA │
│ (fields, types, constraints, defaults) │
└─────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ STATE │
│ (current snapshot → renders UI) │
└───────────────┬─────────────────────────────┬───────────────┘
│ │
Human edits LLM edits
(direct set) (JSON Patch)
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────┐
│ VALIDATION │
│ accept/reject each change with reason │
└─────────────────────────────────────────────────────────────┘
- Schema: what can change + constraints + defaults + labels
- State: current snapshot rendered by the UI
- Edits: JSON patches allowing delta updates
- Validation: accept/reject each patch with a concrete reason
Note: Duet is for client-side state where humans and AI are editing together in a live session—especially draft state that doesn’t exist in your database yet. If your architecture is “agent updates DB -> client syncs later,” you probably don’t need this. Duet is for interactive, client-side state where both editors see changes immediately.
#Why JSON Patch?
JSON Patch is better than a custom payload because it’s already a standard (RFC 6902), and LLMs are well-trained on it. This means fewer errors when LLMs build patches to update the UI.
#Duet for Swift
Core concepts:
- Schema and typed fields/constraints
- Document as observable source of truth (and optional persistence)
- LLMBridge for context generation + patch application
import SwiftUI
import Duet
// 1. Define your schema
let budgetSchema = Schema(
name: "Budget",
fields: [
.number("income", label: "Monthly Income", min: 0),
.number("rent", label: "Rent", min: 0),
.number("savings", label: "Target Savings %", min: 0, max: 100),
]
)
// 2. Create a document
@State private var doc = Document(
schema: budgetSchema,
storageKey: "myApp.budget",
storage: UserDefaultsStorage()
)
// 3. Use in SwiftUI
TextField("Income", text: doc.binding(for: "income"))
// 4. LLM edits via JSON Patch
let bridge = LLMBridge(document: doc)
bridge.getContext() // Schema + values for prompt
bridge.applyLLMResponse(jsonPatch) // Apply LLM's JSON Patch response
#Optional Freebie: DuetChat
Think of this as a reference implementation of using Duet in iOS. I built it for two apps, so it was pretty easy for me to extract it out as a reusable UI component.
// Drop-in chat UI that's already wired to Duet
DuetChatView(bridge: bridge)
.onPatchApplied { patch in
print("Applied: \(patch)")
}
#Testing with the CLI
The repo includes a CLI tool to test schemas and JSON Patch operations without running a full iOS app (shoutout to Rick for the help on PR#1!):
swift run DuetCLI # Interactive mode
swift run DuetCLI --budget # Use the budget schema
swift run DuetCLI --fitness # Use the fitness schema
You can view document state, see the LLM context, and apply patches interactively—great for debugging your schema before wiring up the UI.
#Duet for TypeScript
So this was fun, especially using Claude Code. After building everything for Swift, I wanted a TypeScript version too. I gave Claude Code some constraints and it did a phenomenally good job recreating it in a TypeScript-native way.
TypeScript’s ecosystem is a bit more mature than Swift when it comes to schema and state management. So rather than reinventing the wheel, I reused Zod and Zustand. Those are the constraints I had to provide to make sure it’s built in a TypeScript-friendly way.
Core concepts:
- Zod as schema contract
- Zustand as the host state store
- LLM bridge APIs: context generation, patch application, function schema for tool calling
import { z } from 'zod'
import { createDuet, field } from 'duet-kit'
// 1. Define schema with Zod validation
const useFormStore = createDuet('ContactForm', {
name: field(z.string().min(1), 'Full Name', ''),
email: field(z.string().email(), 'Email', ''),
company: field(z.string(), 'Company', ''),
})
// 2. React UI uses it like any Zustand store
const { data, set } = useFormStore()
// 3. LLM bridge is attached automatically
useFormStore.llm.getContext() // schema + current values for prompt
useFormStore.llm.getFunctionSchema() // OpenAI function calling format
useFormStore.llm.history() // audit trail of all patches
// 4. User says: "Set my name to John and email to [email protected]"
// LLM responds with a JSON Patch (RFC 6902):
const llmResponse = `[
{ "op": "replace", "path": "/name", "value": "John" },
{ "op": "replace", "path": "/email", "value": "[email protected]" }
]`
// 5. Apply the patch - Duet validates against your Zod schema
const result = useFormStore.llm.applyJSON(llmResponse)
// { success: true, applied: 2 }
// or if validation fails:
// { success: false, error: "Invalid email" }
Already using Zustand + Zod? Add LLM capabilities with one line using attachLLM(), no migration is needed here.
#Try the Examples
The repo includes two interactive demos you can run locally:
cd examples && npm install && npm run dev
Trip Planner: Human edits the form, LLM edits via JSON Patch—both update the same validated state.
CRM Voice Dictation: Speak a note, LLM extracts nested CRM fields automatically.
In the next post, I’ll explain how it’s been used in the apps I have built.
Try it out and let me know what you think:
- duet-kit for TypeScript
- duet-kit-swift for Swift