# Verbata

> Codes open-ended survey answers into a codebook

- **Client**: Deka Insight
- **Date**: 2025
- **Tools**: [Next.js](https://nextjs.org), [Vercel AI SDK](https://wastu.net/projects/verbata/tools/vercel-ai-sdk), [OpenRouter](https://wastu.net/projects/verbata/tools/openrouter), Postgres, [Langfuse](https://wastu.net/projects/verbata/tools/langfuse)
- **Metrics**:
  - Answers coded: 380k+ (Open-ended replies processed in production)
  - Live projects: 155 (Commercial research studies completed through September 2026)

## Details

In survey research, people often write open answers in their own words. Researchers call these answers "verbatims". To turn thousands of text answers into charts and numbers, analysts have to read each line and assign it a category label from a codebook.

At [Deka Insight](https://dekainsight.com), researchers and outside helpers spent 1 to 3 full days per project tagging rows in Excel. I sat with their team to see how they work, then built Verbata to do the coding automatically.

# Why embeddings did not work

At first, I tried using AI embeddings to group similar answers together. In theory, grouping similar text sounds simple.

In real Indonesian surveys, it failed. People type with heavy slang, typos, short words, and local languages like Javanese or Sundanese. The embedding model grouped words that just looked similar, instead of grouping answers with the same real meaning. The results were messy, and researchers still had to clean everything by hand.

So I dropped embeddings completely. Instead, Verbata sends the survey answers and the researcher's codebook straight to the language model. The model reads the actual context, picks the right code, and does not need any complex database index to maintain.

# Rebuilding for high-volume batch work

The first version (v1) was a small tool where researchers pasted text to get code suggestions. They used it on real projects for about two months. It proved the AI could code accurately, but the app was too slow for big files with thousands of rows.

To handle real client work, researchers needed a faster, batch-friendly workflow:

1. Upload Excel files directly without reformatting them.
2. Build or reuse a codebook hierarchy (**Net** → **Subnet** → **Code**, like _Taste_ → _Sweetness_ → _Too Sweet_).
3. Run batch jobs that code thousands of answers in a few minutes.
4. Review results in a clean table and quickly fix any edge cases.
5. Export the finished Excel file ready for reporting.

I rebuilt Verbata around this loop, adding nested codebooks, faster background batch processing, and review tools so researchers can finish full projects without hitting limits.

# How we divide the work

I handle the product design, code, and server infrastructure. Deka Insight's research team controls the codebooks, prompts, and final quality checks before sending reports to clients.

As of September 2026, Verbata has coded over 380,000 open-ended answers across 155 commercial projects.

#### Why I used Vercel AI SDK in Verbata

I wanted to choose different models for different research tasks. [Vercel AI SDK](https://ai-sdk.dev) lets me mix and match them through the same SDK, so changing a model doesn't mean changing that foundation.

#### Why I used OpenRouter in Verbata

I like having a choice of models when the balance of quality, speed, and cost changes between research tasks. [OpenRouter](https://openrouter.ai) gives me access to those choices through one service.

#### Why I used Langfuse in Verbata

When the AI puts an answer in the wrong category, I want to understand why. I chose [Langfuse](https://langfuse.com) so I have somewhere to look into those results and review changes to a prompt or model.