# Verbata

> Codes open-ended survey answers into a codebook

- **Client**: Deka Insight
- **Date**: 2024-2026
- **Metrics**:
  - Answers coded: 380k+ (Open-ended replies processed in production)
  - Live projects: 155 (Commercial research studies completed since mid-2025)

## Details

In survey research, people often write open answers in their own words. Researchers call these answers "verbatims". To turn thousands of text answers into charts and numbers, analysts have to read each line and assign it a category label from a codebook.

At [Deka Insight](https://dekainsight.com), researchers and outside helpers spent 1 to 3 full days per project tagging rows in Excel. I sat with their team to see how they work, then built Verbata to do the coding automatically.

## Why embeddings did not work

At first, I tried using AI embeddings to group similar answers together. In theory, grouping similar text sounds simple.

In real Indonesian surveys, it failed. People type with heavy slang, typos, short words, and local languages like Javanese or Sundanese. The embedding model grouped words that just looked similar, instead of grouping answers with the same real meaning. The results were messy, and researchers still had to clean everything by hand.

So I dropped embeddings completely. Instead, Verbata sends the survey answers and the researcher's codebook straight to the language model. The model reads the actual context, picks the right code, and does not need any complex database index to maintain.

## Rebuilding for high-volume batch work

The first version (v1) was a small tool where researchers pasted text to get code suggestions. They used it on real projects for about two months. It proved the AI could code accurately, but the app was too slow for big files with thousands of rows.

To handle real client work, researchers needed a faster, batch-friendly workflow:

1. Upload Excel files directly without reformatting them.
2. Build or reuse a codebook hierarchy (**Net** → **Subnet** → **Code**, like *Taste* → *Sweetness* → *Too Sweet*).
3. Run batch jobs that code thousands of answers in a few minutes.
4. Review results in a clean table and quickly fix any edge cases.
5. Export the finished Excel file ready for reporting.

I rebuilt Verbata around this loop, adding nested codebooks, faster background batch processing, and review tools so researchers can finish full projects without hitting limits.

## How we divide the work

I handle the product design, code, and server infrastructure. Deka Insight's research team controls the codebooks, prompts, and final quality checks before sending reports to clients.

Since mid-2025, Verbata has coded over 380,000 open-ended answers across 155 commercial projects.