llm-evaluation

At LightSight, we've been using DeepEval to evaluate our internal tooling. I wanted to share our thoughts and practices with the tool, and with LLM evaluation in general.

TLDR - Synthetic data at scale is an effective way to pressure-test your LLM-powered applications, but don't make the mistake of relying on it exclusively.

Example DeepEval Workflow

Let's walk through the simplest possible example of using DeepEval - analyzing a "Review Analyzer Assistant" tool, which summarizes Amazon product reviews for a walking treadmill.

Goldens

The very first thing we need to establish is a set of expected answers for different questions. In DeepEval, these are called goldens. Each golden has an input, an expected output, and context (in this case, the actual reviews).

input: "What were the top 3 complaints?"
expected_output: "1) Poor setup instructions 2) App required 3) Noisy at high speeds"
context:
  - "Review 1: Love the treadmill but setup was a nightmare..."
  - "Review 2: Why do I need to download an app just to start walking?..."
  - "Review 3: Great product, but it gets loud above speed 4..."
  - "... (50 reviews total)"

The context here should be the ground-truth data, which the system under test should be referencing while producing its response.

Synthesizer

Handwriting all of these is tedious, and often unnecessary (we'll discuss this more later), so we can use DeepEval's synthesizer to automate golden generation from source documents (the source documents are often the same content that goes into the context of a golden).

Test Cases

A test case is a golden + the system's actual response. With our goldens in hand, we run each against our "Review Analyzer Assistant" and fill in the actual_output to retrieve the final test cases.

Golden vs. Test Case

Golden (Input)

input: "What were the top 3 complaints?"
expected_output: "1) Poor setup instructions 2) App required 3) Noisy at high speeds"
context:
  - "... (50 reviews)"

Test Case (Input + Output)

input: "What were the top 3 complaints?"
expected_output: "1) Poor setup instructions 2) App required 3) Noisy at high speeds"
context:
  - "... (50 reviews)"
actual_output: "The 3 most frequently cited complaints were 1) App download required 2) Poor first time setup instructions 3) High volume at higher speeds"

Metrics & Evaluations

Metrics are the criteria upon which to evaluate your system. DeepEval offers customizable and out-of-the-box metrics: G-Eval, Faithfulness, Answer Relevancy, Hallucination detection, Toxicity and bias checks.

An evaluation is the result of running a test case against your chosen metrics, producing a score, reason, and success/failure result. For simplicity, we'll evaluate against a single G-Eval metric for correctness.

input: "What were the top 3 complaints?"
expected_output: "1) Poor setup instructions 2) App required 3) Noisy at high speeds"
actual_output: "The 3 most frequently cited complaints were 1) App download required 2) Poor first time setup instructions 3) High volume at higher speeds"
metrics:
  - Correctness:
      score: 0.9
      success: true
      reason: "The actual output identifies the same 3 complaints as the expected output. Minor ordering difference (app requirement listed first instead of second) does not affect correctness."

There are a variety of other important concepts in DeepEval (like datasets, single-turn vs multi-turn, prompts), but hopefully this gives you an idea of a typical flow.

Can AI Test AI?

DeepEval's synthesizer means you can have an LLM generate test cases, another LLM produce responses, and a third LLM judge those responses. Fully autonomous, without any human in the loop.

I'm reminded of Blade Runner - a bunch of machines pretending to be human and passing judgment on each other.

It's natural to doubt the efficacy of this - hallucination is one of the largest hurdles in this space, and when a single LLM answers a question, it's simultaneously trying to understand the query, retrieve knowledge, reason about it, and generate fluent text.

In an evaluation pipeline, however, each component has a narrow role:

The judge doesn't need world knowledge - it just compares text against provided context. That is a simpler task that LLMs are empirically better at.

There's also an adversarial dynamic:

None of these systems are obligated to make the others look good.

Why Humans Are Still in the Loop

There are a couple of reasons humans still need to be in the loop.

Content Mismatch

The content you give to your synthesizer might not match what users are actually asking. You may have a larger disconnect between what you're teaching your LLM to answer vs. what users actually want answered.

Low Difficulty

LLM synthesizers struggle to produce conceptually difficult questions - the generated questions tend to be surface-level & trivia-like. Have some human-produced questions that require deep thought. See: What Has Been Lost with Synthetic Evaluation.

No Negative Examples

You should include questions the LLM is not capable of answering, and validate it correctly responds with "I don't know." DeepEval's synthesizer did not generate any of these.

Style & Tone

LLMs often generate overlong responses that are hard to read. The synthesizer doesn't construct tests for this. You should.

How We Handle It

I prefer the following routine:

Golden Generation

85% Synthetic, 15% Manual. The synthesizer handles broad coverage. Humans create goldens for edge cases, known failure modes (see above), and observed regressions. Humans should review synthetic cases after generation.

→

Routine Evaluation

Fully Automated. To catch regressions early, run evaluations on every merge to main with a cheap, fast model (Gemini Flash, GPT-4o-mini).

→

Deep Evaluation

Automated, Human-Triggered. During feature development, run thorough evaluations with stronger models. Your team decides when to run. This stage should also add new goldens following your desired split.

→

Acceptance Testing

Human Review of Samples. Before release, have your team review a random sample of outputs, grade them, and compare notes to the automated grade. This serves as both a final review and a calibration test of the evaluation system. Any learnings or regressions should be added as new goldens.

Note - our preferred split is 85% synthetic and 15% manually created goldens, but this is arbitrary. Use whatever split works best for your team, but in general more sensitive contexts = higher % of manual goldens.

Takeaways

If you're not already evaluating your LLM systems, you should. When you do, don't be afraid to use synthetic data - it's a great way to test your LLM-powered applications.

Just keep in mind, humans are still essential for:

Catching problems with the synthetic questions
Turning production failures into test cases
Nuanced, deep questions
Tone/Style

DeepEval

Beau Dekker

Ready to Transform Your Business with AI?