# NL AI Visibility Benchmark, nulmeting Q2 2026: open dataset

This is the complete dataset behind the report "Noemt AI jou? De Nederlandse AI-zichtbaarheidsbenchmark" / "Does AI mention you? The Dutch AI Visibility Benchmark" published on [identityfirstmedia.com](https://identityfirstmedia.com/blog/ai-visibility-benchmark).

All 1,125 measured answers are included, unedited. Nothing was sampled out.

- **Publisher:** Identity First Media (Paul Veth)
- **Measurement window:** 2026-06-10 through 2026-06-12
- **License:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). You may reuse, redistribute, and build on this data with attribution.
- **Integrity:** `manifest.json` lists the SHA-256 checksum and byte size of every file.

## What was measured

How often and how consistently five AI assistants (ChatGPT, Google Gemini, Perplexity, Claude, Grok) recommend Dutch domain experts by name, across three professions (architects, therapists, photographers), and which sources feed those recommendations.

Design: 3 segments x (10 recommendation prompts + 5 recognition prompts) x 5 platforms x 5 repeat runs = 1,125 answers. Each run used a clean context: no memory, no system instructions, no session carry-over. Repetition is the measurement; a single run is noise.

## Platforms and models

| Platform | Configured | Reported by the API | Notes |
|---|---|---|---|
| ChatGPT | gpt-5 | gpt-5-2025-08-07 | web search enabled |
| Google Gemini | gemini-flash-latest | gemini-3.5-flash | search grounding enabled |
| Perplexity | sonar | sonar | built-in search |
| Claude | sonnet | claude-sonnet-4-6 | web search enabled, via claude CLI |
| Grok | grok-4.3 | grok-4.3 | web/X live search enabled |

The exact model string reported by each API per answer is in the `model_reported` field of every record. Trust that field over this table: models shift under your hands. For 149 of the 225 Claude answers the CLI reports `claude-sonnet-4-6,claude-haiku-4-5-20251001`: the answer itself came from Sonnet, the Haiku entry is CLI-internal tooling. That is visible in the data rather than cleaned away.

## Files

| File | Contents |
|---|---|
| `answers.jsonl` | One record per answer (1,125). Full response text, prompt, platform, reported model version, timestamp, run index, cited URLs. The verbatim API envelope is stripped for readability. |
| `answers-full.jsonl.gz` | The same 1,125 records with the complete raw API envelope per answer, untouched. This is the primary evidence. |
| `extracted.jsonl` | Per answer: extracted persons, organizations, cited URLs with source type, refusal flag, and for recognition questions the human-reviewed `target_recognition` verdict. |
| `prompts.json` | All 45 prompts (15 per segment), with type (A/B/C), region layer, and exact Dutch wording. Fixed before the runs; none were changed mid-measurement. |
| `experts.json` | The expert samples per segment: selection sources (professional registers, trade associations, trade media), prominence tier, and the entity snapshot per expert. |
| `confidence-intervals.json` | Wilson 95% confidence intervals for the headline proportions, computed from `extracted.jsonl`. |
| `manifest.json` | Record counts, byte sizes, SHA-256 checksums. |

## Key record fields (`answers.jsonl`)

- `id`: unique answer id, joins to `extracted.jsonl` via `response_id`
- `ts`: ISO timestamp of the API call
- `segment`, `prompt_id`, `prompt_type` (A = category question, B = problem question, C = name verification), `region` (national / metro / regional)
- `prompt_text`: the exact Dutch prompt sent
- `platform`, `model_configured`, `model_reported`, `settings`
- `run_index`: 1 through 5 (which repeat run)
- `response_text`: the full answer as returned
- `citations`: URLs surfaced by the platform's search tooling, where available

## Definitions used in the report

- **Named answer:** an A/B answer that names at least one concrete person or organization (extracted `persons` + `organizations` non-empty) and is not a refusal.
- **Mention:** one appearance of a normalized person or organization name in one answer. Frequency across the 5 repeat runs of the same prompt is the core metric.
- **Recognized (type C):** the answer describes the sampled expert, verified by hand against firm, region, and profession. An answer describing a namesake or a confabulated person counts as not recognized, even when the model sounded confident. The original automated `recognition` field is retained next to the manual `target_recognition` verdict so you can audit the difference.

## Extraction and validation

Entities and sources were extracted per answer by an LLM extraction pass (Claude, claude-sonnet-4-6); the first 50 extractions per segment were validated by hand before the rest was processed. All 375 recognition answers were manually reviewed for namesake confusion (see `target_recognition`).

## Known limitations

1. API results are not identical to the consumer apps (memory, personalization, interface layer). Manual spot checks in the real apps were part of the study; both are reported in the article.
2. This measures visibility at one moment in time, not buying behavior or revenue.
3. Mention frequency is a proxy for entity strength, not a proven causal mechanism.
4. Five experts per profession at the recognition stage. Wide confidence intervals at n=25 are given in `confidence-intervals.json` rather than hidden.
5. Expert names in this dataset are real professionals, selected via public professional registers and trade media. All underlying facts are publicly available; the dataset adds measurement, not disclosure.

## Reproducing

The prompts, expert samples, and model configuration in this dataset are sufficient to re-run the measurement against the current model generation. Expect different absolute numbers (models change continuously); the structural patterns are the claim, not the point estimates.

## Citation

Identity First Media (2026). NL AI Visibility Benchmark, nulmeting Q2 2026. https://identityfirstmedia.com/blog/ai-visibility-benchmark