Generating Test Data with Claude: Realistic Datasets, Systematically

1. Why good test data determines how meaningful tests are

A technically correct test can still be worthless if the underlying test data does not reflect reality. A test that checks a customer with exactly one order and one address says little about whether the same logic also works for a customer with fifty orders, several shipping addresses, and changing payment methods. The quality of test data generation therefore directly determines how much trust one can place in a green test result.

In practice, test data is often created ad hoc: a developer copies an existing record, changes an ID, and calls it a test case. The result is records that happen to work but reflect neither the range of real data nor its edge cases. Claude can systematize this process by deriving targeted test datasets with a defined range from a data structure or domain model, instead of relying on random copies.

The key distinction here is between two goals: test data that realistically represents the normal case to verify business logic, and test data that deliberately covers edge cases and invalid states to verify robustness. Generating test data with Claude covers both goals, as long as the model is told explicitly which of the two is currently the focus.

2. Realistic vs. synthetic test data

Realistic test data follows the actual distribution and structure of production data: names, addresses, and order patterns that look plausible without touching real people. Purely synthetic test data such as "Test User 1" or "aaa@example.com" serves its purpose for simple functional tests, but fails as soon as logic depends on the shape of the data, for example sorting last names with umlauts or validating international phone numbers.

Claude is especially well suited for the middle category: plausible but guaranteed fictional data that reflects real patterns. A prompt can specifically ask for German last names with umlauts, addresses across several European countries with correctly formatted postal codes, or realistic product names for a specific industry. This data quality often surpasses simple random generators, because Claude actually understands the linguistic and structural patterns instead of combining characters at random.

3. Claude prompts for structured test datasets

The most important principle in generating test data with Claude is to specify the desired output format exactly before asking for content. A prompt such as "generate ten customer records as JSON with the fields id, name, email, country, and registration_date, where registration_date falls between 2023 and 2026" delivers directly machine-readable, consistent data, while an open request for "a few test customers" produces unstructured prose that needs post-processing first.

A second important building block is explicitly naming distributions. Instead of getting ten identical-looking records, Claude can be asked to build in a realistic spread, for example eighty percent customers from Germany, Austria, and Switzerland, the rest from other European countries, or a mix of new and returning customers with different order histories. This spread is essential for writing tests that represent real use cases instead of a single ideal profile.

# Claude Code CLI: generate structured, distributed test data
claude -p "Generate 20 customer records as JSON with fields:
id, name, email, country, registration_date, order_count.
Distribution: 70% Germany/Austria/Switzerland, 30% other EU.
order_count should range from 0 to 45, weighted towards low values.
Output valid JSON only, no explanation text."

4. Generating edge case data and boundary values

Besides realistic normal data, every resilient test suite needs deliberately faulty or boundary datasets: a fifty-character name, an email address with an unusual but valid structure, an order date in the future, a negative stock level caused by a faulty synchronization. Claude can specifically be asked to generate, alongside the regular dataset, a second set of edge case records that intentionally sit at the edges of the valid range.

A proven pattern is asking explicitly for the boundary of every field in the data model: what is the maximum field length, what are the smallest and largest allowed numeric values, which characters are permitted but rarely tested, such as emojis in a name field or an apostrophe in an address. This edge case test data lines up with classic boundary value analysis, but focuses specifically on data generation instead of test logic itself.

{
  "edge_case_customers": [
    {
      "name": "O'Brien-Müller",
      "email": "test+tag@sub.example.co.uk",
      "note": "apostrophe, hyphen, and umlaut in the same name"
    },
    {
      "name": "字符测试用户",
      "email": "unicode-test@example.com",
      "note": "non-Latin characters in a required name field"
    },
    {
      "name": "A",
      "email": "a@b.co",
      "note": "minimum allowed length for name and email"
    },
    {
      "name": "Very Long Customer Name That Approaches The Field Limit Of Fifty",
      "email": "long-name-boundary@example.com",
      "note": "name at exactly the 50 character database column limit"
    }
  ]
}

5. Test data for database fixtures in a Magento context

In Magento projects, test data is often organized as PHP fixtures that establish a specific database state before an integration test runs. Claude can generate a matching fixture class in the expected Magento format directly from a description of the desired state, for example "a customer with three orders in different statuses and two saved addresses", including correct use of the DataFixture attributes and repository calls.

This approach is especially valuable for complex object graphs with several linked entities, for example product, category, price rule, and customer group together. Manually building such nested fixtures is error-prone and time-consuming, while Claude can propose the creation order, the correct foreign key references, and the necessary cleanup steps after the test all at once.

<?php
declare(strict_types=1);

namespace Mironsoft\Sales\Test\Integration;

use Magento\Customer\Test\Fixture\Customer as CustomerFixture;
use Magento\Sales\Test\Fixture\Order as OrderFixture;
use Magento\TestFramework\Fixture\DataFixture;
use PHPUnit\Framework\TestCase;

/**
 * Fixture setup generated with Claude from a plain-language description:
 * "one customer with three orders in different statuses".
 */
final class CustomerOrderHistoryTest extends TestCase
{
    #[DataFixture(CustomerFixture::class, as: 'customer')]
    #[DataFixture(OrderFixture::class, ['customer_id' => '$customer.id$', 'status' => 'pending'], 'order1')]
    #[DataFixture(OrderFixture::class, ['customer_id' => '$customer.id$', 'status' => 'processing'], 'order2')]
    #[DataFixture(OrderFixture::class, ['customer_id' => '$customer.id$', 'status' => 'complete'], 'order3')]
    public function testOrderHistoryShowsAllThreeStatuses(): void
    {
        // Test body uses the fixtures created above via the DI container
        $this->assertTrue(true);
    }
}

6. Anonymization and GDPR-compliant test data

A particularly sensitive point in test data generation is how production-like data is handled. A common but risky approach is copying a production database dump for test purposes because it is "realistic". This usually violates GDPR as soon as test environments are less strictly secured than production, and it creates an unnecessary privacy risk on top. Claude can instead help generate equally realistic replacement data from an anonymized structure that reflects real distributions without referencing real people.

A practical approach is to describe the structure and statistical distribution of a production dataset to Claude without disclosing the actual values, for example "twenty percent of customers have more than three orders, the average order value is 65 euros", and have it generate a structurally equivalent but completely fictional dataset from that. This approach fulfills the purpose of realistic tests without any real personal reference ever entering the test environment.

# Describe production distribution to Claude without sharing real values,
# then let it generate a structurally equivalent, fully fictional dataset

production_summary = {
    "total_customers": 12000,
    "percent_with_more_than_3_orders": 20,
    "average_order_value_eur": 65,
    "top_countries": ["DE", "AT", "CH", "NL"],
}

# Prompt sent to Claude (no real customer data included):
prompt = f"""
Generate 100 fully fictional customer + order records that match
this statistical summary: {production_summary}
No real names, no real emails, no data derived from actual persons.
Output as a Python list of dicts.
"""

7. Generating bulk data and performance test data

Performance and load tests often need tens or hundreds of thousands of records that can neither be created by hand individually nor realistically be checked in detail one by one. Claude is less suited here for directly generating every single record, but very well suited for designing the generation script: which fields should vary randomly, which should follow a fixed distribution, how are referential relationships across millions of rows resolved efficiently, without the script itself becoming a performance problem.

Claude can, for example, propose a PHP or SQL script that works with batch inserts, temporarily disables indexes during the bulk load and rebuilds them afterward, and resolves references via pre-generated ID ranges instead of individual lookups. These design decisions are the difference between minutes and hours of runtime for preparing the test data itself when millions of rows are involved.

# Claude-assisted approach for generating 500k order rows efficiently
# 1. Disable secondary indexes before bulk insert
mysql -e "ALTER TABLE sales_order DISABLE KEYS;"

# 2. Batch insert in chunks of 5000 rows via a generated PHP script
#    (Claude designs the chunking and ID range pre-allocation logic)
php bin/generate-test-orders.php --count=500000 --batch-size=5000

# 3. Re-enable indexes and rebuild statistics after the load completes
mysql -e "ALTER TABLE sales_order ENABLE KEYS; ANALYZE TABLE sales_order;"

8. Ensuring consistency and referential integrity

Generated test data is only useful if it stays internally consistent: an order must reference a customer that actually exists, a coupon code must fall within its valid time range, a product variant must belong to an existing product family. Claude can be explicitly reminded of this referential integrity when designing a generation script, and then proposes creating IDs for all parent entities first before dependent records reference them, instead of inventing referenced IDs that later point to nothing.

A common mistake with manually created test data is silently violating business rules, for example an order date that falls before the customer's registration date. Claude can derive a list of such implicit consistency rules from the domain model and build them into the generation script as a check step, so invalid combinations never arise in the first place instead of only surfacing during the test.

9. Limits and comparison: manual, Faker, Claude

Claude does not replace a specialized test data library such as Faker for pure bulk generation of simple, unstructured values, for example ten thousand random names with no special requirements. For that case, a dedicated library is faster and less resource-intensive. Claude's strength lies where structure, domain knowledge, and targeted edge cases matter, not in raw text volume.

Nor does Claude replace the business review of whether generated test data actually represents the relevant business scenarios. A developer who knows the domain still has to decide which distributions and edge cases genuinely matter for the concrete feature. The table above shows that combining Faker for volume with Claude for structure and edge cases is often the most efficient solution in practice.

10. Summary

Generating test data with Claude improves how meaningful tests are by systematically producing realistic distributions, targeted edge cases, and referentially consistent datasets, instead of relying on chance. Precisely worded prompts with an explicit output format and desired distribution deliver directly usable data for PHPUnit fixtures, database setups, and performance tests. For sensitive production data, Claude helps generate structurally equivalent but entirely fictional replacement data, avoiding GDPR risks.

The biggest gains come from deliberately combining tools: Faker or similar libraries for pure volume, Claude for structure, domain knowledge, and edge cases. Anyone who deliberately uses this combination and anchors consistency rules in the generation script from the start builds test data that actually makes tests meaningful, instead of just making them appear green.

11. FAQ: Generating Test Data with Claude