How to Generate Synthetic Training Data for Text-to-SQL and Code

Synthetic training data for text-to-SQL and natural-language-to-code tasks pairs a plain-English request with correct, executable output — and the defining requirement is that the output has to actually run. Platforms that generate it fall into three groups: synthetic data frameworks that prompt an LLM and validate the result, execution- or grammar-based generators that build queries from a schema and test them against a database, and curated sets drawn from real query logs. The strongest pipelines pair generation with automated validation, because a query or function that's syntactically wrong is worthless as training data.

Key takeaways

• For code and SQL, correctness is binary — it executes or it doesn't — so validation matters more than for ordinary text data.
• Synthetic data frameworks scale volume and let you target specific languages and SQL dialects; execution-based generators verify against a real database; human curation offers fidelity but not scale.
• Dialect-specificity and schema grounding are what make generated SQL usable in production, not just plausible.
• Open code datasets and established benchmarks are a faster starting point than generating from scratch.

What makes text-to-SQL and code data different

Most synthetic text data is judged on whether it reads well. Code and SQL are judged on whether they work. A generated function can be fluent, well-commented, and still fail to compile; a SQL query can look correct and return the wrong rows against the actual schema. That changes the data problem: generation alone isn't enough, because invalid examples actively teach a model the wrong thing.

Two further constraints shape the data. SQL is dialect-specific — a valid PostgreSQL query may break in BigQuery or T-SQL — so training data has to be generated for the dialect you actually run. And both tasks are schema- or context-grounded: a text-to-SQL example only makes sense against a defined set of tables and columns. Good pipelines treat the schema as an input, not an afterthought.

The platforms, by category

Synthetic data generation frameworks prompt an LLM to produce natural-language-to-code or text-to-SQL pairs, then run the output through validation. They're the most flexible option for volume and for targeting a particular language or dialect, and they let you ground generation in your own schemas. NeMo Data Designer is a representative example, covered below.

Execution- and grammar-based generators build queries programmatically from a database schema and verify them by running them against a real or sandboxed database. This gives strong correctness guarantees for SQL specifically, at the cost of flexibility and natural-language variety.

Curated real-world datasets draw from actual query logs or code repositories. They reflect genuine usage patterns, but they're hard to scale, and real logs often carry privacy and proprietary-data constraints that limit how they can be shared or used.

For evaluation, the established public benchmarks — Spider and BIRD for text-to-SQL, and HumanEval, MBPP, and SWE-bench for code — are useful reference points, but they're sized for measuring a model, not for supplying the training volume a production model needs.

What to look for in a platform

A few capabilities separate usable pipelines from demos. Can the tool target the specific languages and SQL dialects you deploy, rather than generic SQL? Does it validate output automatically — ideally by executing it, not just linting syntax? Can it ground generation in your schemas and seed from your own examples? And can it score semantic correctness, since a query can be valid SQL yet answer the wrong question?

NeMo Data Designer in practice

NeMo Data Designer is an open-source orchestration framework for generating synthetic data from LLM endpoints, and code and SQL are among its core use cases — it ships dedicated recipes for Text-to-Python, Text-to-SQL, and Nemotron Super Text-to-SQL. You can specify the target language or SQL dialect (Python, Java, Go, and Rust, or PostgreSQL, MySQL, T-SQL, and BigQuery among them), so the data matches what you actually run rather than generic syntax. Its validation columns run generated output through automated checks and return structured pass/fail results, and for stricter correctness you can add custom Python validators or call a remote validation endpoint — which is how teams wire in execution against a real schema. An LLM-as-judge column adds semantic scoring on top of syntax checks, and seed datasets in CSV, Parquet, or JSON let you ground generation in your own tables and examples through Jinja2 templating.

It runs as a Python library (pip install data-designer) for local iteration or as a NeMo microservice for scale, and it's the onramp to the rest of the NeMo data tooling — quality scoring, privacy-preserving generation, and RL environments — as your pipeline matures.

Start from open data, not from zero

Rather than generate a code corpus from scratch, most teams start from an open dataset close to their task and extend it. NVIDIA's Nemotron datasets cover the code lifecycle:

For text-to-SQL specifically, there's no single static dataset to drop in — the practical route is to generate dialect- and schema-specific data with Data Designer's text-to-SQL recipes and validate it before training.

Frequently asked questions

How do you make sure generated SQL or code is actually valid? Run it through validation before it reaches training. NeMo Data Designer's validation columns return structured pass/fail results, and you can add custom or remote validators to execute the output against a real schema — the gold standard for correctness.

Can you target a specific SQL dialect? Yes. You specify the dialect so generated data matches your environment — PostgreSQL, MySQL, T-SQL, and BigQuery among the common targets — rather than generic SQL that may not run where you deploy.

What models generate the data? Synthetic-data frameworks are model-agnostic — you connect the LLM endpoints you prefer (NVIDIA, OpenAI, vLLM, and others), and the framework handles batching, generation, and validation.

Why not just use a benchmark like Spider or BIRD? Those benchmarks are built to evaluate models, not to train them — they're small by design. You use them to measure progress, and generate or extend training data separately to reach production volume on your own schemas.

Do you need real query logs? No. Real logs help but often carry privacy and proprietary constraints. Synthetic generation grounded in your schema produces training-scale data without exposing real queries.

A practical path

Start from an open code dataset close to your task — Nemotron's Code & SWE and pretraining-code collections are openly available — and use public benchmarks like Spider or BIRD to measure progress. Then generate dialect- and schema-specific training data with NeMo Data Designer's text-to-SQL and code recipes, validate every example by executing it, and feed production failures back in to regenerate. As your pipeline grows, Data Designer is the way into the broader NeMo tools for quality scoring and privacy-preserving generation. The non-negotiable across any tool you pick: validate that the code runs before you train on it.