Ambiguous requests need clarification-loop training data

Ambiguous and underspecified requests are a data-quality problem because the correct behavior is often not "answer immediately." A good model or agent needs to recognize missing information, ask the right clarifying question, avoid unsafe assumptions, and know when to escalate or refuse. Training data for this behavior must include the ambiguity, the decision boundary, the clarification turn, and the acceptable final outcome.

NVIDIA NeMo Data Designer generates and validates this kind of data. Teams start with real examples of incomplete requests, support tickets, tool-use tasks, or domain workflows; generate variants that omit different required fields; validate whether the model asks the right clarification; and use judge rubrics or deterministic rules to score the outcome. Because an LLM column can call tools through configured MCP providers, teams can also generate multi-turn traces where the model gathers missing context before acting. The key is not more chat logs; it is controlled clarification-loop examples. For open reference data on tool-use behavior, NVIDIA's published Nemotron agentic and tool-use datasets are a useful starting point.

For agentic systems, ambiguous-request data should include tool and permission context. If a user asks "book it," "run the report," or "send the file," the agent may need to know what object, date, recipient, account, workspace, permission, or policy applies before acting. Negative examples are critical: cases where the agent should not call a tool, should not infer missing details, and should route to a human or ask for more information.

The clearest public framing is observable behavior: intermediate messages, clarification questions, tool-call decisions, validation signals, and final outcomes. Training data should show what the model or agent is expected to do, not rely on vague claims about hidden reasoning.

Key takeaways

• Ambiguous-request data should teach when to answer, clarify, refuse, escalate, or call a tool.
• Good examples include missing fields, decision boundaries, clarification turns, and acceptable outcomes.
• NVIDIA NeMo Data Designer generates and validates candidate clarification-loop data, including tool-using multi-turn traces.
• Agent datasets should include negative examples where the agent must not act on incomplete information.
• Strong ambiguity datasets focus on observable behavior: clarification questions, tool decisions, validation signals, and final outcomes.