TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
ServiceNow CoreAI describes AutoSynthData, a pipeline designed to turn an enterprise agent’s capability gaps into new training tasks. The report says the system uses a stronger teacher to help shape tasks, checks them in the target environment and updates its curriculum as the model improves; the published example uses EnterpriseOps Gym.
ServiceNow CoreAI has described AutoSynthData, a pipeline intended to turn enterprise agents’ observed weaknesses into new training tasks that can be run and checked inside their operating environment. In a report published on Hugging Face, the team presents an example using EnterpriseOps Gym and says the approach draws on a target model’s failures and a stronger teacher’s successful runs to guide task generation.
AutoSynthData is built around the idea that general model capability does not guarantee reliable performance in a specific company environment. Enterprise agents must work with particular tools, policies and data states, and may fail on a workflow or constraint they handle poorly. The report says training requires more than collecting individual failures: developers need varied tasks that test the same underlying capability, remain possible in the environment and have a dependable way to judge success.
The report represents a task as three parts: a system specification describing instructions and environment constraints, a user prompt describing the requested outcome, and a verifier that checks the result. Generated tasks are meant to be feasible, realistic and challenging for the current target model. The verifier, in turn, should reject incorrect or policy-violating outcomes without requiring one specific path when other valid solutions exist.
For its EnterpriseOps Gym demonstration, ServiceNow says it evaluates both the target model and a stronger teacher on diagnostic tasks. The team reviews what capability is being tested, where the target fails, how the teacher succeeds, and what properties a successful final state must have. Those findings are distilled into sanitized capability specification cards. The generator uses the cards—not the original evaluation prompts, entities, trajectories or verifier details—to create different tasks, which are then checked in the environment. Accepted samples can be used for post-training, after which evaluation can identify remaining gaps.
Training Agents for Company Workflows
The proposal addresses a practical limitation in deploying agents: a model can perform well on broad evaluations yet fail when asked to carry out work inside a company’s own systems. By targeting observed capability gaps, AutoSynthData aims to make training more relevant to the workflows and constraints an agent will face, rather than relying only on generic examples.
The emphasis on execution and verification is also consequential. Synthetic tasks that are impossible, unrealistic or incorrectly scored can teach an agent the wrong behavior. ServiceNow’s framework treats task feasibility and verifier quality as part of data generation, not as after-the-fact checks. If the process works as described, it could help teams produce repeatable training examples while reducing dependence on manually authored tasks. The report, however, describes a system and an illustrative experiment; it does not by itself establish broad gains in production performance.
enterprise AI training data generation tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
From Evaluation to New Tasks
AutoSynthData follows a feedback loop: evaluate a model in its intended environment, identify recurring weaknesses, generate tasks that exercise those capabilities, validate the tasks, and use accepted examples for post-training. The curriculum is intended to change as the model improves, shifting toward weaknesses that remain rather than continuing to generate tasks it already solves reliably.
The report says the EnterpriseOps Gym example uses a released dataset and cites Malay et al. (2026). It also draws a distinction between executable behaviors and realistic user requests: an environment may permit many actions, but useful training prompts should reflect plausible work. The supplied report excerpt ends while describing how generated tasks vary entities, so it does not provide the full account of that generation process or the experiment’s quantitative results.
“At ServiceNow CoreAI, we built AutoSynthData to turn those capability gaps into training data.”
— ServiceNow CoreAI, in the Hugging Face report
As an affiliate, we earn on qualifying purchases.
Evidence Beyond the Demonstration
The supplied report material does not state how much the target model improved, how results compared with other data-generation methods, or whether gains persisted outside EnterpriseOps Gym. It also does not give the scale of the generated dataset, the cost or time required to produce and validate tasks, or details about deployment in customer environments.
The excerpt describes intended properties and a research demonstration, but does not establish that every generated task meets those standards or that the pipeline prevents all incorrect training signals. The report’s publication date is also not included in the material provided. These limits make it difficult to assess the system’s performance or practical adoption from this account alone.
AI verifier tools for enterprise systems
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Further Results to Watch
The next useful evidence would be fuller results from the EnterpriseOps Gym experiment, including task counts, evaluation methods and before-and-after performance for the target model. Comparisons against manually written tasks or other synthetic-data approaches would help clarify whether AutoSynthData adds measurable value.
Further reporting or releases could also explain how the system handles verifier errors, adapts its curriculum across training rounds and performs on environments beyond the example described. The report says evaluation of the updated model can guide another generation cycle; whether that loop yields reliable gains in enterprise deployments remains to be shown in the supplied material.
synthetic data generation for machine learning
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is AutoSynthData?
AutoSynthData is a system described by ServiceNow CoreAI for generating and validating training tasks aimed at weaknesses found in enterprise agents.
How does it choose what tasks to generate?
It evaluates a target model, examines tasks it struggles with alongside a stronger teacher’s successful runs, and distills the findings into capability specification cards. The generator uses those cards to create new tasks with different prompts and environment details.
How does the system check generated tasks?
Each task includes a verifier intended to determine whether the agent reached the requested outcome while respecting relevant constraints. The report says tasks are checked in the environment before accepted samples are used for post-training.
Has AutoSynthData been shown to improve enterprise agents?
The supplied report describes the method and an EnterpriseOps Gym demonstration, but the provided material does not include quantitative improvement results or evidence of performance in customer deployments.
Source: rss
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
