TransformingWeb-Workflows with Next-Gen Agents

We ensure agents make smart micro-decisions, navigate complex UIs with ease, and consistently deliver accurate results-backed by a reliable second layer of quality control.

Use Case

1. Prompt and Trajectory Creation

The goal is to collect human-generated plans and browser actions for everyday tasks-like ordering food or bookingflights-using a secure browser extension. This extension captures multi-tab activity, annotates data safely, and storeseverything in the client’s data warehouse.

2.Agent Plan and Execution Evaluation

2.A. Evaluating the plan generated by an AI browser agent

The objective is to evaluate the agent’s plan for each collected task, with every step annotated and checked for mistakes.

2.B. Evaluating the execution of an AI browser agent

Objective is to assess execution of the plan generated by the agent. This is being done by the human experts.

The Bottleneck: Data Quality in Intelligent Agent Training

Training your intelligent agents to navigate the digital world effectively hinges on the quality of their learning data. Imagine trying to teach someone to drive with blurry maps and incomplete instructions – the outcome wouldn't be pretty. Similarly, flawed or incomplete browser trajectory data leads to:
Inefficient Agent Planning
Poor User Experience
Missed Business Opportunities

The Deccan AI Difference

  1. Secure, Client-Side Data Collection and Annotation
    We developed a browser extension that captures trajectories—including screenshots and user actions—across multiple tabs,with all annotation performed client-side. This ensures data never leaves the client’s environment unless explicitly permitted.Data is stored directly in the client’s data warehouse, maintaining full control and compliance with internal policies.
  2. End-to-End Workflow Coverage
    Our tools enable the collection of both human and agent-generated plans, supporting a wide range of use cases (e.g., e-commerce, travel, support). Annotators can label each step, flag errors, and add context, ensuring rich, actionable datasets.
  3. Advanced Plan and Navigation Evaluation
    We implemented a dual-layer evaluation framework:
Plan Evaluation:
Each step in the agent’s proposed plan is annotated and reviewed for accuracy and logic.
Navigation Evaluation
Human experts assess the execution of the plan in real browser environments, identifying deviations, inefficiencies, or errors.
  • Scalability and Flexibility
    Our architecture supports high-throughput data collection and annotation, with flexible integration into existing clientworkflows and data infrastructure.

Our Gold Standard: End-to-End Quality Control (QC)

We don't rely on assumptions. Our multi-stage QC process ensures the highest level of annotation accuracy and consistency:

Prompt and Trajectory Annotation Workflow

Identify Trending Domains
We continuously monitor and select trending domains (e.g., electronics, travel, food delivery) to ensure our prompts are relevant and valuable.
Prompt Creation by Complexity
High-quality prompts are crafted and divided into two segments:
Low Complexity: Less than 15 steps
High Complexity: More than 15 steps
Prompt Verification
Every prompt undergoes a QC review and sample check by the Project Manager to ensure clarity and coverage.
Trajectory Recording
Annotators execute the prompt, and the entire annotation process is recorded (screen capture + video) to capture every action and decision.
Automated Validation
A validation script checks that all required steps are present and that annotators have provided necessary inputs at each stage.
QC Review with Video Reference
The trajectory annotation is reviewed by QC personnel, who use the recorded video to verify accuracy, completeness, and adherence to guidelines.

ModelEvaluationWork ow

Plan Step Verification
We verify each planner step generated by the model, annotating any mistakes in the agent’s plan.
Error Categorization
Errors are annotated and classified into:
Task Success: Yes / No
ModelPerformance: Perfect/Imperfect/Incorrect
Navigation Step Verification
We review the navigation steps, annotating any mistakes such as incorrect element selection or failure to access an element.
Error Definition
Annotations specify whether the model chose the wrong element, missed a required action, or encountered an accessibility issue.
Final Justification
A comprehensive justification is provided for each evaluation, summarizing task outcome and model performance.

Pristine Data. At Scale.
With Speed.

Contact Us