OFFENSAI's Autonomous AgentO: All 122 Datadog Pathfinding Labs Solved. Here's What Made It Work.

tl;dr

We pointed our autonomous offensive agent, AgentO, at Datadog's Pathfinding Labs, a new, public collection of intentionally vulnerable AWS environments built by Seth Art, author of IAM Vulnerable. Starting each lab from a near-zero foothold (an identity that can do only a handful of actions), the agent had to enumerate the environment, find the privilege-escalation path, and execute it to capture the flag.

It solved all 122 live labs. Average path: 9 turns, 25% of the total being solved in less than 5 turns. No human in the loop, no solution access, and — because the labs are new — nothing the underlying model could have seen during training.

The model driving it was not a frontier model. It was an open-source ~35B-parameter open MoE (Mixture of Experts) with roughly 3B active parameters per token, running in our environment. The difference that made this feasible? A multi-cloud Knowledge Graph trained using OFFENSAI's proprietary Universal Offensive Ontology (UOO), built for autonomous cloud exploitation.

By grounding smaller models in the right context, we can achieve performance rivaling frontier-scale systems. This approach enables secure, private, and self-hosted deployments, eliminating reliance on third-party APIs while significantly lowering hosting and operational costs.

Why this is hard, and why scanners don't do it

A cloud posture scan answers a static question: does a misconfiguration matching a known signature exist? That is useful and necessary. But it is not what an attacker does.

An attacker lands on a foothold with almost no context, requiring them to build the context. They have to enumerate identities, read policies, notice that this role can pass that role, that this function can be overwritten, that this access key can be minted for a user it has no business touching, and then they chain those observations into a path to the objective. The misconfiguration that matters is rarely a single flagged line. It's the relationship between several individually unremarkable permissions, specific to one environment, that no signature anticipates.

This is why every environment is effectively a unique puzzle. The primitives are well known: PassRole, CreateAccessKey, AssumeRole chains, function overwrites. What's not known in advance is which primitives are reachable from where you happen to land, in this account, today. Discovering that requires reasoning over live state, not matching against a list.

That's the capability we set out to build and, more importantly, to measure.

What we tested it against

Pathfinding Labs, developed by Seth Art, Cloud Security Research and Advocacy at Datadog and author of the popular IAM Vulnerable lab, deploys intentionally vulnerable IAM configurations into a sandbox AWS account and drops the tester into the position of an attacker who has just compromised a low-privilege identity. The starting user holds exactly two permissions: sts:GetCallerIdentity and iam:GetUser, enough to know who you are and nothing else. From there, the path to admin (or to a sensitive S3 bucket) has to be discovered and walked.

The labs span the realistic surface: classic IAM escalations, compute-service paths through EC2, ECS, Lambda, CodeBuild, Glue, SageMaker and a long tail of less-trodden services, plus cross-account role-assumption chains. Across the full set there are single-step paths and multi-step ones, AWS-only by design.

Two properties make this fair:

• The agent never saw the answers. Pathfinding Labs ships demo scripts that reveal each solution; the agent had no access to them. It solved each lab by interacting with the live environment, not by reading a walkthrough.
• The labs were new, so there was nothing for the model to memorize. The building blocks of AWS privilege escalation are no secret and they're thoroughly documented and sit in any model's training data, and we're not claiming the agent rediscovered them. Knowing that PassRole exists is the easy part. The hard part is working out that this identity, in this environment, can reach admin through one specific PassRole edge, one that only surfaces after several steps of enumeration. You can't recall your way to a flag in a live account you've never seen. You have to discover it.

Results: AgentO with our knowledge graph

AgentO captured the flag on all 122 live labs.

Every single solve came in at or under 17 turns, comfortably below the 30-turn cutoff we set in advance for declaring a failure. There was no lab the agent struggled across the line on.

The spread is the interesting part. The simplest paths — direct role-assumption chains, a one-turn role-chain-to-s3 — fall in 1–3 turns. The cost concentrates not where you'd guess. It isn't the cross-account or "multi-hop" labels that stress the agent (those resolved fastest in our runs). It's the privilege-escalation paths through services without a well-worn public playbook: Cognito Identity, SageMaker, App Runner, Batch. These are where the agent spends 13–17 turns reasoning out an escalation primitive that isn't sitting in a blog post somewhere. That is exactly the case static tooling and pattern-matching handle the worst, and it's where contextual discovery earns its keep.

Per-category breakdown (single-run categories are one data point, not an average):

The 16 further categories, which cover services like cognitoidentity, codedeploy, emr, gamelift, braket, imagebuilder, and kinesisanalytics, are reported as individual runs, not averages.

After reviewing our results, Seth Art, the labs' author, shared his take on what the numbers mean:

"OFFENSAI's results are a proof point that projects like Pathfinding Labs can help measure how quickly autonomous offensive security is evolving in the cloud."

— Seth Art, Cloud Security Research and Advocacy at Datadog, author of Pathfinding Labs

What this does and doesn't prove

The scope matters as much as the score, so here's exactly what these results cover, and what they don't.

• AWS only. Pathfinding Labs is AWS-only by design. OFFENSAI's CSRE spans AWS, Azure, GCP and cross-cloud paths, but this experiment tested AWS privilege escalation. We are not claiming these results demonstrate multi-cloud capability, that's a separate body of work we have in our sight.
• Privilege escalation, specifically. These labs test identity-based escalation and cross-account movement. They are not a measure of the full offensive lifecycle.
• The labs derive from known primitives. As mentioned above, the techniques are public; the difficulty is environmental discovery, not novel exploitation.
• Scope of the engineering claim. Our ablation isolates the CSRE against a fixed agent harness. It shows the CSRE contribution given AgentO. It does not separately quantify the harness's own contribution. We claim accordingly.
• Single runs in the long tail. Sixteen categories are represented by one lab; their figures are individual runs, not averages. The aggregate (122 labs) is where the stable signal lives, anchored by IAM, Lambda, ECS and Glue.

None of these undercut the headline. A small, private model autonomously solved every live AWS privilege-escalation lab in a new environment it had never seen, in under 17 turns, with no access to the answers.

Conclusions

Effective cloud security doesn't require sending your sensitive environment data to a black-box model. By grounding autonomous agents in the structural reality of your cloud (the actual relationships and paths that exist today) we can shift from reactive scanning to proactive, deterministic attack chains.

If you're ready to see how your environment's unique attack surface looks when mapped and navigated by an agent that actually understands it, let's talk. Reach out to us at hello@offensai.com to discuss how you can deploy AgentO within your own VPC.