What Is an AI Browser Agent?

An AI crawler (Googlebot, GPTBot, ClaudeBot) fetches a page once to feed a training set or answer index, identifies itself with a distinct user agent, and typically respects robots.txt as a matter of policy. An AI browser agent instead runs inside a real, full browser session on behalf of a single user's live task — filling out a form, comparing prices across tabs, completing a purchase — and its traffic carries the genuine user agent and rendering behavior of whatever browser it's driving, not a bot-labeled string.

That's what makes agent traffic hard to police with the tools that catch crawlers: header checks, rate limits, and robots.txt all assume a client that identifies itself or requests at bot-like volume. An agent looks, on the wire, like one real person using one real browser — because in a meaningful sense, that's exactly what it's doing.

Because agent traffic passes the checks that catch scrapers, detection has moved to signals that only exist once a page actually runs: canvas and WebGL output, session cadence, and increasingly behavior that's too fast, too continuous, or too precisely reactive to be human — an agent that hits checkout the instant stock changes, with no pause to read the page, is a different shape of traffic than a person refreshing a page. Anti-bot vendors have started shipping products that classify traffic into three buckets instead of two — human, authorized/trusted agent, and bot — rather than treating all automation as equally hostile.

The practical effect for anyone building on top of AI agents: running inside a real browser is necessary but no longer sufficient to be treated as legitimate traffic. Sites increasingly grant differentiated access to agents they can recognize and trust, which raises the same identity question this glossary's Web Bot Auth entry covers for crawlers, now applied to agents that act rather than just read.