My browser use sucks on this stress test... am I using it wrong?

[nickguyai]

I tried to use claude-4, gemini, o3 etc to run this stress test with the latest version of browser use
vision, memory, planning... all default setting
But I kept getting 8-10/31 on this
What's the expected score? It feels this should be passing at a much higher rate somehow

[pirate]

That's a decent score actually. I wrote this list specifically with the knowledge of how our systems work internally to highlight areas where we're falling short and need to improve, it's not just a random selection of tasks from the internet. e.g. polygon hovering trajectories is really rough to do without CUA / using our current approaches. I also make it deliberately harder by throwing in curveballs in the html and css, e.g. disabling some fields that look enabled visually, messing with agent inputs using JS hooks to simulate laggy forms, etc.

Some of these seem like they should be easy but if you consider the architecture we use to detect interactive elements, or that we dont do xy-coordinate based automation (at least for now), they get quite difficult.

On our more realistic day-to-day tasks (e.g. logging into popular websites, booking flights, comparing prices online, filling out forms, etc) we usually see 70-80%+ success rates depending on model and settings used.

We are working hard to hit 100% in all evals - it's our top priority, with speed a close second and stealth 3rd.

[hgp22]

@pirate is it possible to create a dataset where users can submit the llm model and test score for this stress test?