Puppeteer Browser Automation & Scraping

Pain Points Solved & Core Value

Traditional web scraping scripts break frequently when DOM structures change or sites implement CAPTCHAs. The Puppeteer Browser Automation MCP eliminates this fragility by giving OpenClaw a real headless (or headful) Chrome browser to "see" and "click" pages exactly like a human would.

• Dynamic SPA Rendering: Works perfectly on Next.js, React, and Vue sites where content is loaded asynchronously via API calls.
• Visual Navigation: By observing CSS selectors and even visual coordinates, the agent can navigate complex login flows, menus, and popups.
• Action Replay: Automate daily mundane tasks—like downloading invoices from portables or checking stock inventory—without writing a single line of Python or Node.js logic.

Architecture under the Hood

This skill bridges the Model Context Protocol (MCP) with Google's Puppeteer library. When you prompt the agent to "Find the cheapest flights to Tokyo on Kayak," it dynamically generates puppeteer commands (e.g., page.goto(), page.click(), page.type()), streaming the browser's DOM state and screenshots back into the agent's context window. This creates a closed-loop visual reasoning cycle.

Top 5 Magic Prompts to Try

1. "Open Twitter, search for '$NVDA', take a screenshot of the top 3 posts, and summarize the market sentiment."
2. "Navigate to the AWS Console, log in using the credentials in my local vault, and download last month's billing PDF."
3. "Go to booking.com, scrape the top 10 hotels in Paris under $150 for this weekend, and format the result into a markdown table."
4. "Fill out this giant Google Form link using my resume data, handling all multipage 'Next' buttons."
5. "Monitor this real estate search page every 5 minutes. If a new listing appears under my budget, trigger a sound alert."

Server Configuration Example (config.json)

To enable this skill, inject the following configuration into your OpenClaw or Anthropic Desktop config.json:

"mcpServers": {
  "puppeteer": {
    "command": "npx",
    "args": [
      "-y",
      "@modelcontextprotocol/server-puppeteer",
      "--headless=false",
      "--window-size=1920,1080"
    ]
  }
}

Pro-Tip: Setting --headless=false allows you to watch the automation unfold on your screen, which is amazing for debugging and security audits!

Troubleshooting Notes

• Issue: Element Not Found. If the bot gets stuck, tell it to "Take a screenshot to understand the current viewport layout."
• Issue: Chromium Download Failure. Run PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true npm install and manually point to your local Chrome binary via arguments.

Security & Permissions

High Risk Warning: A browser MCP has total access to your local network and can potentially click malicious links or leak session cookies. NEVER use this plugin combined with autonomous multi-agent systems on highly sensitive financial accounts without human-in-the-loop approval steps.