Web Scraping & Search

Paid web crawling and search API at https://web.surf.cascade.fyi . Crawl costs $0.005 USDC, search costs $0.01 USDC per call via x402 on Solana. Use the x_payment tool for all requests. Use this when the built-in web_fetch tool fails or is blocked (anti-bot, JS-rendered pages, paywalled content).

Endpoints

Crawl Web Pages

Fetch and extract content from any URL. Automatically escalates through fetching tiers if blocked: fast HTTP -> headless browser -> stealth browser with proxy.

x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{"url": "https://example.com\", "format": "markdown"}" })

Body parameters:

Param Type Default Description

url string

Single URL to crawl (required if no urls )

urls string[]

Multiple URLs to bulk crawl (required if no url )

format string markdown Output format: markdown , html , or text

selector string

CSS selector to extract specific elements

proxy boolean false Force proxy usage (auto-enabled on stealth tier)

Single URL response:

{ "status": 200, "content": ["# Page Title\n\nPage content in markdown..."], "url": "https://example.com/" }

Bulk URL response:

[ {"status": 200, "content": ["..."], "url": "https://example.com/"}, {"status": 200, "content": ["..."], "url": "https://other.com/"} ]

Cascade escalation: The crawler automatically tries increasingly powerful methods:

• get - Fast HTTP request (cheapest, handles most static sites)

• fetch - Headless Chromium (for JS-rendered pages)

• stealthy_fetch - Stealth browser with proxy (bypasses Cloudflare, anti-bot)

Escalation triggers: HTTP 403/429/503, empty content, or content under 100 characters.

Search the Web

Search the web using Exa's search API. Returns titles, URLs, and text snippets.

x_payment({ "url": "https://web.surf.cascade.fyi/v1/search", "method": "POST", "params": "{"query": "x402 protocol crypto payments", "num_results": 10}" })

Body parameters:

Param Type Default Description

query string required Search query

num_results 1-20 5 Number of results to return

Response:

{ "results": [ { "title": "Page Title", "url": "https://example.com/page", "snippet": "Relevant text excerpt from the page..." } ] }

Usage Patterns

Scrape a page as markdown

x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{"url": "https://docs.solana.com\", "format": "markdown"}" })

Extract specific elements with a CSS selector

x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{"url": "https://news.ycombinator.com\", "format": "html", "selector": ".titleline"}" })

Bulk crawl multiple URLs

x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{"urls": ["https://example.com", "https://httpbin.org/get"], "format": "text"}" })

Search and then crawl top results

First search:

x_payment({ "url": "https://web.surf.cascade.fyi/v1/search", "method": "POST", "params": "{"query": "ERC-8004 agent identity standard", "num_results": 5}" })

Then crawl the most relevant result:

x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{"url": "https://eips.ethereum.org/EIPS/eip-8004\", "format": "markdown"}" })

Cost

• Crawl: $0.005 USDC per call (single or bulk)

• Search: $0.01 USDC per call

All payments on Solana mainnet. Each request is a separate call. For bulk crawls, one payment covers all URLs in the batch.

Errors

HTTP Meaning

400 Invalid parameters (check body format)

402 Payment required (handled automatically by x_payment)

429 Too many concurrent requests (retry later)

502 Upstream crawl/search error