Common Patterns

Ready-to-use code patterns for common TinyFish Web Agent use cases

These patterns cover the most common ways to integrate TinyFish Web Agent into your application.

Simple Extraction

Use the synchronous endpoint for quick, one-off extractions where you need the result immediately.

import { TinyFish, RunStatus } from "@tiny-fish/sdk";

const client = new TinyFish();

async function extractData(url: string, dataDescription: string) {
  const run = await client.agent.run({
    url,
    goal: `Extract ${dataDescription}. Return as JSON.`,
  });

  return run.status === RunStatus.COMPLETED ? run.result : null;
}

// Usage
async function main() {
  const products = await extractData(
    "https://example.com/products",
    "all product names and prices"
  );
  console.log(products);
}

main();

---

Batch Processing

For multiple URLs, use the async endpoint to submit all tasks at once, then poll for results. This avoids blocking while waiting for each task to complete.

import { TinyFish, RunStatus } from "@tiny-fish/sdk";

const client = new TinyFish();

async function processBatch(tasks: { url: string; goal: string }[]) {
  // Submit all tasks
  const responses = await Promise.all(
    tasks.map((task) => client.agent.queue(task))
  );

  // Poll for completion
  const maxAttempts = 150; // 5 minutes at 2s intervals
  const results = await Promise.all(
    responses.map(async (r) => {
      if (r.error) {
        throw r.error;
      }
      for (let attempt = 0; attempt < maxAttempts; attempt++) {
        const run = await client.runs.get(r.run_id);
        if (
          run.status === RunStatus.COMPLETED ||
          run.status === RunStatus.FAILED ||
          run.status === RunStatus.CANCELLED
        ) {
          return run;
        }
        await new Promise((resolve) => setTimeout(resolve, 2000));
      }
      throw new Error(`Run ${r.run_id} timed out after ${maxAttempts} attempts`);
    })
  );

  return results;
}

// Usage
async function main() {
  const results = await processBatch([
    { url: "https://example.com/page1", goal: "Extract product info" },
    { url: "https://example.com/page2", goal: "Extract product info" },
  ]);
  console.log(results);
}

main();

---

Retry with Stealth Mode

Some sites block automated requests. Start with lite mode for speed, then automatically retry with stealth mode if you get blocked.

import { TinyFish, RunStatus, BrowserProfile, ProxyCountryCode } from "@tiny-fish/sdk";

const client = new TinyFish();

async function extractWithFallback(url: string, goal: string) {
  // Try standard mode first
  let result = await client.agent.run({
    url,
    goal,
    browser_profile: BrowserProfile.LITE,
  });

  if (result.status === RunStatus.FAILED && result.error?.message.includes("blocked")) {
    // Retry with stealth mode
    result = await client.agent.run({
      url,
      goal,
      browser_profile: BrowserProfile.STEALTH,
      proxy_config: { enabled: true, country_code: ProxyCountryCode.US },
    });
  }

  return result;
}

---

Result Validation

A run with COMPLETED status means the agent finished, but the result may still describe a failure (e.g., the site showed a captcha or access-denied page). Always validate the result content.

Python

def is_real_success(result):
    """COMPLETED status is necessary but not sufficient."""
    if not result:
        return False
    result_str = str(result).lower()
    failure_signals = ["captcha", "blocked", "access denied", "could not", "unable to"]
    return not any(signal in result_str for signal in failure_signals)

# Usage
from tinyfish import TinyFish, RunStatus

client = TinyFish()
run = client.agent.run(
    url="https://example.com",
    goal="Extract pricing data. Return as JSON.",
)
if run.status == RunStatus.COMPLETED and is_real_success(run.result):
    print("Success:", run.result)
else:
    print("Needs retry or manual review")

---

Rate Limit Handling

TinyFish has concurrency limits based on your plan. The SDK automatically retries 429 and 5xx errors with exponential backoff (up to maxRetries attempts, default 2).

import { TinyFish, RateLimitError } from "@tiny-fish/sdk";

// Adjust retry behavior via client options
const client = new TinyFish({
  maxRetries: 3,  // default is 2
});

async function main() {
  try {
    const run = await client.agent.run({
      url: "https://example.com",
      goal: "Extract data",
    });
    console.log(run.result);
  } catch (e) {
    if (e instanceof RateLimitError) {
      console.log("Rate limited after all retries exhausted");
    }
    throw e;
  }
}

main();

---

Authenticated Automation

Use vault credentials to automate sites that require login. Connect your password manager once via the vault setup, then pass credentials to each run.

cURL

# Use all vault credentials
curl -X POST https://agent.tinyfish.ai/v1/automation/run \
  -H "X-API-Key: $TINYFISH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.linkedin.com",
    "goal": "Log in and extract the latest 3 posts from my feed. Return as JSON.",
    "use_vault": true
  }'

# Scope to a specific credential
curl -X POST https://agent.tinyfish.ai/v1/automation/run \
  -H "X-API-Key: $TINYFISH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.linkedin.com",
    "goal": "Log in and extract the latest 3 posts from my feed. Return as JSON.",
    "use_vault": true,
    "credential_item_ids": ["cred:conn-abc:Work:item-linkedin"]
  }'

Use credential_item_ids to scope runs to specific credentials when you have multiple accounts on the same domain. See Vault Credentials for details.

---

Cross-API Workflows

Chain multiple TinyFish APIs together for complex workflows.

Search + Fetch

Search for URLs, then fetch full content from the top results:

Python

from tinyfish import TinyFish

client = TinyFish()

# Step 1: Search for relevant pages
search_results = client.search.query("best python web frameworks 2026")

# Step 2: Fetch content from top 3 results
urls = [r.url for r in search_results.results[:3]]
fetched = client.fetch.get_contents(urls=urls, format="markdown")

for page in fetched.results:
    print(f"{page.title}: {page.text[:200]}...")

Search + Agent

Search for a URL, then extract structured data via the Agent API:

Python

from tinyfish import TinyFish, RunStatus

client = TinyFish()

# Step 1: Find a product page
search_results = client.search.query("scrapeme pokemon shop bulbasaur")
target_url = search_results.results[0].url

# Step 2: Extract structured data with the Agent
run = client.agent.run(
    url=target_url,
    goal="""Extract from this product page:
    - product_name, price (number), in_stock (boolean)
    Return as JSON.""",
)

if run.status == RunStatus.COMPLETED:
    print(run.result)

---

Related

AI Integration Guide

Best practices for AI agents

Examples

More detailed examples