> ## Documentation Index > Fetch the complete documentation index at: https://docs.tinyfish.ai/llms.txt > Use this file to discover all available pages before exploring further. # Fetch API > Fetch URLs and extract clean text — no external APIs required Fetch does not use credits. The TinyFish Fetch API fetches web pages, renders JavaScript-heavy pages when needed, and returns clean extracted text in your preferred format. Submit a URL, get back structured content. ```bash theme={null} POST https://api.fetch.tinyfish.ai ``` `api.fetch.tinyfish.ai` is the public Fetch API endpoint. ## Before You Start Visit [agent.tinyfish.ai/api-keys](https://agent.tinyfish.ai/api-keys) and create a key. Store it in your environment: ```bash theme={null} export TINYFISH_API_KEY="your_api_key_here" ``` All requests require the `X-API-Key` header. See [Authentication](/authentication) for the full setup and troubleshooting guide. ## Your First Request ```python Python theme={null} from tinyfish import TinyFish client = TinyFish() result = client.fetch.get_contents(urls=["https://www.tinyfish.ai/"]) print(result.results[0].title) print(result.results[0].text) ``` ```typescript TypeScript theme={null} import { TinyFish } from "@tiny-fish/sdk"; const client = new TinyFish(); const result = await client.fetch.getContents({ urls: ["https://www.tinyfish.ai/"], }); console.log(result.results[0].title); console.log(result.results[0].text); ``` ```bash cURL theme={null} curl -X POST https://api.fetch.tinyfish.ai \ -H "X-API-Key: $TINYFISH_API_KEY" \ -H "Content-Type: application/json" \ -d '{"urls": ["https://www.tinyfish.ai/"]}' ``` ## What Success Looks Like ```json theme={null} { "results": [ { "url": "https://www.tinyfish.ai/", "final_url": "https://www.tinyfish.ai/", "title": "TinyFish | Enterprise Web Agent Infrastructure", "description": "TinyFish provides enterprise infrastructure for AI web agents.", "language": "en", "format": "markdown", "text": "# TinyFish | Enterprise Web Agent Infrastructure\n\nTinyFish provides enterprise infrastructure for AI web agents...\n" } ], "errors": [] } ``` ## When to Use Fetch vs the Other APIs * Use **Fetch** when you already know the URL and need clean extracted page content. * Use **Search** when you need help finding the right URLs first. * Use **Agent** when TinyFish should perform a multi-step workflow on the site. * Use **Browser** when you need direct browser control from your own code. *** ## Fetching Multiple URLs Submit up to 10 URLs in a single request. Each URL is processed independently — one failure doesn't affect the others. ```python Python theme={null} from tinyfish import TinyFish client = TinyFish() result = client.fetch.get_contents( urls=[ "https://www.tinyfish.ai/", "https://en.wikipedia.org/wiki/Web_scraping", "https://docs.python.org/3/tutorial/index.html", ] ) for page in result.results: print(page.url, "→", page.title) for error in result.errors: print("Failed:", error.url, "–", error.error) ``` ```typescript TypeScript theme={null} import { TinyFish } from "@tiny-fish/sdk"; const client = new TinyFish(); const result = await client.fetch.getContents({ urls: [ "https://www.tinyfish.ai/", "https://en.wikipedia.org/wiki/Web_scraping", "https://docs.python.org/3/tutorial/index.html", ], }); result.results.forEach((r) => console.log(r.url, "→", r.title)); result.errors.forEach((e) => console.log("Failed:", e.url, "–", e.error)); ``` ```bash cURL theme={null} curl -X POST https://api.fetch.tinyfish.ai \ -H "X-API-Key: $TINYFISH_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "urls": [ "https://www.tinyfish.ai/", "https://en.wikipedia.org/wiki/Web_scraping", "https://docs.python.org/3/tutorial/index.html" ] }' ``` Per-URL failures (timeouts, DNS errors, anti-bot blocks) appear in `errors[]` alongside a `200` response — they do not cause the entire request to fail. *** ## Output Formats Control the format of the `text` field with the `format` parameter. When omitted, the default is `markdown`. Semantic HTML. ```json theme={null} { "format": "html", "text": "

Async Fn in Traits Are Now Available

Starting with Rust 1.75, you can use async fn directly inside traits.

What Changed

" } ``` Clean Markdown. Ideal for LLM consumption and readable storage. ```json theme={null} { "format": "markdown", "text": "# Async Fn in Traits Are Now Available Starting with Rust 1.75, you can use `async fn` directly inside traits. ## What Changed Previously, writing async functions in traits required the `async-trait` crate... - Works in all stable traits - No heap allocation for simple cases - Compatible with `Send` bounds" } ``` Structured document tree. Useful for programmatic content processing. ```json theme={null} { "format": "json", "text": { "type": "document", "children": [ { "type": "heading", "level": 1, "text": "Async Fn in Traits Are Now Available" }, { "type": "paragraph", "text": "Starting with Rust 1.75, you can use async fn directly inside traits." }, { "type": "heading", "level": 2, "text": "What Changed" }, { "type": "list", "ordered": false, "items": ["Works in all stable traits", "No heap allocation for simple cases"] } ] } } ``` *** ## Cache Freshness By default, Fetch may serve an existing cached entry for the URL when one is available. Pass `ttl` to set a freshness preference in seconds. ```json theme={null} { "urls": ["https://example.com"], "ttl": 3600 } ``` * Omit `ttl` to accept any cached entry. * Set `ttl` to `0` when you want a live fetch. * Set `ttl` to a positive integer to accept cached entries younger than that many seconds. *** ## Conditional Requests Skip reprocessing pages that haven't changed since your last fetch. Set `include_etag_and_last_modified: true` to get `etag` / `last_modified` validators back on each result, save them, then replay the `etag` as `if_none_match` (or the `last_modified` as `if_modified_since`) the next time you fetch that URL. ```json theme={null} { "urls": ["https://example.com"], "if_none_match": "W/\"abc123\"", "include_etag_and_last_modified": true } ``` * `if_none_match` and `if_modified_since` are single-URL only — combining either with a batch of URLs returns a `400`. * When the origin confirms nothing changed, the result comes back with `not_modified: true` so you can skip re-processing. * Fetch persists nothing — you own storing and replaying the validators. See the [full reference](/fetch-api/reference#parameters) and a [worked example](/fetch-api/examples#detect-page-changes-with-conditional-requests). *** ## CSS Selector Scoping Extract only the part of the page you care about. Pass `include_selectors` (an array of CSS selectors, max 20) to scope extracted content to elements matching any entry, and/or `exclude_selectors` to strip matching elements before extraction. ```json theme={null} { "urls": ["https://example.com/blog/post"], "include_selectors": ["article"], "exclude_selectors": [".comments", ".newsletter-signup"] } ``` * Selected content is returned verbatim in your requested format — automatic boilerplate removal is bypassed. Page-level metadata (`title`, `description`, etc.) still comes from the full document. * `exclude_selectors` is applied first, then `include_selectors` scopes what remains. * If some entries match and others don't, the URL still succeeds and the misses are listed in the result's `unmatched_selectors`. If no entry matches anything, that URL fails with `selector_not_matched` in `errors[]`, carrying `unmatched_selectors` plus `candidate_selectors` retry hints — agents can retry with `include_selectors` drawn from that list. `exclude_selectors` entries that match nothing are a no-op. See the [full reference](/fetch-api/reference#parameters) and a [worked example](/fetch-api/examples#scope-extraction-to-part-of-the-page). *** ## Supported Content Types The Fetch API handles more than just HTML. See the [full content types table](/fetch-api/reference#supported-content-types) in the reference — highlights: PDF text extraction works, JSON endpoints return raw JSON, but binary files (images, video) return an error. *** ## Fetch Intent The optional `purpose` parameter lets you state *why* you are fetching — the underlying goal or task the content will be used for. A URL alone says nothing about what you need from the page. Passing `purpose` gives us additional signal to further inform and deliver better-quality results. It works the same way as the [Search API's `purpose` parameter](/search-api/index#search-intent). * `purpose` is always optional. Omitting it leaves fetch behaviour unchanged. * Keep it to a short phrase or sentence (maximum 2000 characters), for example `Compare pricing tiers across vendors for a procurement report`. *** ## Read Next Full request and response schema API key setup Common Fetch request patterns One page that routes an agent to the right TinyFish API