> ## Documentation Index > Fetch the complete documentation index at: https://docs.tinyfish.ai/llms.txt > Use this file to discover all available pages before exploring further. # Fetch and extract content from URLs > Fetches web pages, renders JavaScript-heavy pages when needed, and returns clean extracted content in your preferred format. Submit up to 10 URLs, get back structured content. Per-URL failures appear in `errors[]` and do not fail the entire request. **Per-URL error codes** (in `errors[].error`): - `target_http_error` — target server returned a non-2xx HTTP status other than 404/410; the raw status code is in `errors[].status` - `page_not_found` — target URL returned HTTP 404 or 410; the raw status code is in `errors[].status` - `target_unreachable` — connection refused, TLS failure, DNS failure, or other network error - `timeout` — request timed out - `proxy_error` — proxy tunnel failure - `bot_blocked` — bot-challenge page detected (Cloudflare, etc.) - `empty_content` — page loaded but no extractable text was found - `invalid_url` — malformed URL or SSRF-blocked address - `invalid_redirect_url` — redirect target rejected before fetch - `conditional_unsupported` — conditional requests (`if_none_match` / `if_modified_since`) are supported on the fast path only; this URL requires browser rendering - `selector_not_matched` — no elements matching any `include_selectors` entry remained after `exclude_selectors` was applied; the error carries `unmatched_selectors` plus `candidate_selectors` retry hints (a partial miss is not an error — it's reported on the result's `unmatched_selectors`) - `selector_unsupported` — `include_selectors` / `exclude_selectors` sent for a URL that resolves to a direct PDF/CSV download (no HTML to scope) ## OpenAPI ````yaml https://agent.tinyfish.ai/v1/openapi/fetch post / openapi: 3.0.0 info: title: TinyFish Fetch API version: 1.0.0 description: Fetch URLs and extract clean page content. contact: name: TinyFish Support email: support@tinyfish.ai servers: - url: https://api.fetch.tinyfish.ai security: [] paths: /: post: summary: Fetch and extract content from URLs description: >- Fetches web pages, renders JavaScript-heavy pages when needed, and returns clean extracted content in your preferred format. Submit up to 10 URLs, get back structured content. Per-URL failures appear in `errors[]` and do not fail the entire request. **Per-URL error codes** (in `errors[].error`): - `target_http_error` — target server returned a non-2xx HTTP status other than 404/410; the raw status code is in `errors[].status` - `page_not_found` — target URL returned HTTP 404 or 410; the raw status code is in `errors[].status` - `target_unreachable` — connection refused, TLS failure, DNS failure, or other network error - `timeout` — request timed out - `proxy_error` — proxy tunnel failure - `bot_blocked` — bot-challenge page detected (Cloudflare, etc.) - `empty_content` — page loaded but no extractable text was found - `invalid_url` — malformed URL or SSRF-blocked address - `invalid_redirect_url` — redirect target rejected before fetch - `conditional_unsupported` — conditional requests (`if_none_match` / `if_modified_since`) are supported on the fast path only; this URL requires browser rendering - `selector_not_matched` — no elements matching any `include_selectors` entry remained after `exclude_selectors` was applied; the error carries `unmatched_selectors` plus `candidate_selectors` retry hints (a partial miss is not an error — it's reported on the result's `unmatched_selectors`) - `selector_unsupported` — `include_selectors` / `exclude_selectors` sent for a URL that resolves to a direct PDF/CSV download (no HTML to scope) operationId: fetchUrls requestBody: description: URLs to fetch and extraction options required: true content: application/json: schema: type: object properties: urls: type: array items: type: string format: uri minItems: 1 maxItems: 10 description: >- Array of URLs to fetch (1-10). All URLs are fetched in parallel. Each URL is processed independently — if one fails, others still return successfully. Errors are reported per-URL in the errors array. example: - https://example.com - https://example.org purpose: type: string minLength: 1 maxLength: 2000 description: >- Why these URLs are being fetched — the underlying goal or task the content will be used for. Used to better tailor fetching and extraction to your intent. example: >- Compare pricing tiers across vendors for a procurement report format: type: string enum: - markdown - html - json default: markdown description: >- Output format for extracted content. "markdown" (default) is ideal for LLM consumption. "html" returns cleaned semantic HTML. "json" returns a structured document tree. example: markdown links: type: boolean default: false description: >- Extract all outbound links () from each page. Useful for discovering related pages or navigating to specific content. Links are returned as absolute URLs in the links array of each result. example: false image_links: type: boolean default: false description: >- Extract all image URLs () from each page. Useful for finding visual content or media assets. Image links are returned as absolute URLs in the image_links array of each result. example: false ttl: type: integer minimum: 0 description: >- Caller freshness tolerance in seconds for the cached entry. Omit (default) for unlimited tolerance — any cached entry is acceptable. Set to 0 to prefer a live fetch; a cached entry is still served if the origin's Cache-Control: max-age covers its age, or the host is in the small allowlist of operator-pinned never-expire domains. Set to N > 0 to accept a cached entry whose age is below N; the upstream Cache-Control: max-age and the never-expire allowlist may extend (never shorten) this tolerance. example: 0 per_url_timeout_ms: type: integer minimum: 1 maximum: 110000 description: >- Wall-clock timeout budget in milliseconds applied independently to each URL. If one URL exceeds this budget, it returns a per-URL timeout error while other URLs in the same request continue. example: 45000 if_none_match: type: string minLength: 1 description: >- ETag validator from a prior fetch of this URL, forwarded verbatim as the If-None-Match header on the origin request. Only valid with a single URL — combining with a batch of URLs returns a 400. tf-fetch does not persist validators; the caller owns replaying them. example: W/"abc123" if_modified_since: type: string minLength: 1 description: >- Last-Modified validator from a prior fetch of this URL, forwarded verbatim as the If-Modified-Since header on the origin request. Only valid with a single URL — combining with a batch of URLs returns a 400. tf-fetch does not persist validators; the caller owns replaying them. example: Wed, 21 Oct 2015 07:28:00 GMT include_etag_and_last_modified: type: boolean description: >- Opt-in to receiving `etag` / `last_modified` validators (and `not_modified` detection) on each result. Defaults to false — tf-fetch omits these fields unless requested. Independent of `if_none_match` / `if_modified_since`: works with a single URL or a batch. example: true include_selectors: type: array items: type: string minLength: 1 maxLength: 1000 minItems: 1 maxItems: 20 description: >- Array of CSS selectors (1-20 entries, each 1-1000 characters) that scope extracted content (`text`, `links`, `image_links`) to elements matching ANY entry, concatenated in document order. Tag selectors cover semantic sections (`main`, `article`, `nav`); entries may themselves use CSS comma-grouping. Selected content is returned verbatim in the requested format (scripts/styles stripped) — automatic boilerplate removal is bypassed. Page-level metadata (`title`, `description`, `language`, `author`, `published_date`) still comes from the full document. When some entries match and others do not, the URL still succeeds and the misses are reported in the result's `unmatched_selectors`. When no entry matches anything, that URL fails with the per-URL error code `selector_not_matched` (carrying `unmatched_selectors` and `candidate_selectors` retry hints) — never a silent full-page fallback. URLs that resolve to direct PDF/CSV downloads fail with `selector_unsupported`. Invalid CSS selector syntax is rejected with a 422. Applied post-fetch: caching and routing are unchanged. example: - article exclude_selectors: type: array items: type: string minLength: 1 maxLength: 1000 minItems: 1 maxItems: 20 description: >- Array of CSS selectors (1-20 entries, each 1-1000 characters) for elements to remove before extraction — applied before `include_selectors` scopes what remains, so it also prunes inside selected regions. Entries may themselves use CSS comma-grouping. Entries that match nothing are a no-op, never an error, but URLs that resolve to direct PDF/CSV downloads fail with `selector_unsupported`. Invalid CSS selector syntax is rejected with a 422. Applied post-fetch: caching and routing are unchanged. example: - .comments - .newsletter-signup required: - urls responses: '200': description: Fetch completed. Check `errors[]` for any per-URL failures. content: application/json: schema: type: object properties: results: type: array items: type: object properties: url: type: string description: Original requested URL example: https://example.com final_url: type: string nullable: true description: Final URL after redirects example: https://www.example.com title: type: string nullable: true description: Page title example: Example Domain description: type: string nullable: true description: Page meta description example: This domain is for use in illustrative examples. language: type: string nullable: true description: Detected language code example: en format: type: string enum: - markdown - html - json description: Format of the extracted text example: markdown text: anyOf: - type: string - {} - nullable: true description: >- Extracted content (string for markdown/html, object for json). Null if extraction failed. author: type: string nullable: true description: Page author if available example: John Doe published_date: type: string nullable: true description: Published date if available example: '2024-01-15' links: type: array items: type: string description: Extracted links (only if links=true in request) image_links: type: array items: type: string description: >- Extracted image links (only if image_links=true in request) latency_ms: type: number nullable: true description: >- Fetch latency for this URL in milliseconds if available from tf-fetch example: 1183.4 not_modified: type: boolean description: >- Present and `true` only when the origin returned 304 Not Modified for your `if_none_match`/`if_modified_since`; omitted otherwise. example: true etag: type: string description: >- The origin's current ETag validator, present only when the origin sent one and you set `include_etag_and_last_modified: true`; omitted otherwise. Store it and replay it as `if_none_match`. example: W/"abc123" last_modified: type: string description: >- The origin's current Last-Modified validator, present only when the origin sent one and you set `include_etag_and_last_modified: true`; omitted otherwise. Store it and replay it as `if_modified_since`. example: Wed, 21 Oct 2015 07:28:00 GMT unmatched_selectors: type: array items: type: string description: >- Partial-miss report for `include_selectors`: the entries that matched no elements on this page, present only when some (but not all) entries matched — the result still carries the matched content. Omitted when every entry matched. A total miss is not a result at all: it is reported as a `selector_not_matched` entry in `errors[]`. example: - aside.related required: - url - final_url - title - description - language - format - text - author - published_date description: Successfully fetched URLs errors: type: array items: type: object properties: url: type: string description: URL that failed to fetch example: https://invalid.example.com error: type: string description: >- Error code describing the failure. Possible values: `target_http_error` (target server returned a non-2xx HTTP status other than 404/410 — see `status` field), `page_not_found` (target returned HTTP 404 or 410), `target_unreachable` (connection refused, TLS failure, or network error), `timeout` (request timed out), `proxy_error` (proxy tunnel failure), `bot_blocked` (bot-challenge page detected), `empty_content` (page loaded but no extractable text), `invalid_url` (malformed URL or SSRF-blocked address), `invalid_redirect_url` (redirect target rejected before fetch), `conditional_unsupported` (conditional requests are supported on the fast path only; this URL requires browser rendering), `selector_not_matched` (no elements matching any `include_selectors` entry remained after `exclude_selectors` was applied — a total miss; carries `unmatched_selectors` and `candidate_selectors` retry hints. A partial miss is NOT an error: it is reported via the result's `unmatched_selectors`), `selector_unsupported` (`include_selectors`/`exclude_selectors` sent for a URL that resolves to a direct PDF/CSV download — no HTML to scope). example: target_http_error status: type: integer description: >- Upstream HTTP status code. Only present when `error` is `target_http_error` or `page_not_found`. example: 404 unmatched_selectors: type: array items: type: string description: >- The `include_selectors` entries that matched no elements on the fetched page. Only present when `error` is `selector_not_matched`. example: - article - '#content' candidate_selectors: type: array items: type: string description: >- Cheap retry hints (max 10), only present when `error` is `selector_not_matched`: landmark tags present on the page (e.g. `main`, `article`, `nav`) plus `#id` selectors of content-heavy elements. Retry the fetch with `include_selectors` drawn from this list. example: - main - nav - '#content' required: - url - error description: A URL that failed to fetch description: URLs that failed to fetch request_id: type: string description: >- Request correlation ID, also returned as the X-Request-ID response header. Present only when errors is non-empty; include it when reporting failed URLs. example: 8f9dba20-e37b-4749-a919-2269e28b4a2c required: - results - errors description: Fetch response with results and errors '400': description: >- Invalid request — missing `urls`, too many URLs (max 10), bad parameter value, or `if_none_match`/`if_modified_since` combined with a batch of URLs content: application/json: schema: type: object properties: error: type: object properties: code: type: string enum: - MISSING_API_KEY - INVALID_API_KEY - INVALID_INPUT - RATE_LIMIT_EXCEEDED - INTERNAL_ERROR - RETRY_REQUIRED - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - SERVICE_BUSY - TIMEOUT - INSUFFICIENT_CREDITS - CONTENT_POLICY_VIOLATION - MAX_STEPS_EXCEEDED - SITE_BLOCKED - TASK_FAILED - CANCELLED description: Machine-readable error code example: INVALID_INPUT message: type: string description: Human-readable error message example: Field "url" is required and must be a string details: nullable: true description: Additional error details (validation errors, etc.) required: - code - message request_id: type: string description: >- Request correlation ID, also returned as the X-Request-ID response header. Include it when reporting issues. example: 8f9dba20-e37b-4749-a919-2269e28b4a2c required: - error description: Standard error response format '401': description: Unauthorized - Invalid or missing API key content: application/json: schema: type: object properties: error: type: object properties: code: type: string enum: - MISSING_API_KEY - INVALID_API_KEY - INVALID_INPUT - RATE_LIMIT_EXCEEDED - INTERNAL_ERROR - RETRY_REQUIRED - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - SERVICE_BUSY - TIMEOUT - INSUFFICIENT_CREDITS - CONTENT_POLICY_VIOLATION - MAX_STEPS_EXCEEDED - SITE_BLOCKED - TASK_FAILED - CANCELLED description: Machine-readable error code example: INVALID_INPUT message: type: string description: Human-readable error message example: Field "url" is required and must be a string details: nullable: true description: Additional error details (validation errors, etc.) required: - code - message request_id: type: string description: >- Request correlation ID, also returned as the X-Request-ID response header. Include it when reporting issues. example: 8f9dba20-e37b-4749-a919-2269e28b4a2c required: - error description: Standard error response format '429': description: Rate limit exceeded content: application/json: schema: type: object properties: error: type: object properties: code: type: string enum: - MISSING_API_KEY - INVALID_API_KEY - INVALID_INPUT - RATE_LIMIT_EXCEEDED - INTERNAL_ERROR - RETRY_REQUIRED - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - SERVICE_BUSY - TIMEOUT - INSUFFICIENT_CREDITS - CONTENT_POLICY_VIOLATION - MAX_STEPS_EXCEEDED - SITE_BLOCKED - TASK_FAILED - CANCELLED description: Machine-readable error code example: INVALID_INPUT message: type: string description: Human-readable error message example: Field "url" is required and must be a string details: nullable: true description: Additional error details (validation errors, etc.) required: - code - message request_id: type: string description: >- Request correlation ID, also returned as the X-Request-ID response header. Include it when reporting issues. example: 8f9dba20-e37b-4749-a919-2269e28b4a2c required: - error description: Standard error response format '500': description: Internal server error content: application/json: schema: type: object properties: error: type: object properties: code: type: string enum: - MISSING_API_KEY - INVALID_API_KEY - INVALID_INPUT - RATE_LIMIT_EXCEEDED - INTERNAL_ERROR - RETRY_REQUIRED - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - SERVICE_BUSY - TIMEOUT - INSUFFICIENT_CREDITS - CONTENT_POLICY_VIOLATION - MAX_STEPS_EXCEEDED - SITE_BLOCKED - TASK_FAILED - CANCELLED description: Machine-readable error code example: INVALID_INPUT message: type: string description: Human-readable error message example: Field "url" is required and must be a string details: nullable: true description: Additional error details (validation errors, etc.) required: - code - message request_id: type: string description: >- Request correlation ID, also returned as the X-Request-ID response header. Include it when reporting issues. example: 8f9dba20-e37b-4749-a919-2269e28b4a2c required: - error description: Standard error response format security: - ApiKeyAuth: [] components: securitySchemes: ApiKeyAuth: type: apiKey in: header name: X-API-Key description: API key for authentication. Get your key from the API Keys page. ````