Getting started

xyte implements Zyte API's POST /v1/extract on any proxy provider, or none. It runs on Node.js ≥ 22 and Bun.

npm i @xtandard/xyte
# browser requests (browserHtml, screenshot, actions, networkCapture):
npm i playwright && npx playwright install chromium
# proxies on Node.js (Bun has proxy support built into fetch); undici 8 needs Node 22.19+:
npm i undici

The command it installs is xyte.

Library

import { Xyte, oxylabs, decodeBody } from '@xtandard/xyte';

const xyte = new Xyte({
  proxy: oxylabs({ username: process.env.OXYLABS_USERNAME!, password: process.env.OXYLABS_PASSWORD! }),
});

// Raw HTTP (cheap, fast)
const page = await xyte.extract({ url: 'https://example.com', httpResponseBody: true, geolocation: 'US' });
page.statusCode; // 200; a target 404 is a successful response with statusCode 404, as on Zyte
decodeBody(page.httpResponseBody!); // the HTML (bodies are base64, as on Zyte)

// Rendered in a browser, with Zyte actions
const rendered = await xyte.extract({
  url: 'https://example.com/app',
  browserHtml: true,
  actions: [
    { action: 'waitForSelector', selector: { type: 'css', value: '#results' }, timeout: 10 },
    { action: 'scrollBottom' },
  ],
  networkCapture: [{ filterType: 'url', matchType: 'contains', value: '/api/', httpResponseBody: true }],
});

await xyte.close(); // closes the shared browser

Closing the client

The first render launches a browser that the client reuses, and it keeps your process alive until you close the client:

await using xyte = new Xyte({ proxy }); // closed when the block ends, even on an exception
// or, where that syntax doesn't run (plain JavaScript on Node.js 22):
const xyte = new Xyte({ proxy });
try { … } finally { await xyte.close(); }

The client's lifecycle explains await using, where it runs, the return await trap, services (one client, closed on SIGTERM), Redis stores and tests.

Many at once

Many at once: for await (const r of xyte.extractMany(requests)) … yields results as they finish ({ index, response } or { index, error }), at most concurrency in flight, from any list or async source; extractAll() collects them in order. From a shell: xyte batch urls.txt > out.jsonl.

Requests and responses are typed from Zyte's OpenAPI spec. All options (retries, timeout, concurrency, isBan, onRetry…) are in configuration.

Drop-in server

XYTE_PROVIDER=oxylabs OXYLABS_USERNAME=… OXYLABS_PASSWORD=… npx @xtandard/xyte serve --port 8011

- const res = await fetch('https://api.zyte.com/v1/extract', {
+ const res = await fetch('http://localhost:8011/v1/extract', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', Authorization: 'Basic ' + btoa(apiKey + ':') },
    body: JSON.stringify({ url, httpResponseBody: true }),
  });

As an HTTP proxy, like Zyte's :8011: xyte serve --proxy-port 8012, then curl -x http://localhost:8012 -U key: --cacert <(xyte ca) https://site (proxy mode).

Long renders or queues: POST /v1/jobs (the same body, plus an optional callbackUrl) answers 202 { id } at once; GET /v1/jobs/{id} has the result, which is also POSTed (signed) to callbackUrl (jobs).

Set XYTE_API_KEYS=key1,key2 to require a key (sent the way Zyte expects it: basic-auth username). Docker images (full with Chromium, or slim HTTP-only) and Cloud Run / Fly / Render / App Platform / Modal configs: deploy. To serve it from your own app: createHandler(xyte) (Hono, Bun.serve, Next.js…) or toNodeHandler(…) (Express, Fastify), see mounting.

CLI one-offs:

npx @xtandard/xyte extract https://example.com --raw              # HTTP, prints the body
npx @xtandard/xyte extract https://example.com --browser --country GB
npx @xtandard/xyte extract --json '{"url":"https://example.com","browserHtml":true}'
npx @xtandard/xyte doctor --country US   # each configured provider: credentials, exit IP/country, quota

Providers

Preset	Exit	Geolocation / session encoded as
direct() (default)	this machine	none (geolocation can't apply: you get a warning)
proxyUrl(url | urls[])	any proxy; several rotate across retries	none
oxylabs({ username, password, type? })	pr.oxylabs.io:7777 / dc.oxylabs.io:8000	customer-U-cc-US-sessid-…-sesstime-10
brightData({ customerId, zone, password })	brd.superproxy.io:33335	brd-customer-…-zone-…-country-us-session-…
decodo({ username, password }) (alias smartproxy)	gate.decodo.com:7000	user-U-country-us-session-…-sessionduration-10
iproyal({ username, password })	geo.iproyal.com:12321	password P_country-us_session-…_lifetime-10m
zenrows({ apiKey })	ZenRows API proxy mode	premium_proxy=true&proxy_country=us&session_id=…
scrapingBee({ apiKey })	ScrapingBee proxy mode	premium_proxy=True&country_code=us&session_id=…
scraperApi({ apiKey })	ScraperAPI proxy mode	scraperapi.country_code=us.session_number=…
zyteApi({ apiKey, name? })	Zyte API itself	the request, forwarded unchanged
xyteApi({ endpoint, apiKey, name? })	another xyte serve (a deployment with its own proxy)	the request, forwarded; its attempts come back in the report
template('http://u[-country-{country:lower}]-session-{session}:p@host:port')	any provider, from its URL format	placeholders + optional [parts]
custom(name, ctx => url)	anything you build	you decide

…and NodeMaven, SOAX, NetNut, Webshare, Rayobyte, PacketStream, Evomi, DataImpulse, Massive, Proxy-Seller, Thordata, Anyip, ABCProxy, ProxyEmpire, RapidProxy, Swiftproxy, 922 S5, LunaProxy, plus Crawlbase, Scrape.do, ScrapingAnt, ScrapeOps, Scrapfly, Oxylabs Web Unblocker and Zyte's proxy mode, hosted browsers (Bright Data Scraping Browser, Browserless, ZenRows, Oxylabs, NodeMaven), and API-only services (Scrappey, ScrapingBot). The full catalog, with credential formats, env names and which ones were checked against a live account: providers.

From env (what the CLI uses): XYTE_PROVIDER=oxylabs plus OXYLABS_USERNAME / OXYLABS_PASSWORD, XYTE_PROXY_URL=http://u:p@host:port, or XYTE_PROXY_TEMPLATE=….

The scraping-API presets (ZenRows, ScrapingBee, ScraperAPI) re-sign TLS, so they set insecure automatically. They bill per request, so in a browser every sub-request of the page is billed. Use them with httpResponseBody.

Mixing providers, Zyte included

No single provider is the cheapest and gets through everywhere. Three combinators mix them behind one client (details and recipes):

import { direct, oxylabs, rotate, route, Xyte, zyteApi } from '@xtandard/xyte';

const residential = oxylabs({ username, password });
const zyte = zyteApi({ apiKey: process.env.ZYTE_API_KEY! });

const xyte = new Xyte({
  proxy: route(
    {
      'wolt.com': direct(),                               // answers anyone: free
      'restaurantguru.com': rotate([residential, zyte]),  // Zyte only after a ban
      'doordash.com': zyte,                               // only Zyte gets through
    },
    residential, // every other site
  ),
});

• rotate([a, b]) picks by attempt: the first try uses a, the retry b, then a again. A fallback after a ban, or when a provider is down. Env: XYTE_PROVIDER=oxylabs,zyte-api.

• route({ host: provider }, fallback) picks by the target's host (subdomains included). Env: XYTE_ROUTES=wolt.com=direct,doordash.com=zyte-api,restaurantguru.com=oxylabs+zyte-api.

• adaptive([direct(), residential, zyte]) learns it instead: per site, the cheapest provider that has been working, moving up after bans and back down when they stop. Env: XYTE_ADAPTIVE=direct,oxylabs,zyte-api.

• byKind({ http, browser }), byCountry({ US: p }, fallback), byIpType({ datacenter, residential }) pick by request kind, geolocation, ipType; weighted([[a, 9], [b, 1]]) splits traffic.

• healthy(p) is a circuit breaker (a provider that keeps failing is skipped for a while instead of costing an attempt per request), budget(p, { max: 500, per: 'hour' }) caps what a provider (say, the Zyte fallback) can spend.

• watch(p, { check }) asks a provider's usage API or status page ahead of requests (ScrapingBee, ZenRows, ScraperAPI, Oxylabs, any Statuspage), so one that ran out of credits or is down is skipped before an attempt is spent on it. Env: XYTE_WATCH=true.

With XYTE_REDIS_URL (or a store), budgets, open circuits and checks are shared by every instance. browserFallback: true retries a banned plain GET in a browser (JavaScript challenges), returning the raw response as Zyte does.

Spending less

• cache: { ttl: 3600 } answers repeats from a cache (per site TTLs, Redis to share it), and identical requests in flight share one run.
• rateLimits: { 'yelp.co.uk': '2/s', '*': '10/s' } paces attempts per site: hammering one site is the fastest way to get banned.
• rates: { oxylabs: { perGB: 6 }, 'zyte-api': { perRequest: 0.0013 } }, your prices, put costUsd on every attempt and report.

Seeing what it did

Every request can tell you what it took, without changing the Zyte response:

new Xyte({ proxy, onRequest: (report) => analytics.insert('scrapes', report) });
// report: outcome, provider that served it, every attempt (provider, route, result, ms,
// bytes on the wire), totals per provider, whether it escalated to a browser.

xyte serve adds X-Xyte-Attempts: oxylabs-residential:ban,zyte-api:ok and X-Xyte-Transfer-Bytes to its responses, and XYTE_LOG_REQUESTS=true logs each report as a JSON line, ready for BigQuery or Datadog. Success rates and MB per provider and site, and what they cost at your rates: observability.

Zyte as a provider. zyteApi({ apiKey }) forwards the request to Zyte unchanged, so every Zyte feature works, AI extraction included, and Zyte's answers (and errors) come back as Zyte sent them. Even with only Zyte behind it, you get the library: typed requests, XyteErrors, retries, a concurrency limit, and the option to move sites to cheaper exits later without touching calling code. zyteProxy({ apiKey }) uses Zyte's proxy mode instead: xyte fetches and renders, exiting through Zyte's IPs (no AI extraction). Env: XYTE_PROVIDER=zyte-api or zyte, with ZYTE_API_KEY.

To find out which exit works for which site, measure it: bun scripts/bench.ts cases.json --providers zyte-api,direct,oxylabs reports, per site and provider, how often the real page came back (a captcha served with a 200 counts as a failure), the median latency and the page size.

Errors and retries

extract() resolves with the Zyte 200 body, or rejects with an XyteError carrying Zyte's status, type, title and detail (error.toJSON() is the problem body the server sends).

Status	type	When	Retried by xyte
400	/request/invalid	malformed request (bad url, header syntax), a field xyte doesn't support, an AI field without an extractor	no
400	/request/invalid-json	the body isn't JSON (server)	no
401	/auth/key-not-found	wrong or missing key (server with XYTE_API_KEYS)	no
401, 403	/auth/*	zyteApi() / xyteApi() alone: the upstream's key is wrong or its account suspended, as it answered. In a combinator the next provider gets the request instead	no
421	/website/domain-unreachable	the domain doesn't resolve (direct() only; through a proxy it's a 520)	no
422	/request/unprocessable	incompatible parameters (browserHtml + httpResponseBody, actions on an HTTP request…), or nothing requested	no
500	/server/internal	xyte misconfigured (a provider that can't build its URL, Playwright not installed)	no
520	/download/website-ban	every attempt was a ban	yes, then returned
520	/download/temporary-error	every attempt failed at the proxy or network level	yes, then returned
500	/client/closed	xyte.close() was called while the request was in flight	no

A target's own 4xx/5xx is not an error: it's a 200 with statusCode set, as on Zyte. Bans are 403, 429, 503, or a known challenge page (Cloudflare, DataDome, PerimeterX, Imperva, Akamai); override with isBan. Each retry goes through a fresh session (a new IP), with exponential backoff; onRetry reports them. With zyteApi(), Zyte's own last problem is returned as Zyte gave it. An account problem from Zyte (a bad key, a suspended account) isn't the request's fault: inside a combinator the next provider gets the request and Zyte isn't tried again for it. error.retryable says whether retrying later makes sense, as with Zyte's own clients (true for 429, 500, 503 and 520, except xyte's own misconfiguration 500s).

How Zyte behaviour maps

Zyte	xyte
geolocation	the provider's country parameter; where the exit can't apply it (direct(), a plain proxy URL), the request runs with an X-Xyte-Warning header / onWarning
ipType	byIpType()
session.id	provider sticky session (same exit IP) plus a cookie jar kept across requests
no session	fresh exit per attempt: redirects and browser sub-requests share it, retries change it
sessionContext	a pool of sessions per context, initialized by sessionContextParameters.actions, retired after 4 h or 3 bans
bans	retried on a new exit, then 520 /download/website-ban
target 4xx/5xx	200 with statusCode set
a Cookie custom header	dropped, as Zyte does: cookies go in requestCookies (authenticated pages)
browser rendering	waits for the network to settle after load, so XHR data lands in browserHtml and networkCapture

Checked side by side against api.zyte.com with bun run test:live: see migrating from Zyte.

Supported fields

HTTP: httpResponseBody, httpResponseHeaders, httpRequestMethod, httpRequestBody, httpRequestText, customHttpRequestHeaders, requestHeaders.referer, followRedirect, device, requestCookies, responseCookies, cookieManagement, session, geolocation, ipType, echoData, jobId.

Browser (Playwright, or patchright and other forks, or a remote browser): browserHtml, screenshot + screenshotOptions, javascript, viewport, includeIframes, networkCapture (url / resourceType filters, at most 10 captures and 5 MiB), the 60 s browser time budget, and the actions click, doubleClick, hover, hide, evaluate, goto, reload, keyPress, type, select, scrollTo, scrollBottom, waitForNavigation, waitForRequest, waitForResponse, waitForSelector, waitForTimeout (with onError).

Markdown: built in and non-AI like Zyte's, with mainContent / fullPage scopes.

AI extraction (product, article, jobPosting, serp, customAttributes…): answered by zyteApi(), or through any LLM using Zyte's own output schemas, so responses keep Zyte's shape (extraction):

new Xyte({ extractors: llmExtractors({ complete: openAICompatible({ baseURL, apiKey, model }) }) });

Without either, those fields get a 400.

Sessions: session.id (client-managed: sticky exit + cookie jar) and sessionContext + sessionContextParameters (server-managed pool).

Zyte's own actions (searchKeyword, setLocation, interaction) rely on Zyte's per-site knowledge and hosted scripts. With zyteApi() Zyte runs them; elsewhere they run through handlers in actions (a built-in search-box heuristic for searchKeyword, yours for the rest), which always run locally. Without a handler, unsupportedActions decides: skip with a warning (default), 'error', or 'zyte' to hand them to zyteApi() in a combinator. Where every action runs, provider by provider: actions. Beyond /v1/extract: xyte's server also answers Zyte's proxy mode (--proxy-port) and its Stats API (/api/stats, /api/v1/stats); Zyte's Search API isn't implemented. Field-by-field status: coverage.