Skip to content

Web Unlocker is an advanced scraping API that bypasses sophisticated anti-bot protections such as DataDome, Cloudflare, and PerimeterX.

Notifications You must be signed in to change notification settings

piloterr/webunlocker-api

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 

Repository files navigation

WebUnlocker API

WebUnlocker is an API that bypasses advanced anti-bot protections on approved domains. It blends our Website Rendering and Website Crawler technologies to reliably return the final content of protected pages (Cloudflare, DataDome, PerimeterX, Akamai).

  • Cost: 2 credits per request.
  • Allowlist: the target domain must be allowlisted. Contact us to add a domain.

Useful links: Website WebUnlocker docsStatus pageCreate an account (50 free credits)

Pricing

  • Startup — $249/mo : 110,000 credits → 55,000 requests/mo (CPM ~$4.53)
  • Premium+ — $99/mo : 40,000 credits → 20,000 requests/mo (CPM $4.95)

Example request & response

Real‑world example: the norauto.es website is protected by DataDome. WebUnlocker returned the final content in 4.38 seconds.

Norauto.es example


Endpoint and authentication

  • Method: GET
  • URL: https://piloterr.com/api/v2/website/webunlocker
  • Required header: x-api-key: <YOUR_API_KEY>

Query parameters

Parameter Type Required Default Description
query string (URL) Not required, but recommended Full page URL to fetch (must include http or https).
allow_redirects boolean No true Follow HTTP redirects.
return_page_source boolean No false Response mode.

Notes:

  • If query is missing, the endpoint may return a default example for testing. For production, always provide a URL.

Prerequisites

  1. Create a Piloterr account and obtain your x-api-key.
  2. Request allowlisting for target domains if needed.
  3. Optional: store the key in an environment variable PILOTERR_API_KEY.

Usage examples

Always replace YOUR_API_KEY with your key. You can test with https://example.com.

cURL

curl -G "https://piloterr.com/api/v2/website/webunlocker" \
  -H "x-api-key: YOUR_API_KEY" \
  --data-urlencode "query=https://example.com"

Python (requests)

import os
import requests

API_URL = "https://piloterr.com/api/v2/website/webunlocker"
API_KEY = os.getenv("PILOTERR_API_KEY", "YOUR_API_KEY")

params = {
    "query": "https://example.com"
}

headers = {"x-api-key": API_KEY}

resp = requests.get(API_URL, headers=headers, params=params, timeout=60)
resp.raise_for_status()
print(resp.text)

Node.js (native fetch ≥ 18)

const API_URL = "https://piloterr.com/api/v2/website/webunlocker";
const API_KEY = process.env.PILOTERR_API_KEY || "YOUR_API_KEY";

const params = new URLSearchParams({
  query: "https://example.com"
});

const res = await fetch(`${API_URL}?${params.toString()}`, {
  method: "GET",
  headers: { "x-api-key": API_KEY }
});

if (!res.ok) {
  const errorText = await res.text();
  throw new Error(`HTTP ${res.status}: ${errorText}`);
}

const html = await res.text();
console.log(html);

Node.js (axios)

import axios from "axios";

const API_URL = "https://piloterr.com/api/v2/website/webunlocker";
const API_KEY = process.env.PILOTERR_API_KEY || "YOUR_API_KEY";

const { data: html } = await axios.get(API_URL, {
  headers: { "x-api-key": API_KEY },
  params: {
    query: "https://example.com"
  },
  responseType: "text"
});

console.log(html);

PHP (cURL)

<?php
$apiUrl = 'https://piloterr.com/api/v2/website/webunlocker';
$apiKey = getenv('PILOTERR_API_KEY') ?: 'YOUR_API_KEY';

$query = http_build_query([
  'query' => 'https://example.com'
]);

$ch = curl_init($apiUrl . '?' . $query);
curl_setopt_array($ch, [
  CURLOPT_HTTPHEADER => ["x-api-key: $apiKey"],
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_TIMEOUT => 60
]);

$body = curl_exec($ch);
$status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ($body === false || $status >= 400) {
  throw new Exception('HTTP ' . $status . ': ' . curl_error($ch));
}

echo $body; // HTML

Ruby (Net::HTTP)

require 'uri'
require 'net/http'

api_url = URI('https://piloterr.com/api/v2/website/webunlocker')
params = {
  query: 'https://example.com'
}
api_url.query = URI.encode_www_form(params)

req = Net::HTTP::Get.new(api_url)
req['x-api-key'] = ENV.fetch('PILOTERR_API_KEY', 'YOUR_API_KEY')

http = Net::HTTP.new(api_url.host, api_url.port)
http.use_ssl = true
http.read_timeout = 60

res = http.request(req)
raise "HTTP #{res.code}: #{res.body}" unless res.is_a?(Net::HTTPSuccess)

puts res.body # HTML

Error handling

  • HTTP codes: 4xx (invalid input, unauthorized, domain not allowlisted), 5xx (internal/transient error).
  • JSON error example:
{
    "error": "Bad Request"
}

or

{
    "error": "Internal Error",
}

Tips:

  • Check allowlist status, URL validity, and your API key.
  • Use reasonable timeouts (50–60s) for heavy pages.
  • Implement retry with backoff for transient 5xx errors.

Best practices

  • Pick the right product:
    • Website Crawler: fastest/lowest cost (1 credit), no JS execution; ideal for APIs/static HTML.
    • Website Rendering: executes JS and waits for DOM; higher cost (2 credits).
    • Website WebUnlocker: advanced anti‑bot, approved domains, 2 credits.
  • Compliance: follow site terms and applicable laws.
  • Security: keep x-api-key in environment variables, never hardcode.
  • Observability: log URL, status, latency, and return mode (HTML/JSON).

Support

Need help enabling a domain, troubleshooting an error, or optimizing calls? Reach out via the Piloterr website or our Discord community. We respond quickly to keep your production workflows healthy.

About

Web Unlocker is an advanced scraping API that bypasses sophisticated anti-bot protections such as DataDome, Cloudflare, and PerimeterX.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published