​
API Description

As a versatile solution in our data service suite, the General Scrape API goes beyond e-commerce. It enables public data collection from various websites, including news portals, social media platforms, forums, and corporate sites. With a stable synchronous API, you can fetch real-time data on public sentiment, industry trends, and competitor intelligence. Ideal for market research, brand monitoring, and investment analysis, it offers timely, comprehensive data to support smarter business decisions.
General Scrape API can achieve a success rate of over 90% for regular web page scraping.

​
Request URL

POST https://scrapeapi.pangolinfo.com/api/v1/scrape/batch

​
Request Headers

ParametersParameter valueTypeDescription
Content-Typeapplication/jsonstringData format
AuthorizationBearer xxxstringAuthorization Token

​
Request Parameters

ParametersRequiredTypeDescription
urlsYesstring[]Target URLs
formatYesstringrawHtml, markdown
timeoutNointTimeout time, unit: ms

​
Response value

Return valueTypeDescription
codeintStatus Code
messagestringMessage
dataobjectReturned data

​
Return parameter data description

Return valueTypeDescription
jsonstring[]Return JSON data
urlstringTask URL
taskIdstringTask ID

​
Request example

curl --request POST \
  --url https://scrapeapi.pangolinfo.com/api/v1/scrape/batch \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "urls": [
    "https://www.amazon.com/dp/B0B41YH9B6"
  ],
  "format": "rawHtml",
  "timeout": 40000
}'

​
Return example

{
  "code": 0,
  "message": "ok",
  "data": [
    {
      "status_code": "200",
      "method": "GET",
      "rawHtml": [
        "<html><head>...</head><body><div class=\"content\">Example content</div></body></html>"
      ],
      "url": "https://www.amazon.com/dp/B0B41YH9B6",
      "taskId": "ac6e32c0554244fe865ff2c0379a307e"
    }
  ]
}

​
Credit Consumption

  • Average response time: 5s
  • Credit Consumption:
format parameterCredits Consumption
Original page HTMLrawHtml、markdown1 Credit/Per Request