Create Batch

Starts a new batch. You receive an id that you can use to track the progress of the batch as shown here. Note: Processing time is constant regardless of batch size

POST /v1/batches

Start a new batch

cURL

curl --request POST \
  --url https://api.olostep.com/v1/batches \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "items": [
    {
      "custom_id": "<string>",
      "url": "<string>"
    }
  ],
  "country": "<string>",
  "parser": {
    "id": "<string>"
  },
  "links_on_page": {
    "absolute_links": true,
    "include_links": [
      "<string>"
    ],
    "exclude_links": [
      "<string>"
    ]
  },
  "metadata": {
    "order_id": "12345",
    "customer_name": "John Doe",
    "priority": "high",
    "processed": "true"
  },
  "webhook": "<string>"
}
'

200

{
  "id": "<string>",
  "object": "<string>",
  "status": "<string>",
  "created": 123,
  "total_urls": 123,
  "completed_urls": 123,
  "parser": "<string>",
  "country": "<string>",
  "metadata": {
    "order_id": "12345",
    "customer_name": "John Doe",
    "priority": "high",
    "processed": "true"
  },
  "webhook": "<string>"
}

Get notified on completion: Pass the webhook parameter with your endpoint URL to receive an HTTP POST when the batch completes. See Webhooks for details.

Authorizations

Authorization · string · header · required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

items · object[] · required

Array of items to be processed in the batch.

Show child attributes

country · string

Country for the batch execution. Provide in ISO 3166-1 alpha-2 codes like US(USA), IN(India), etc

parser · object

You can use this parameter to specify the parser to use. Parsers are useful to extract structured content from web pages. Olostep has a few parsers built in for most common web pages, and you can also create your own parsers.

Show child attributes

links_on_page · object

Get all the links present on each page in the batch.

Show child attributes

metadata · object

Set of key-value pairs for storing additional information about an object. Follows Stripe's approach with validation rules: max 50 keys, key max 40 characters (no square brackets), value max 500 characters, all values stored as strings.

Show child attributes

Example:

{
  "order_id": "12345",
  "customer_name": "John Doe",
  "priority": "high",
  "processed": "true"
}

webhook · string<uri>

HTTPS URL to receive a POST request when the batch completes. Must be a publicly accessible URL using http:// or https:// protocol. Cannot point to localhost or private IP addresses. See Webhooks for payload format and retry behavior.

Response

200

application/json

Batch started successfully.

id · string

Batch ID

object · string

The kind of object. "batch" for this endpoint.

status · string

in_progress or completed

created · number

Created epoch

total_urls · number

Count of URLs in the batch

completed_urls · number

Count of completed URLs

parser · string

country · string

metadata · object

Set of key-value pairs for storing additional information about an object. Follows Stripe's approach with validation rules: max 50 keys, key max 40 characters (no square brackets), value max 500 characters, all values stored as strings.

Show child attributes

Example:

{
  "order_id": "12345",
  "customer_name": "John Doe",
  "priority": "high",
  "processed": "true"
}

webhook · string

Webhook URL to receive completion notification