Queue URLs

Queues a list of URLs for the specified crawler. Request body must be a JSON array of URL strings.

Prerequisite: the crawler must be manually activated in MyCludo before queued URLs will be indexed. URLs queued against an inactive crawler are accepted by this endpoint (you will get 200 OK with Status: Ok per URL) but they will sit in the queue and never be crawled.

POST /api/v4/{customerId}/crawler/{crawlerId}/paths

Queue URLs

cURL

curl --request POST \
  --url https://api.cludo.com/api/v4/{customerId}/crawler/{crawlerId}/paths \
  --header 'Authorization: Basic <encoded-value>' \
  --header 'Content-Type: application/json' \
  --data '[
  "<string>"
]'

import requests

url = "https://api.cludo.com/api/v4/{customerId}/crawler/{crawlerId}/paths"

payload = ["<string>"]
headers = {
    "Authorization": "Basic <encoded-value>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Basic <encoded-value>', 'Content-Type': 'application/json'},
  body: JSON.stringify(['<string>'])
};

fetch('https://api.cludo.com/api/v4/{customerId}/crawler/{crawlerId}/paths', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

const options = {
  method: 'POST',
  headers: {Authorization: 'Basic <encoded-value>', 'Content-Type': 'application/json'},
  body: JSON.stringify(['<string>'])
};

fetch('https://api.cludo.com/api/v4/{customerId}/crawler/{crawlerId}/paths', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

200

[
  {
    "url": "<string>",
    "status": "<string>"
  }
]

{
  "message": "The request could not be processed. Check the request body and required parameters."
}

{
  "message": "unauthorized"
}

{
  "status": "InternalServerError",
  "code": 500,
  "messages": [
    "Error occur. Please contact support with error ID: a1b2c3d4-e5f6-7890-abcd-ef1234567890"
  ]
}

Authorizations

Authorization · string · header · required

Basic authentication. Use your Customer ID as the username and API Key as the password. Example header: Authorization: Basic MTIzNDU2NzpteS1hcGkta2V5.

Path Parameters

customerId · integer · required

Your Cludo Customer ID (same as in MyCludo and Basic auth username).

crawlerId · integer · required

Crawler (content source) ID from MyCludo.

Body

application/json

Response

200

application/json

URLs queued for crawling.

url · string<uri>

URL that was submitted to the crawl queue.

status · string

Per-URL acceptance code (e.g. Ok when queued successfully).