JittatorBot

If you see this user agent in your logs:

JittatorBot/0.1 (+https://jittator.com/bot)

it is the browser behind Jittator, a paid API that takes screenshots and PDFs of web pages. It is not a crawler. It loads a page only when a customer of the API asks for that specific URL, renders it once, sends the result to that customer and does not keep it. It does not follow links or index anything.

What it does and does not do

• Fetches only public http and https URLs on ports 80 and 443.
• Reads /robots.txt before loading a requested page and stops if the path is disallowed for it.
• Loads the page the way a normal browser does, including its images, styles and scripts. It may also load such files from your site when HTML a customer sends to the API refers to them. Every one of those requests uses the same user agent.
• Sends no cookies, logins or other credentials.
• Does not solve CAPTCHAs or try to get around anti-bot systems, paywalls or blocks. If your site answers it with an error status (4xx or 5xx), the customer gets an error message, not a picture of your page. A block page served with status 200 is rendered like any other page.

robots.txt

JittatorBot follows RFC 9309. Its product token is JittatorBot. If your robots.txt has a group for JittatorBot, that group applies; otherwise the * group applies. Allow, Disallow, * and $ are understood, and the longest matching rule wins.

The check is made on the URL the customer asked for, before the page is loaded. Images, styles and other files the page loads are not checked separately. A missing robots.txt, or one that cannot be fetched or answers with an error, is treated as allowing everything. Your robots.txt is cached for up to 10 minutes per site (1 minute if it could not be fetched), so a change takes effect within that time.

Block JittatorBot from your whole site

User-agent: JittatorBot
Disallow: /

Block it from part of your site

User-agent: JittatorBot
Disallow: /private/
Disallow: /drafts/

Blocking by user agent at your web server or firewall also works. If you answer with an error status, the API reports it to its customer as an error; anything you serve with status 200 is rendered as it is.

Contact

To report a problem with JittatorBot or abuse of the service, use the contact form. The rules customers agree to are in the acceptable use policy.