Crawl Crawler BETA


In this document are the terms we need to agree upon before you use this, the service.

§-1 Cookies servers will not place a cookie on your computer, nor several. If you've cookies on your drive, rest assured they are not from us.

§0 Tracking

We will not track you. We don't use cookies, we don't even use javascript. Having said that, we could still track you server-side. But we don't.

§1 Logging servers will store IP addresses with timestamps and the URL for the requested resource for all users for as long as 30 days, in order to ensure a secure service with high availability.

We need this privilege, to store IP addresses, to be able to add them to our blacklist when actors misuse this service. IP addresses in our blacklist will be stored for up to 183 days.

Log files are not shared, not even among our friends.

§2 Data hosting

Apart from logging IP addresses we will store (host) indices of textual data and collections of document text extracts that originates from Common Crawl and the public WWW. When fetching data from WWW we do our very best to interpret and honor your robots.txt.

No indices are private indices and no document collections are private collections. Instead, all indices are public indices and all document collections are public collections.

Consequently, all Crawl Crawler's data, except our log files, is public.

§3 Data removal

Please go ahead and contact us in matters of data removal.