SaaS Side: Crawler rate limitation -- Squirrly SEO. Improved crawling success on all hosting platforms

  • Focus Pages
  • SEO Audits section
  • Inner Links + Automated Inner Links

These tools inside of Squirrly SEO run from our Cloud Services and they crawl the website just like Google and others do.

We've had a lot of tests made to find the sweet spot on how many crawls / second we need to do, to make sure that our crawlers will not trigger the hosting platforms to temporarily block our crawling IPs.

We've seen plenty of success with the new rates during November and the early part of December 2025.