If Googlebot is crawling your website too often and causing performance issues, you may be able to request a temporary reduction in crawl rate. This option is especially useful if your website is hosted on a shared or limited server that cannot handle frequent requests.
In this blog, we will explain what crawl rate means, why you might need to reduce it, and how to submit an exceptional request using Google Search Console.
What is Crawl Rate?
Crawl rate refers to the number of requests per second that Googlebot makes to your website while crawling its pages. Google uses this crawl activity to index and update your content in search results.
Most websites do not face any issues with Googlebot’s crawling. In fact, Google automatically adjusts the crawl rate based on your site’s server capacity and response time. However, in rare situations, Googlebot may crawl your site too frequently, causing it to slow down or crash.
Why You May Need to Reduce Crawl Rate
There are a few reasons why you may want to limit how often Googlebot accesses your site:
- Your server is under stress due to frequent crawl requests.
- You are using shared hosting with limited bandwidth.
- Your website becomes slow or unresponsive when Googlebot visits.
- You cannot configure your server to return proper 503 (Service Unavailable) or 429 (Too Many Requests) error codes during heavy load.
In such situations, you can request Google to reduce the crawl rate through a special form. This is known as an exceptional request.
Important Limitations
Before submitting a request, it’s important to understand what Google allows:
You can:
- Request a temporary decrease in crawl rate if it is affecting your server.
You cannot:
- Request an increase in crawl rate.
- Get permanent control over crawl speed.
- Expect real-time crawl rate changes.
Google processes these requests manually, and it may take a few days for changes to take effect.
When Should You Submit a Request?
You should only submit this request if all the following apply:
- Googlebot is crawling your site too often.
- The crawling activity is causing real performance problems.
- You are unable to serve proper error codes like 503 or 429 to signal overload.
- You have already tried using robots.txt or other standard methods without success.
If your site meets these conditions, a manual request may be the best option.
How to Submit an Exceptional Request to Reduce Crawl Rate
Here is a step-by-step guide to filing a request through Google Search Console:
Step 1: Open the Report Form
Go to this official Google form:
https://search.google.com/search-console/googlebot-report
Step 2: Select the Issue
Choose the option that says “Google is crawling my site too quickly”.
Step 3: Enter Website Details
Provide your website URL in full format, such as https://www.example.com.
Step 4: Explain the Issue
Write a short and clear explanation of the problem. Be specific about:
- The issue is caused by Googlebot’s frequent visits.
- When the issue started.
- Any steps you have already taken?
- The crawl rate you consider acceptable (e.g., 1 request per second).
Example explanation:
“Our site is hosted on a shared server and has been experiencing slow load times due to heavy Googlebot activity. We cannot configure our server to return 503 errors. We request to reduce the crawl rate to one request per second for the time being.”
Step 5: Submit the Request
After filling out the form, click the Submit button. Google will review your request and make a decision. If approved, the crawl rate will be temporarily reduced. This may take a few days to take effect.
Other Ways to Control Crawl Rate
Before submitting a request, you may want to try the following alternatives:
- Use robots.txt to block non-essential pages from being crawled.
- Reduce duplicate or thin content that triggers unnecessary crawling.
- Use server configurations to return
503or429error codes during overload. - Monitor crawl logs to identify and fix crawl issues.
- Use efficient caching and server optimization techniques.
These methods are recommended because they give you more control without requiring manual intervention from Google.
How Long Does the Reduction Last?
If your request is approved, Google will reduce the crawl rate for a limited time. The duration may vary depending on your case. Eventually, Googlebot will return to its normal crawling behaviour unless new problems occur.
Keep monitoring your website’s performance using server logs and Google Search Console.
Conclusion
Googlebot is designed to crawl your website in a way that does not overload your server. However, in certain cases, it may crawl too aggressively, especially on low-resource websites. If standard tools like robots.txt and server error responses are not an option for you, submitting an exceptional request to reduce crawl rate is a reliable solution.
To submit a request, use the official Googlebot report form here:
https://search.google.com/search-console/googlebot-report
Always try to resolve crawl-related issues using built-in tools first. But if your site’s performance is affected and you need help, the special request form is available for this exact purpose.