Free Robots.txt Tester
Paste a robots.txt file, enter a URL path, and choose a user-agent to see whether that path would be allowed or blocked. The tester follows Google's published matching rules: the most specific user-agent group, the longest matching path, and Allow when those paths tie.
This is a checker for text you already have, not a live fetch of /robots.txt and not a Google ranking score. A browser page cannot freely request other websites, so nothing is uploaded and nothing is fetched for you.
robots.txt file
URL path to test
User-agent
Improvement checklist
These reminders can help you review a file. They are not a Google score, and they do not change the result above.
Report
Copy this if you want a text record of the test. This page does not change your site or your Search Console property.
Checks
No items in this filter.
What the tool checks
This page is a tester for robots.txt rules you already have. Paste the file, enter a path or a full URL, and choose a user-agent such as Googlebot, Bingbot, or the * group. The result is either allowed or blocked for that crawler, based on Google's published interpretation of the Robots Exclusion Protocol.
Group selection uses the most specific matching user-agent. A Googlebot group is not combined with the * group. Path matching is case-sensitive. The longest matching rule path wins. If the longest matching rules conflict, Google uses the least restrictive rule, so Allow wins that tie. The wildcards * and $ are supported, and a trailing asterisk is ignored, just as the official examples describe.
The tester also lints the pasted file. It can warn about a missing user-agent, unknown fields, a noindex line (which Google does not support in robots.txt), a crawl-delay line (which Google ignores), a Sitemap line placed between User-agent lines, or a rule that can block the whole site. Sitemap URLs in the file are listed, because those lines are not tied to one crawler.
The input checklist score only says whether the pasted text can be used and whether any cautions apply. It is not a Google score, and it is not a grade of your website. A blocked admin path can still produce a useful, honest result. The score is not asking whether you should have blocked that path. That decision still belongs to you, and to Search Console after the file is live.
How to use it
Open your public robots.txt file in a browser, or copy a draft from your editor. Paste the text into this page. The tool cannot fetch https://example.com/robots.txt for you, because a browser page is not allowed to request another domain freely.
Enter a path such as /blog/post, or paste a full URL. If the live address includes a query string, include it, because that string is part of the match. Then choose a user-agent. Use Custom if you need a name such as Googlebot-News.
Load the sample to see an ordinary small-site file, with /admin/ blocked and a blog path allowed. Load the issue sample to see a sitewide Disallow rule plus unsupported crawl-delay and noindex lines. Those samples are demonstrations, not live reports and not targets. Reset clears the form. Copy report and Download report give you a text record of the path, the user-agent, and the winning rule. Nothing is saved on this page after you leave.
After you are happy with a draft, publish it at the correct host and protocol, then inspect a key URL in Search Console. Google generally caches robots.txt for up to 24 hours, so a change may not be visible at once.
How to read the result
Start with the large result. Allowed means no winning Disallow rule was found. That can happen when a more specific Allow rule applies, when a tie goes to Allow, or when no Allow or Disallow rule matches. Blocked means the most specific matching rule is Disallow. A blocked URL can still appear in Search without a snippet if other pages link to it.
The winning rule card shows the path that decided the result and its length. The checks list explains group selection, lint findings, and official limits. A warning about crawl-delay or noindex does not change the allow or block result by itself; those fields are simply not supported as crawl rules by Google.
Use the improvement checklist as a review, not as a scoring system. Keep needed style and script files allowed. Test the homepage, a normal post, and the exact section you meant to block, for both Googlebot and the * group when both exist in the file.
One grouping detail is worth a second look. Google's parser ignores fields other than user-agent, allow, and disallow when it builds groups. If you place a Sitemap line between two User-agent lines, those agents can be treated as one group. The tester will warn when that pattern appears in the pasted text, and it will apply the same grouping when it decides whether a path is allowed.
Path length is measured from the rule path, after a trailing asterisk is ignored. That is why Allow: /page and Disallow: /*.htm are not a tie: the Disallow path is longer, so it wins for a URL such as /page.htm. When the lengths are the same, as with Allow: /folder and Disallow: /folder, Allow wins because it is the least restrictive rule.
What this tool cannot do
It cannot fetch a live robots.txt file, crawl your site, or log into Search Console. It cannot see whether Google has already cached an older file, and it cannot apply the HTTP status of a live /robots.txt URL. A 5xx response or a missing file is a different problem, described in Google's official documentation, and it has to be checked on the live host.
It also cannot force a URL out of Search results. robots.txt manages crawling. If you need a page kept out of results, use a noindex rule the crawler can fetch, or password-protect the page. Use this tester to read a draft, then confirm the published file in Search Console. URL Inspection can show whether a URL is blocked by robots.txt, and the robots.txt report can show whether Google was able to fetch the file. Those official reports remain the record of what Google has seen, especially after a cache refresh.