Search

Robots txt Generator Tool

Allow or disallow search engines bots to crawl your web pages with the Robots txt generator tool.

 

Robots txt Generator

Allow or disallow search engines spiders to crawl your site.

Use the Robots txt tool to generate a simple robots.txt codes for your website. These codes will enable you to to allow or disallow search engines bots/spiders from crawling certain pages within your website. You can check your robots.txt file with our Robot Syntax Checker tool.

Tell robots to ignore this page
Tell robots to not index this page in the search engines
Tell robots to not follow any of the links on this page

Insert this meta tag in the <head> section of your Web page:


[Netscape, Firefox, and Opera users - Press button to select and [Ctrl]+C to copy]


Tell only the Googlebot to ignore this page

Insert this meta tag in the <head> section of your Web page:


[Netscape, Firefox, and Opera users - Press button to select and [Ctrl]+C to copy]

Support our Webmaster Tools

Our webmaster tools are completely free to use however, they require time & money to maintain.
Each query uses our server's resource so feel free to donate towards our free services.
You can donate in two ways:

A- by donating any amount of money using Paypal or
B- by adding a small link or banner to your website.
Please support us by linking to our website

Please note: Donations will help our team to develop new tools & keep existing tools running efficiently.