About this plugin
5 it makes up 60% of the training data mix) to crawl websites. Here is the information about the Common Crawl bot (CCBot).
>
txt file to tell Common Crawl bots to not to crawl and index parts of this website.
Please only activate this plugin if you know what you’re doing.
What can I do with this plugin?
txt file that WordPress creates automagically to block the Common Crawl bot (CCBot) is used to train AI algorithms like ChatGPT to crawl websites.
What ideas is this plugin based on?
Our plugin to block the OpenAI ChatGTP bot after the launch of the OpenAI ChatGPT plugins that crawl websites.
What is the robots.txt file?
txt file is a plain text file located at the root folder of a domain (or subdomain) which tells web crawlers (like Googlebot) what parts of the website they should access and index.
txt file and it controls how search engine spiders see and interact with the web pages.
System requirements
4 or greater.
4 or higher because we believe that everybody should be running a modern PHP version. 4 or higher. 4 has no security updates since 28 Nov 2022.
How to get this plugin in your Language!
The first release is avaliable in English and Spanish. In the “languages” folder we have included the necessary files to translate this plugin.
If you would like the plugin in your language and you’re good at translating, please use the native WordPress Translation functionality.
New to Translating a plugin? First read through the Translator Handbook, then select your locale at Translating WordPress and finally go to the translation page for this plugin to translate it.
Further Reading
txt en español.
Contact
For further information please send us an email.