Googlebot cannot access my WordPress robots.txt

2 Views

I have some troubles with Google indexing on my WordPress site.
When I added my site to Google webmaster the .htaccess was the following 

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

Now, I tried to add these lines to .htaccess file, but Google still does not index my site.

Read More
# BOT SETTINGS
SetEnvIfNoCase User-Agent .*google.* search_robot
SetEnvIfNoCase User-Agent .*yahoo.* search_robot
SetEnvIfNoCase User-Agent .*bot.* search_robot
SetEnvIfNoCase User-Agent .*ask.* search_robot

Order Deny,Allow
#Deny from All
Allow from env=search_robot

Where am I doing wrong?

Post Views: 2

Related posts

1 comment

  1. Create robots.txt file and upload your wordpress root folder with following code.

    User-agent: google
Disallow:

User-agent: yahoo
Disallow:

User-agent: bot
Disallow:

User-agent: ask
Disallow:

User-agent: *
Disallow: /

    And try to remove #BOT SETTING in your .htaccess.

Comments are closed.