Hello,
I’m searching many source and saw this code:
RewriteCond %{HTTP_USER_AGENT} !(googlebot-image|msnbot|psbot|yahoo-mmcrawler) [NC]
Do anyone can confirm for me that this code is correct that allow Google Images… can crawl my images?
Thanks.
I do not see why you would need it at all. I do not have it and Google has index some of my images.
Try images:websitename in google and see what you get.
u_c,
Without the rest of your mod_rewrite (concerning the bots), there’s no way to tell. If you’re blocking bots, just omit the bot you want to catalog your website.
Note: Good bots will honor robots.txt directives but bad bots won’t. If you’re using mod_rewrite to block bots, many/most of the bad ones include the ability to change their bot identification and become whatever “browser” you will admit to your website. IMHO, it is a losing proposition for the webmaster to try to block bots as it’s just too easy to get around blocks with the tools available.
Regards,
DK