Disallow directory from search engines without writing fullname

Server Config
1

hi

i want to disallow a particular directory from search engines.


User-agent: *
Disallow: /folder1/

User-Agent: Googlebot
Disallow: /folder2/

The above code works but i dont want to mention the full name of the directory in robots.txt

what should i do

vineet

2

It is not possible to block any kind of directory without their original name Hence if you want to block particular directory then it is require to mention full name or path. Check below details you will get more ideas about robots.txt : http://www.robotstxt.org/robotstxt.html

3

vin,

It is possible:

  1. Do not link to the directory. Security by obfuscation is little security but SE’s don’t try to find non-linked pages.

  2. As KJ said, you must list the directory in robots.txt to tell honest SE’s they’re not allowed to follow links to that directory. Bad SE’s, however, will merely use that information to take them into the directories you want to protect.

  3. If you must link, then use password protection on the directory. That’ll stop SE’s in their tracks!

  4. If none of those suit you, create a form whose action is to take visitors to the forbidden land via submit. SE’s won’t be searching for that as a link so it may work to hide the link.

Think outside the box: robots.txt is next to worthless so be creative with other solutions.

Regards,

DK