lse123 — 2013-09-08T12:03:01-04:00 — #1
robots.txt can made private file not viewable by humans?
if i have 1st robots.txt in primary domain ggg.com
if folder1domain.com is a secondary domain...
www,ggg.com/folder1/ === www.folder1domain.com, this latter root folder when accessed from www.folder1domain.com, should needed another robots.txt ???
mittineague — 2013-09-08T15:11:28-04:00 — #2
The robots.txt file provides no security and in fact may lessen it.
What it does is tell honest search engines your preferences.
Granted, "Allow" and "Disallow" are misleading and better words could have been used (ignore?) but for better or worse that's what they are.
You may find that "dishonest" search engines (or humans) use the file to find out what you don't want them to see. (eg. if you set up a "honeypot").
If you really want to protect something either don't put it online, or if you must, use some other method like password protection.
lse123 — 2013-09-08T15:51:35-04:00 — #3
thks for the info...can you reply and second Q... in other words each hosted domain (each subfolder of primary domain) needs a robot.txt, in case use it... to really do its partly job?
yes correct. / no just primary domain root folder needs it only.
mittineague — 2013-09-08T16:15:41-04:00 — #4