How
Can I Prevent Search Engines From Indexing a Web Page?
FAQ
#4:
Q:
How
can I prevent search engines from indexing a web page?
A:
If the information
on your web page is sensitive or proprietary in nature,
you may want to password protect it. Search engines can
not index password protected pages.
Another way to prevent search engines
from indexing a web page, is to instruct the robot not
to. You can do this by creating a robot.txt file. Robots
that comply with the “Robots
Exclusion Standard” will read the commands in this
file and obey them. The primary purpose of a robots.txt
file is to direct spiders to ignore documents or directories
/ folders that may contain private or unnecessary information.
Robot.txt files are also sometimes used when creating search
engines specific pages.
