Aug 16

Google introduced X-Robots-Tag directive

Tag: SEOWebnauts @ 2:07 am

Google introduced the new X-Robots-Tag directive, which adds Robots Exclusion Protocol (REP) META tag support for non-HTML pages, like PDF, Videos, Spreadsheet files, etc. You can have the same control over such indexed file types.

X-Robots-Tag: unavailable_after: 31 Dec 2007 23:59:59 EST

Also be aware that REP META tags can be useful for implementing noarchive, nosnippet, and now unavailable_after tags for page-level instruction, as opposed to robots.txt, which is controlled at the domain root.

Bookmark button - New window opens

Leave a Reply

You must be logged in to post a comment.



Bad Behavior has blocked 179 access attempts in the last 7 days.