← All text tools

X-Robots-Tag Header Generator

Build X-Robots-Tag headers and ready Apache, Nginx and PHP rules that noindex PDFs, documents, images and other files that cannot carry a robots meta tag.

  • Free
  • No sign-up
  • Works on any device

About X-Robots-Tag Header Generator

The robots meta tag only works inside HTML, so a PDF, a spreadsheet or an image cannot use it. The X-Robots-Tag HTTP header does the same job for any file, because the server sends it with the response. This tool builds the header value from directives such as noindex, nofollow, noarchive, max-snippet and unavailable_after, can limit it to one crawler, and writes the matching Apache FilesMatch block, Nginx location block and PHP call for the file extensions you list.

How to use X-Robots-Tag Header Generator

  1. List the file extensions to cover, such as pdf and docx, and tick the directives you want.
  2. Optionally name a single crawler or add an unavailable_after date or snippet limits.
  3. Copy the Apache, Nginx or PHP rule into your server setup, then test a file with a header checker.

Why use this tool?

Also known as

Frequently asked questions

Why must these files not be blocked in robots.txt?

A crawler that is blocked never requests the file, so it never sees the header. The address can then stay in results without a snippet.

Do I need mod_headers on Apache?

Yes, the Header directive comes from mod_headers. Most hosts enable it, and you can check with apachectl -M.

Is noindex enough to remove a file that is already listed?

It is removed the next time the crawler visits the file and reads the header. Search Console can speed this up.

Is X-Robots-Tag Header Generator free?

Yes. X-Robots-Tag Header Generator on Toolipedia is free to use, with no sign-up and no watermark.

Is it safe to use?

Yes. Everything is worked out in your browser, and nothing you enter is sent to our server.

Does it work on my phone?

Yes. It works in any modern browser on Android and iPhone, as well as on tablets and computers. There's nothing to install.

Do I need an account?

No. Just open the page and use the tool. There's no sign-up and no email needed.

Can I put X-Robots-Tag Header Generator on my own website?

Yes. Use the Embed on your site button under the tool to copy a short piece of code and paste it into your page. It is free, needs no account, and the tool keeps working inside your page.

Related tools