|

The Crucial SEO Importance of HTTP Headers

ByGreenlane

If you’re not checking your client’s HTTP headers, you’re not giving them good service.

I’m not talking about the stuff in between the <head> and </head> tags, either. I’m talking about the server response that you get before you get all that nice HTML, or that fancy PDF, or whatever else your client’s website is slinging.

That’s because, well, your client’s website isn’t slinging anything. It’s being slung by a server, and the server’s HTTP response is the first thing a web browser – or a web robot like Google’s crawler – will see. There’s information there you may not be properly reviewing.

Implementing important SEO items in the response header is more common than you may think, and most of the major SEO tools don’t check them. HTTP headers are so critical to the SEO efforts that we built a tool to use internally called Head Exam. It checks the HTTP response for SEO factors and lets us bulk check a list of URLs so we can audit a whole site.

Head Exam is so important to our SEO efforts that we’re releasing it as a free tool.

Tool updated 4/25/18, current version 1.16b

Open Greenlane Head Exam

The canonicals in the above screenshot aren’t being pulled from the web page source. Those canonicals are coming from the HTTP header response. Use this tool to check your clients’ headers right now. You might be surprised at what you’ve missed.

[toggle title=”Watch the video walkthrough” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]

[/toggle]

Here’s a scenario – let’s pretend you’re working on a technical audit for a client, and you call out an absence of a canonical tag (which is traditionally found in the source code). You compile your findings in your SEO report and send to your client. The client responds to your recommendation and sets forth with implementing a canonical tag to live in the source code. (Keep in mind, for some clients there might be a cost involved in this implementation.) But what if the HTTP header was already properly displaying a canonical URL request? Google reads this HTTP request and may have already honored it. You could be wasting your client’s time and money, and displaying a degree of your own negligence.

Why HTTP headers are absolutely critical for every SEO

Did you know that you can declare canonical URLs in the HTTP header? Did you know that you can declare canonical tags in the response header and not have to declare them in the HTML code? If not, then I’m glad you’re reading this now before you audit a client’s website and tell them to implement canonicals, and they look at you blankly and say that they have. In the HTTP headers. That you didn’t even check. The bigger the client, the more likely they’ve done this. Like who? Well, whitehouse.gov for example. Just take a look at their HTTP header:

whitehouse-header-canonical

In this snippet of a header (pulled from our Head Exam tool), we can see that the Whitehouse is declaring canonicals, short links and even share images in the header!

This is because the Whitehouse.gov site is built with Drupal, which is used by a full 6% of the Quantcast top 10k sites. In fact, Builtwith is tracking 617,090 live top websites using Drupal right now. Drupal puts canonicals in the HTTP header by default.

OK, but is anyone really putting this stuff in their HTTP headers?

Um, yes. I plopped in a random sampling from Alexa top sites… prepare to have your socks blown off:

sites-using-http-canonicals

Not only do all of these huge websites use canonicals, short links and other SEO elements in the HTTP headers, but two of them have huge glaring mistakes:

– washingtonexaminer.com declares a canonical that’s a 404. Further, the domain newsok.com in the canonical is for The Oklahoman, not the Washinton Examiner.
– radaronline.com has a short link declared that redirects to WordPress. Not their WordPress, but straight up the WordPress.com homepage.

The fact is this: WordPress and Drupal both include a lot of important SEO information in the HTTP headers by default, and together those platforms make up something like 70% of the top sites on the internet according to Quantcast. That makes looking at HTTP headers pretty damn important for you as an SEO.

What else can be hiding in the HTTP header?

Tons of stuff. In addition to canonical tags, some of the most important things for SEO experts are:

  • HREFLANG tags -tells Google which language you are using on a specific page, so the search engine can serve that result to users searching in that language.
  • Shortlinks -a shorter version of a URL you can use to distribute content online.
  • Share image
  • Encoding
  • Language
  • Content length
  • File type

Is your client using one or more of these? Have you ever checked their HTTP response headers? If and when your client calls you out on this, “I didn’t know to check there” isn’t going to cut it.

5-minute HTTP header audit checklist

Use this checklist, along with our Head Exam tool above, to do a quick 5-minute health check of your clients’ HTTP headers. Click the items to expand and see what you should check if an item applies to your site.

[toggle title=”Are there canonical tags in the HTTP header?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]

☑ Are they correct?

☑ Do they contradict any on-page canonicals?[/toggle][toggle title=”Are there hreflang tags in the HTTP header?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]

☑ Is the hreflang tag chain unbroken?

☑ Do they contradict with any on-page hreflang tags?[/toggle][toggle title=”Is there a shortlink in the HTTP header?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]

☑ Is it right?[/toggle][toggle title=”Does the HTTP header declare a share image?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]

☑ Is it formatted correctly?

☑ Does it resolve?[/toggle][toggle title=”Is a language declared?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]

☑ Is it right?

☑ Does it match the language declared in page source?

☑ Does it match hreflang tags (if applicable)?[/toggle]

How to use Head Exam for Bulk HTTP Header Checks

Using head exam is really easy. You have two tabs:

http-tabs

Use the Bulk SEO HTTP header check tab to check HTTP headers for a bulk list of URLs. Use the Single HTTP header examiner tab to take an in-depth look at the full HTTP headers for a URL.

Bulk SEO HTTP header check tab

On this tab, simply paste a list of URLs into Column B, starting at row 6. The tool will pull each URL, split up the HTTP headers, and parse out relevant SEO metrics from the headers into the cells to the right. You can see canonicals, shortlinks, share images, and more.

Be careful not to change any of the other cells, as they will populate automatically.

Single HTTP Header Examiner tab

On this tab, you can see the full header response for a single URL, parsed out with important stuff (like link rel tags) highlighted. Simply enter your URL in cell C4, hit enter, and wait for the magic to happen.

single-http-header-examiner

That’s it! Hope you start checking HTTP headers every time now. Any questions on how to better use Head Exam, tweet the @greenlane_team!

Similar Posts

Using Google Search Console Tools To Clean Up Your (404) Act
| |

Using Google Search Console Tools To Clean Up Your (404) Act

ByBill Sebald

This article is an entire rewrite from its original 2012 version. When Google launched the new Search Console (thus killing the legacy 404 report), it rendered the old version of this post useless. The bad news – the new URL inspector doesn’t work the same as the legacy report. In the previous report, you could…

Step-By-Step Google Disavow Process
| |

Step-By-Step Google Disavow Process

ByBill Sebald

Read Full Article

For one reason or another, plenty of sites are in the doghouse. The dust has settled a bit. Google has gotten more specific about the penalties and warnings through their notifications, and much of the confusion is no longer… as confusing. We’re now in the aftermath – the grass is slowly growing again and the sky is starting to clear. A lot of companies that sold black hat link building work have vanished (and seem to have their phone off the hook). Some companies who sold black hat work are now even charging to remove the links they built for you (we know who you are!). But at the end of the day, if you were snared by Google for willingly – or maybe unknowingly – creating “unnatural links,” the only thing to do is get yourself out of the doghouse.

optimize
|

Optimize NOW For Entities and Relationships

ByBill Sebald

I remember a few years ago blowing the mind of a boss with a theory that Google would eventually rank (in part) based on their own internal understanding of your object. If Wikipedia could know so much about an object, why couldn’t Google? In the end, I was basically describing semantic search and entities, something that has already lived as a concept in the fringe of the mainstream.

How Entities (and Knowledge Cards) Can Help With Query Intent
| |

How Entities (and Knowledge Cards) Can Help With Query Intent

ByBill Sebald

Entity optimization as a big SEO play isn’t quite upon us yet. It’s a slow, growing Google addition. I know – it frustrates me too. So much potential, of which I believe will greatly improve search results in the future. Google isn’t nearly showing the fruits of everything it knows through entities, whether through cards or search results – at least not relative to the way they rank on keywords alone. But can knowledge cards help bring qualified traffic while considering searcher intent? SEOs always talk about searchers intent. Anyone who’s been doing SEO for a while knows that building for intent can be a challenge.

seo report
| |

SEO Reporting: How to Build Meaningful Analytics Reports

ByBill Sebald

The SEO report. It’s a calling card for some agencies. These reports can be ornate or no-frills (everyone has their own style). Smart companies use APIs to compile reports without spending manual hours. Some rely on automatic SEO reporting tools. For other companies, it’s a time-intensive and considerably low-value exercise.

How To Work Relationships and Concepts Into Your Copy
| |

How To Work Relationships and Concepts Into Your Copy

ByBill Sebald

The folks at Mountain View made the conscious decision that keywords alone couldn’t deliver them the results they wanted to see (ahem, “their users wanted to see”). Google tried some different modeling but ultimately came around to semantic search (that is, using semantic technology to refine the query results). Now I said much of the industry has picked up on it. Not all. I still see a lot of pretending Panda, Penguin and Hummingbird never happened. That’s unfortunate for innocent clients around the world. But for most of us probably reading this, we’re students of a new lexicon. With words like “triples” and “entities” and “semiotics” and “topic modeling.”