X

Quick header response to check a list of domains

Using a simple Excel formula and some creative page previewing teamed up with the Link Counter add-on for Firefox can help check header response, server status codes on a list of URLs in no time.

Brian R. Brown
Brian Brown is a Consultant & Natural Search Marketing Strategist for Netconcepts. Brian assists with leading retail clients on their natural search needs, analyzing their sites for creative optimization and link building opportunities to maximize the value of their natural search program. Prior to entering the online world, Brian served in various sales, product management, and new product development roles within divisions of Newell Rubbermaid. He made the dramatic shift from consumer packaged goods with the launch of his own web presence development company, where he served diverse clients, from small startups to large corporate divisions. He brings not only strong SEO skills to client engagements, but a technical background in standards based web design, including table-less XHTML & CSS. Disclosure.
See full bio
Brian R. Brown
3 min read

At Netconcepts, we often work with clients who have portfolios of domains. Some of these may be domains from other businesses or sites that have been acquired that are no longer active, while others are typo and brand protection names, and still others may be used for marketing purposes. These portfolios can range from a handful to hundreds or even thousands.

When kicking off work with a new client, one of the things we look at is their portfolio to see which domains are in use, what other sites they have, and which domains are parked or have redirects in place. We want to establish whether any domains are being used inefficiently. If a domain is returning a 404 Not Found and isn't currently in use, then we'd like to redirect it to a more appropriate destination to capture any traffic or link juice that may be going to the old domain.

What is more likely to be the case though is that the domains are just redirected to the main site. So what we are really interested in is how they are redirected. Many times, these domains are set up with 302 Temporary Redirects. While these redirects will still get the traffic and search engine spiders to the right destination, unfortunately these redirects will not pass along any of the PageRank or link popularity.

Once this has been identified, it is a pretty easy thing for the client's IT group to make sure their domain portfolio is working optimally. As you can imagine though, when working with a portfolio with hundreds or thousands of domains, this can be quite a task. There are individual header checkers like Rex Swain's HTTP Viewer (which is great and there is rarely a day that goes by that I don't find myself there) and Firefox add-ons, but that can still be a task with several URLs. There are some bulk checkers, but even those tend to have limits on how many URLs can be checked at a time.

But here is a quick and easy solution, demonstrated with some of CNET's own domains, that anyone can use to check a ton--maybe even two tons--of URLs using Excel and a simple formula and one of my favorite Firefox add-ons, Link Counter (see that link for an earlier post on using Link Counter and download).

Step 1 - copy and paste the URLs to be checked into Excel.

List of URLs in Excel spreadsheet.

Step 2 - if "http://" wasn't already present for the URLs, place it in a cell by itself.

Step 3 - write out this simple formula (adjusting your cell references if need be):
=HYPERLINK(CONCATENATE($A$1,A1),A1)
*if the URL list already includes the "http://" protocol, then the formula is even simpler:
=HYPERLINK(A1,A1)

Hyperlink formula to create live links.

Step 4 - copy that down for your entire list.

Step 5 - go to the "File" menu and select "Web Page Preview"

Web page preview with live links.

Step 6 - when this opens in Firefox, right-click on Link Counter in the browser status bar and select "Check link status."

Server status overlay using Link Counter.

Step 7 - now would be a good time to do some spot checking on some of the URLs, but otherwise, rejoice in the time that has been saved.

This can also be a way to double check whole lists of domains for canonicalization being in place, similar to the examples used here.