Category Archives: Keyword Research

Competitor Research

The final step of keyword research and getting started on SEO is to do some competitor research. Here you want to know who your online competition is, how authoritative their sites and pages are, and how well optimized their sites and pages are.

This information will tell you how difficult it will be rank for given keywords, and could even change your SEO strategy: if your competition is much stronger than you had thought, you might switch to a long tail strategy and not worry about your head terms at the beginning, or you might set some keywords aside to work on after your site has grown in authority.

There are a lot of tools out there to help you with competitor research: some of them have value, and others are just a waste of your time. Happily, the very best method of evaluating competition is completely free: just look at the SERPs yourself.

Who is your competition?

When I was working in an agency just about every other client I had completely misunderstood who their competition was. These guys were generally brick & mortal businesses: retail or service or manufacturing. When we asked about their competition they inevitably listed the guys down the street or the big players in their industry. But most of the time these were not their online competition.

If someone is searching on Google for your products, service, or information, then those people clearly don’t already know where to go. To them, the only options are the sites that come up on the top of the search results. Those sites are your competition.

So when you’re scouting the competition, what you’re really looking for is the sites that rank in the top 5-10 results for the searches that you’re going after.

Finding Your Real Competitors

A good way of evaluating your competition is to sit down with your list of keywords that you developed in the keyword research phase, and search in Google for each of those keywords. Make sure you have personalization turned off for this (how to do that is explained on xxx).

For each keyword you’re targeting, make a note of what sites are in the top five position (or top 10 if you’re being very thorough). Remember we’re only looking at organic results here, and not paid ads.

Once you’ve gone through all your keywords, now take a look and see what sites appear most often on your list. Those top five or ten sites are your online competition. Those are the guys that you’re trying to outrank.

Evaluating the Competition

Now go back to some of your keywords and go visit the sites that are ranking in the top three positions. Check out the page that’s ranking and see how well optimized the page is: does it have good title tags, URL, keyword use on the page, and how long is the article or page? If the page is fairly weak in on-page optimization, that’s great news — this indicates that you can make a far better page and easily get better topicality signals, which means you will need lower authority signals to outrank that page.

The next step is to evaluate the authority of the site as a whole, and the page that’s ranking in specific. To do this you have to use some tools (which you generally have to pay for with a subscription) that evaluate the links to a page or site. The simplest route is to measure the Domain Authority of the page and the site’s homepage. Better answers can come from evaluating the number of linking domains pointing to the page,. We’ll talk about what tools are available in xxx.

Evaluating Page Authority

Evaluating the authority of the ranking pages is the most important part of competitor research. Unless they have very weak on-page optimization, you can generally assume that you will need authority signals at least as strong as theirs to have a chance of outranking them.

If you come across a page that’s Page Authority 50 with 200 linking domains pointing at the target page… that page is not going to be something you’re likely to outrank with a new site, not for a long time. It is going to take a lot of time to build up hundreds of linking domains to a specific interior page (there are shortcuts, of course, but these are generally against the rules and are likely to get a site penalized).

Evaluating Domain Authority

The overall authority of competing domains is also going to give you an idea of how much authority your site is going to need to do well in your niche. Take all of the top sites on your competitor list and check out the number of linking domains to their entire site. While you’re at it, spot check their linking domains to make sure most of them are legitimate links (many sites have a large number of garbage links that aren’t really helping them much — links from directories, scrapers, press releases, or article sites). In general, your SEO goal is going to be to get more linking domains than your competition has, and to get those links from sites that are at least as authoritative as the sites linking to the competition.

If your competition are all Domain Authority 20 or 30 sites with a couple hundred linking domains — good news! You should have no problem building something more authoritative over the course of a year.

If the competition are at Domain Authority 40 or 50 and have 800 or more linking domains, then you’re going to have a lot of work. It will likely take a couple years to build up to that level, and you’re going to have to invest significant time, and likely some money, into marketing and outreach.

If your competition is at Domain Authority 60+ and has thousands of linking domains, then they are a truly large site with substantial authority. You’re going to need to be an equally large site with similar reach and marketing budgets, or you’re going to have to have something go viral, and you’re going to have to be smarter than them at SEO.

Page Authority Can Beat Massive Domain Authority

Keep in mind, however, that you’re comparing the Page Authority or linking domains of the page that is ranking (not the entire site) against the page on your site that you want to rank. Sure, Amazon is an Authority behemoth, but I’ve worked on dozens of sites that have moved up to outrank Amazon.com for their searches. While Amazon has massive authority (and often pretty darned good topicality), the specific page that you’re competing against may not.

I’ve see Domain Authority 20 sites climb up and pass Amazon.com rankings — but against pages on Amazon.com that only have a few or no linking domains pointing to the page.

The big lesson to take home from your competitor research is that you should come out of it with a realistic expectation of how much work and time it will take for you to start ranking well in your niche.

The good news is that it’s not an all or nothing game.You don’t have to outrank Amazon to start driving SEO traffic.

As you start optimizing your site and building your authority, you’ll slowly start gaining in long tail, and then torso rankings. Long before you’re able to compete for your head terms you’ll be getting good amounts of search engine traffic from all the rest of your terms – and always remember that those long tail terms represent far more traffic than the big intimidating head terms.

Ranking the Long Tail

So we know we care about the long tail keywords because we know they make up around 75% of all searches. The other big advantage of the long tail is that it’s often easier to rank for long tail searches.

If I started a site today selling custom-built laptops, I would have no practical likelihood of ranking for the head term “laptops.” Instead a smart SEO strategy would be to include a lot of very thorough articles about how to customize laptops, and detailed reviews of different machines or components — in other words I’m creating the kind of content that places like NewEgg and BestBuy.com don’t have, which gives me the opportunity to capture long tail searches that the big guys either aren’t ranking for, or aren’t very topically relevant for.

Optimizing for the long tail is a bit more subtle, since we often don’t even know what terms we’re specifically trying to capture. The key to long tail optimization is to have a lot of very good content (and by content, I mean word count).

More Content for More (Long Tail) Keywords

Sites targeting only head terms often have only a paragraph of text that stressing their head term keywords and, frankly, doesn’t really provide much information.

Long tail optimization stresses having much longer text that provides truly useful information that people are looking for, and answering questions that people might have. The more quality content you have, the better the odds that you struck some vital word combination that someone might search for.

Long articles that aren’t truly useful, or good, tend not to do very well with long tail, because by the very nature of bloating your word count with fluff, you aren’t saying things that people are really looking for (indeed, these types of articles aren’t really saying anything at all). No one is searching for your stream of consciousness ramble about different types of laptops. By focusing on providing real information that real users are looking for, you’ll end up hitting lots of things people are, or will be, searching for.

Rank the Head to Rank the Tail

The other part of long tail optimization is simply to rank for the head term. A site that ranks for the head terms is far more likely to rank for the long tail, as long as it’s topically relevant to the search query.

Thus doing all the things you can to build authority and rank for the head term — even if you know you have no chance of ranking for it — will increase your ability to rank for all the related long tail terms.

In my laptop example above: while the new site can write some awesome articles to capture long tail traffic; if BestBuy or NewEgg had the same articles, they would outrank the new site due to their overwhelming authority.

Happily giant sites (or rather, giant corporations) tend to be really bad about providing lots of great content — or indeed much of it at all. My theory is that once they get really big, they get the idea that it’s not worth pursuing things that take lots of man-hours, but instead only want to do something they can scale across their massive site. As a result they leave lots of scraps around for the little guys — and a smart site owner can use those scraps to grow their brand to the point where it can compete with the big guys.

The Head, the Tail, and Everything in Between

In the world of SEO you’ll hear a lot about Head Terms and the Long Tail. Sometimes you’ll hear talk about the in between, called torso terms, and sometimes called the chubby middle (fat head, chubby middle, long tail).

The concept of head and tail terms is very important for SEOs to understand, as it has huge implications for how best to optimize a site.

  • Head Terms: the keywords that drive a hugely disproportionate amount of traffic. “iphone” is a head term with over a million searches per month, while “how to get my iphone to flush down a toilet” is a long tail term that maybe has one search a month. There is no hard definition of what makes a head term, other than they are the very top 5-10% of your keywords. How much search volume they have depends on your industry: for BestBuy.com head terms might have a search volume of over 100,000 per month. AwesomeDice.com’s head terms have a search volume of over 500 searches per month.
  • Long tail: These are the keywords with very, very little search volume. If you looked up the search volume for these keywords Google would report < 10 or 0 searches for most of them. In most sites these are keywords that drive no more than 5 visits a month, and often only 1 or 2 visits a year.
  • Torso Terms: These are just everything in between. Torso terms are usually those keywords that would be a head term to someone smaller than you. At BestBuy.com or Wayfair.com there were thousands of keywords with a few hundred searches per month that weren’t worth my time to chase down, but that might be worthwhile for a smaller niche site to focus on (since they don’t have a chance at what I considered head terms).

When people do their keyword research, they are usually focusing on head terms — and this makes perfect sense. After all, why would you spend time researching terms that show up as having no search volume at all?

But here’s the thing: The long tail is larger than all of the head terms combined. Much larger.

For most sites, the long tail represents up to 75% of the traffic — all that from keywords that are driving only a few visits per month. While the head terms drive huge amounts of visits, the long tail overcomes the head by having a massive number of different keywords — many of which are slight variations.

Head Terms vs Long Tail Keywords

Head terms drive big chunks of traffic alone — but this graph is truncated at the end. If you continued it to include every keyword it would stretch to the right for page after page. The total surface area of the entire tail is around four times the surface area of the head.

The reason for this is that everyone searches differently, and more and more people are searching Google for increasingly specific things. Instead of searching for “iphone” people might be searching for “where can i buy an iphone that my kids won’t destroy in a week.” The first is a head term; the second is a long tail search query.

In fact, Google reports that a whopping 16% – 20% of all searches made each year are for a keyword phrase that has never been searched before. That means nearly a quarter of all searches made will never show up in any keyword tool… because they haven’t been searched for yet.

The long tail is very important. Happily, pursuing the head is one part of a strategy to pursue that long tail.

Keyword Mapping

Now that you have a list of keywords that you want to target on your site, the next step is to map those keywords to specific pages. Just take your spreadsheet of keywords and in a column next to them enter the page that will be trying to rank for that keyword.

Here are some rules of thumb for keyword mapping:

  • One page per keyword: you should have one page that is trying to rank for any given keyword. You do not want two unrelated keywords targeted on the same page.
  • Related keywords on the same page: logically related keywords should all be targeted on the same page. This includes plural versions and synonyms. Don’t have one page to target “armoires” and a different page to target “wardrobes.” They’re the same thing and it doesn’t make sense to have different pages for each.
  • One topic per page: while you want to cluster related keywords together, each page should be entirely about one topic. If you’re targeting Snowflake, Snowflakes, Custom Snowflakes, and Artisan Snowflakes on the same page (as you should) then that page should be all about custom artisan snowflakes. It shouldn’t just have one paragraph about them! The way that a lot of companies fall into this bad habit is by having one page that lists all of their services and then tries to rank for each of those services on that page. It’s fine to have that page, but it should link to a separate page on each of the services.

Keyword mapping is a useful exercise because it tells you what kind of pages your site needs to have, and it can also help inform how your site should be structured.

For example: when I was first building AwesomeDice.com I did keyword research. I discovered that some people searched for dice by the game the dice were used for, while others searched for the color of dice, and others searched more generic terms like “dice sets” or “gaming dice.”

This told me not only that I wanted a page to target each of these terms, but it also told me that I should structure my site so that users could look for their dice by the kind of set, the color, or the game the dice were for. The keyword research informed the structure of the site to make it easier for users to find what they’re looking for. So this was not only good for SEO purposes, but also made by site better for users.

Common Mistakes

I see two common mistakes with keyword mapping made often by beginners with only tenuous SEO knowledge:

  • Too many unrelated keywords per page: a computer manufacturer says “I want this page to rank for RAM, processors, and power supplies.” Those are three different topics that should each have their own page. Each of those keywords has a different intent, and a customer searching for “RAM” will need different information on the page from a customer searching for “power supply.” Making one giant page with huge sections on each of the three yields a crappy user experience for everyone, and poor topicality targeting since the page is kind of all over the place.
  • Making different pages for similar keywords: this is the exact opposite problem as the above. Someone wants to have one page for “garden hose” and another for “yard hose,” even though they are the same thing and served by the same products. In this case the intent of the user is the same, and the user will need the same information regardless of which she searches for. There should just be one page — even if that means that you’ll rank slightly worse for one of the phrases (which is true right now, though Google is making big strides in getting better at understanding searcher intent, rather than just the keywords entered).

For the most part when people get their keyword mapping truly horribly wrong it’s because they’re thinking too much about SEO targeting, and not enough about what makes sense to the user. If you keep the mindset that you want a deeply informative, high quality site that is so great to use that it’s clearly better than any other site on the internet, it will be easy to avoid creating too many pointless pages or trying to smash keywords into pages that aren’t focused on those terms.

Other Keyword Research Tools

When you’ve finished mining both your brain and Google Keyword Planner, you can either decide that you have what you need, or you can go to other tools to try to expand your keyword list further. There are a variety of tools out there that will try to suggest related keywords for you — the idea is that you go to the tool to generate more keywords, then take those keywords back to the Google Keyword Planner to see whether there’s any significant search volume.

Here are a couple of ones I’ve used and like:

Ubersuggest

One of the tools that I like is Ubersuggest. When you type a search query into Google, the search engine will try to predict what you’re searching for and gives several auto-complete options below the search box (Google calls this Instant Suggest). Those suggestions are based largely on what other people have searched for.

Ubersuggest

Ubersuggest mines those Google suggestions to find possible related keywords. When you enter a word, like “headboards,” Ubersuggest then gets all the Google instant suggestions. Then it adds the letter A — “headboards a” and gets all the suggestions based on that. Then it adds B, then C, and so on.

With this you can get a nicely expanded list of potential keywords, which you can then run through the Google Keyword Planner to get the search volumes (and possibly additional suggested keywords).

Keyword Explorer

Published by the SEO software company with the best reputation in the business, Keyword Explorer is a relatively new tool that aims to be a one-stop shop for keyword research. It combines tools like Ubersuggest with the Keyword Planner suggestions with its crawls of the web to find keywords that commonly occur along side the keyword you’re searching for on pages that rank for that term, and then gives you an estimated search volume for each term.

Moz Keyword Explorer

Like all tools, it’s going to give you some weird unrelated keywords, but it’s very nice and does simplify keyword research once you know how to use it. It is, however, part of Moz’s paid suite of tools, so you’ll need to pay a monthly fee for access. That said, many SEOs already consider it necessary to pay for a Moz subscription for access to their many other tools (which we’ll talk about much later).

Google Trends

Another useful tool to be aware of is Google Trends. Google Trends will show you the search volume for a given query over time. With it you can see whether searches for a given term are generally increasing or decreasing, and more importantly you can see the seasonality of terms.

Google Trends

You can tell if it spikes in the summer or winter every year, which helps set your expectations and sometimes can put a deadline on some of your SEO efforts. Just be careful not to get too obsessed with it or read too much into it: it’s somewhat useful, but should not determine which keywords you target. Instead use it to determine when you target those keywords (a couple of months before the seasonal spikes).

Google Keyword Planner

Now that you have your starting list of keywords, we’re ready to start using Google’s Keyword Planner. This is a tool that Google developed to help Adwords buyers purchase ads on Google, but it’s hugely useful for SEO as well. In addition to various ad-related info, the tool will tell us how many people search for a keyword each month, and will give us suggestions of related keywords that people also search for.

Google gets this data from everyone who searches on Google, as you can imagine. However, there are a couple things you should be aware of.

  • Google Keyword Planner search volume is based on the average of a year’s worth of searches. That means that all seasonality is averaged out. If Google says there are 12,000 searches per month, that might mean that there are actually 6k per month in the spring, but 20k per month during the holidays. Most searches have some kind of seasonality to them. The Keyword Planner will show you some bar graphs on top with the seasonality, but the numbers in the grid below are averages.
  • The tool will only give you search numbers for exactly the phase you entered. Perhaps you searched for “headboard” and saw it has 18,100 searches per month… but if you didn’t check headboards (plural) you wouldn’t have known that version has 49,500 searches per month!
  • In addition to being averaged, the search volumes are lumped into preset buckets. For example, a keyword might have 320 searches per month, but it will never have 413, or 390, or 450. The next bucket is 480. Google’s not trying to give you exact numbers, but instead is just trying to give you the qualitative information you need to have a good idea how much better or worse different keywords are when compared to each other.
  • Google will not actually show you all of the related searches that it knows about. The tool is trying to be helpful for people buying ads, so it’s less likely to show your searches that aren’t good for advertising. If you enter the term, Google will still give you the search volume, but it’s up to you to make sure you’re checking all the right terms.

Okay, time to get started!

First, head over to the Google Keyword Planner. You will have to have a Google account to use it, and Google now also requires you to have an adwords account. Don’t worry, you don’t have to spend anything or buy any ads. Just go ahead and sign up, then get back to the first page of the Keyword Planner, which looks something like this:

Google Keyword Planner Start page

You want to choose “Search for new keyword and ad group ides.” You’ll then get to a page like this one:

Google Keyword Planner Keyword Entry Page

In the top box, just copy and paste you list of keywords. On this page you can also set what region and language you want to check: just set it for your country and language. There’s also an option to limit your suggestions by entering negative keywords (in which case Google won’t suggest any phrase including that word) or a required keyword (in which case Google will only make suggestions that include that word). These options can be very useful for looking up specific things, but you can ignore them at first.

Now we click Get ideas and Google is going to bring us to a page that shows a bunch of data for each keyword (note: look at the “Keyword ideas” tab. Sometimes Google takes you to the “Ad group ideas” tab – you don’t want that). Since we’re not interesting in buying ads on Google, we can ignore everything except for the Search Volume number. That is the average number of people who search for that exact keyword phrase each month — on desktop computers, tablets, or mobile phones.

Google Keyword Planner Results Page

Now we can export this list by clicking Download from the top right of the data table. Our next step is to spend a few minutes looking over our output list. I like to sort it by volume, then delete anything that I think is too low — exactly how little is too little depends on your industry: for AwesomeDice.com I look at anything at least in the hundreds; for enterprise ecommerce sites I’d commonly only look at things in the thousands.

Note on the Competition Column: ignore this column. This is not a measure of how competitive it is to rank for that term. It is a measure of how competitive it is to advertise for that term with paid ads. The two are often quite different. For SEO purposes, ignore this column entirely.

With the garbage removed, I then like to sort it alphabetically and get a feel for where the good search volume is. Once we get to topicality optimization, you’ll see that while we optimize for keywords, we’re also really optimizing around clusters of keywords. A page optimized for headboards is probably going to rank pretty well for headboard, and head boards — and we definitely shouldn’t have a separate page for each (which creates a crappy site, and can lead to Google penalties).

Expand Your Keyword List

Our next step in the keyword research process is to expand our list beyond what we thought of in our brainstorming. Now we want to sort our list by search volume, and take just the top keywords.

Starting at the top, enter just one keyword into Google Keyword Planner. The reason we’re doing this is because the fewer keywords you enter, the more suggestions Google will give us. We’ll enter the keyword, tell the Keyword Planner to sort the recommendations by search volume, and see if there’s anything here that we didn’t think of.

Google Keyword Planner Results for Headboard

This is also the stage where it can be worth your while to use the filters to exclude or include keywords (using include will only show keyword phrases that include that keyword – in the example above I’m including “headboard”). For certain keywords Google’s suggestions can be all over the map, and limiting the results can help give you less junk to wade through. For example, if you’re selling apple pies, you probably want negative keywords like “computer, ipod, ipad, iphone” to exclude apple computer products — or you can set “pie” as a required keyword.

As you find suggestions that are relevant to your site, have a search volume over the minimum that you’ve decided, and that you don’t already have on your list, go ahead and add them to your list.

Keep an eye out for keywords related to your niche or industry that are informational in nature. For example, if I’m doing research for my site selling headboards and I discover some decent search volume for “how to refinish headboard” I want to make a note of that. Eventually I’ll probably want to write an article about refinishing headboards for my site. These kinds of searches can give you ideas for excellent content articles, which you’ll use later to attract links to your site.

Once you’ve gone through the first few pages of suggestions and have added all the keywords with decent search volume, move on to the next keyword on your list. Repeat this process with all of your high volume keywords. There’s no need to repeat closely related ones (headboard, headboards, head board, wooden headboards) — every one of those is likely to give the same suggestions. So you’ll only need to include substantially different keywords.

Keyword Research

All SEO begins with keyword research and for the most part centers around Google’s keyword research tool, the Google Keyword Planner. In this tool Google will tell us the average number of searches any keyword receives each month, in our country of choice. This is amazing data to have access to!

Step one of our keyword research process is to write down a list of all of the keywords you think might be relevant to your site. Each keyword is a phrase that someone might enter into Google when searching for something for which you want your site to rank.

When you’re making this list, be sure to include every variant you can think of: include singular and plural versions, include common features, etc. For example, if I wanted to do keyword research on headboards, my starting list might look something like this:

  • headboards
  • headboard
  • head boards
  • head board
  • modern headboards
  • modern headboard
  • traditional headboards
  • traditional headboard
  • wooden headboards
  • wooden headboard
  • wood headboards
  • wood headboard
  • bed headboards
  • etc.

There are several tools out there that can help you expand your list (which we’ll talk about in Other Keyword Research Tools), but for now let’s just begin by using your brain.

You can make your list as long as you want — by including all the variations you’ll be learning how people typically search for the product, service, or information that your site offers. You’d be amazed at how often the language customers use is different from what business owners think they should be using.

A couple of examples: I worked on a large site that insisted on using the term “mobile phone” throughout their site. About 10 seconds of keyword research showed that in the US, “cell phone” was massively more popular. In fact, I calculated that the site would make about an extra $5 million per year just by switching from “mobile phone” to “cell phone” throughout their site. Another site I worked on had a great selection of accent pillows for couches. Some quick research showed that the phrase “decorative pillows” had six times the search volume. The point here is that the language used by the majority of potential customers is not always what you, or your bosses, think it is.

Remember, there’s no cost to anything in the keyword research stage, so feel free to include the most obscure variants you want. Cover all of the bases and save yourself regrets down the road.