Competition can be scary, especially when it comes to SEO.
When trying to rank for a keyword, you might be going up against a junior intern, or you might be going up against an experienced SEO company.
There are tools like Market Samurai, Long Tail Pro or SEO Spyglass that can help, but you need to understand how to use them effectively.
Typically, you’ll be competing with other SEOs around your own skill level, although that’s not always the case when you complete a competitor analysis.
The good news is that if you take the time to do comprehensive keyword competition analysis, you’ll be able to identify keywords that haven’t been targeted by the most skilled SEOs in your niche.
While it will take you a lot of up front time, it will save you much more in the long run. You’ll be able to develop a keyword planner that competes with all competitor keyword phrases.
Assuming you’re producing high quality content, you’ll have a much easier time getting it seen and getting it ranking highly on Google and other search engines.
You will need exponentially fewer backlinks, which will save you either a ton of time or money.
In truth, there are only a small amount of low competition keywords in a niche at any given time on the search engine(new keywords pop-up as others disappear), but that’s all you need to increase your search volume.
Ranking for just a handful of low volume, but low competition keywords will get your organic search volume started. Traffic has a tendency to grow exponentially.
As you build your authority and trust with visitors and search engines, you’ll be able to start ranking for more competitive terms, down the line, without any major new investment.
I’m going to show you a comprehensive method for analyzing keywords to target. None of it is too complicated, but it is a lot of work. Be prepared to put in the time now and it will save you effort in the long-term. Average business competitor keyword planning will not yield the same results, giving you an edge.
Learn how I grew my traffic by 395,526 visitors through advanced SEO.
Why no competition analysis is perfect
Before we get going, there’s something important that you need to learn.
Keyword competitor analysis is an estimate of the competition for a keyword, but it’s not a science.
Just because a keyword appears to be easy, or easier than another keyword to rank for, doesn’t mean that it will be easy to rank for in all cases.
Yes, competition analysis is very useful. It will give you a good idea of what you’re up against and where your opportunities are.
However, remember that your keyword research results won’t always be accurate. There are 2 main reasons for this.
First, no one knows what the search engine god, Google is thinking: While we understand which ranking factors tend to be the most important, for the most part, we can’t exactly quantify them.
Your competition analysis is a reflection of how you think Google ranks sites. But, even teams of very smart analysts have been unable to recreate the Google ranking algorithm.
This means that our competition analysis methodology isn’t 100% accurate.
In addition, Google constantly changes their algorithm (about 500 times per year).
Even if you’re able to perfectly predict the competition level for a keyword today, it might be a bit off in a few weeks or a month.
Second, there’s a tradeoff: When doing keyword analysis, you will always face a tradeoff between efficiency and accuracy.
The more ranking factors that you try to take into account, the more resources (time and money) you will need to do your competition analysis.
If you only take 1 or 2 ranking factors into account, you can do competition analysis quickly, but it won’t be particularly accurate. As you add more ranking factors into the equation, you start to get more accurate results, but it takes more time.
It’s up to you to decide on a good balance between efficiency and accuracy for your situation in doing keyword research and implementing a keyword planner based on competitor analysis.
The most important SEO factors to consider
The main strategy behind keyword competition analysis is to look at how the top rankings fare when it comes to the most important SEO factors, including long tail phrases often used as anchor text.
For example, one of the factors we will look at is the number of backlinks. If a page has 0 backlinks, it’s likely to be easier to beat than a page with 100 or 1,000 anchor text links.
When we do this for several factors, we are able to see if a keyword search query is “low competition” or “high competition.” Then, you can decide whether or not to target that keyword.
Our first step is to decide on which factors to consider in our competition analysis.
To do that, let’s turn to the SearchMetrics 100 page report of the most important ranking factors.
This report’s data comes from over 100,000 different search engine result pages (SERPs). The team looked at different potential ranking factors, to see if top ranked sites tended to score higher (in regards to these factors) than low ranking sites.
Below is an image excerpt of the factors with the highest correlations, which means that top sites tended to have more of these factors:
Remember, when doing keyword research, that correlation does not equal causation.
What this means is that just because top ranked sites had a lot of Facebook likes and engagement doesn’t mean that Facebook likes and engagement causes you to rank highly.
In fact, in that specific case, it’s more likely that it doesn’t increase search volume when doing a competitor analysis – don’t get intimidated by their numbers. Sites that rank highly get more traffic, which likely leads to the boost in social sharing.
This is part of the competitor analysis that has nothing to do with the competitor keyword long tail phrase use. Understanding the metric can help you improve your websites search query numbers.
Are any of those correlations useful?
Yes, because some are legitimately caused by the factor helping sites to rank better.
It’s our job, as SEOs, to test each factor individually and figure out which ones do and don’t help.
For years, we’ve known that backlinks with anchor text help. However, Google has come out and stated that social signals do not affect rank. Some case studies have seen temporary ranking increases from social signals, but it’s not a factor I would suggest focusing your attention on at this time.
For the purposes of this post, I’m going to look at the most important factors that are known to help pages rank. If you’d like to add more on top, later in your own analysis, then you’re welcome to.
Factor #1 – Backlinks: When a site links to another site, it counts as a “vote” for the site being linked to. That’s nothing new. We know that backlinks are a key ranking factor and that they need to be part of any analysis.
However, we need to look at them on a few different levels, which is where it can get a bit tricky for beginners.
First, we need to consider that backlinks are important on both a page level (links to the exact page), and on a domain level (total amount of links to all pages on the domain).
Second, we need to consider that not all links are created equally. Links can have different value, based on which page they are located on and where they come from and how anchor text is phrased.
To analyze backlinks, you need a backlink database tool. For a serious analysis, you’re going to need a paid plan to one of the best tools. I recommend Ahrefs or Majestic.
They are, by far, the 2 most comprehensive backlink database tools. These are referred to as site explorer tools, examining the backlinks to ensure they are working properly and serving you well.
When you want to analyze a specific page (from a SERP), you’ll simply plug it into the textbox on one of these tools.
As an example, let’s say that you saw my beginner’s guide to online marketing in a search result and decided you wanted to see how hard it would be to outrank it.
Typing the URL into the Majestic site explorer reveals that there are 232 domains that link to that specific page and over 4,600 total backlinks from those domains.
That’s quite a bit for any single page.
In addition, you also need to check how strong the domain is, in general, with the site explorer. By switching the dropdown beside the URL, or changing the URL to the root domain, you can see all of the links to Quick Sprout:
Just about 16,000 referring domains and well over 1 million backlinks. This is the start for your competitor analysis.
So that’s how you look at quantity, but how do you look at quality?
One high quality link is worth hundreds or thousands of low quality links, so it’s important to not just go by numbers.
You could examine the quality of each link individually. A high quality link is:
- on a page that has a lot of links to itself
- relatively high up on the page
- found naturally in the page’s text (surrounded by appropriate description)
- is on a relevant page
- is on a page without too many links (link power is divided by number of links)
In other words, quite a bit goes into it.
It would be impossible to evaluate this for every single link even with an efficient site explorer.
Luckily, link database tools have a pretty good solution for us. They algorithmically try to determine the quality of each link. It’s not perfect, but it’s pretty good.
On Ahrefs site explorer, every page and domain get a score:
- URL rating: A score that represents the overall quality and quantity of links pointing to that specific URL, on a scale from 0-100.
- Domain rating: A score that represents the overall quality and quantity of links pointing to anywhere on the domain, also on a scale from 0-100.
Majestic’s site explorer is slightly different. It uses 2 metrics:
- Trust flow: A score purely based on the quality of links to the page that you enter.
- Citation flow: A score based on the quantity of links to the page that you enter.
In general, citation flow will be a bit higher than trust flow, but if it’s more than about 1.5 times higher, it’s likely that the page has a lot of low quality links.
Instead of manually checking the quality of every link to a page, we’ll be using these metrics (or similar) for a quick check.
Factor #2 – Relevance: When it comes to search results, relevance is the most important factor. When someone searches for “yellow tables”, they will be disappointed unless they find a page about yellow tables.
A long time ago, relevance was mainly determined by having the exact keyword in the domain, title and body of the page. This is why keyword planner tools are vital.
So now, if a searcher types in “yellow tables”, Google knows that they probably want results where they can buy a table, not a fluff article about what a yellow table is. That’s user intent.
In addition, Google would also include synonyms for either yellow or table in the results. So, you might see results for “golden tables” or “yellow stands.”
This is a factor that you will have to assess manually. I have not come across a reliable wide-source automated tool for this area of improving your search query results.
If you can become a long tail pro in your own keyword planner ideas, you will be well on your way to organic content that is designed for search volume.
We can look at basic keyword density, but it’s very difficult to understand user intent and include the right synonyms, without manually looking at a page.
Let’s walk through a quick example to compare competitor keyword phrases.
Pretend you were searching for:
“content promotional tactics”
Google will bold synonyms in meta descriptions and URLs. In this case, we see that Google knows that “content promotion strategies” means essentially the same thing as what we did a search query for.
If we’re assessing the top few results for search engine relevance, our main question is: “how well does this satisfy someone who searches for the keyword?”
If the answer is, “not very much”, you can probably outrank it.
Here’s the second result from that search:
This page has “50 promotional tactics”.
And, while that’s fairly comprehensive search query, I feel that the quality could be improved. The picture above is of the first 2 tactics, which aren’t really promotional tactics at all.
They are also not very data-driven and there’s no clear-cut examples or walkthroughs, which the visitor would probably appreciate.
If you come across a keyword where the first 3 results are nearly perfect answers for the query, don’t bother trying to outrank them, it’s going to be very difficult. However, that’s also a rare situation.
Factor #3 – User satisfaction: This is related to relevance, but there are a few differences in how we will evaluate it.
It also needs to be done manually, but we can look at a few different factors, to determine how much users typically like the page with the competitor keyword use.
Since we can’t see things like bounce rate and time on page, we need to rely on public information.
First, we can start with how many social shares it has. A page that everyone loves will have a decent amount of social shares. If we see a page with few shares (in niches where people aren’t embarrassed to share), we know that it’s probably not fully satisfying searchers.
On most sites, you can see the share count displayed somewhere prominent:
But, if you can’t find the share count, use a share count tally tool.
Just type in the URL when prompted and submit it:
It will pull up the number of shares from the most popular networks:
The second place that we’re going to look is the comment section (if there is one).
If people are saying things like:
- “Amazing post!”
- “This changed my life”
- “This is the best post on (topic) I’ve ever read”
Then, they’re likely satisfied. On the other hand, if there are a lot of complaints or suggestions, most visitors probably left the page unsatisfied and went back to the search results.
Look for both the number of comments, and what’s in the comments themselves.
Factor #4 – Do you consider all your visitors?: Google has made it clear that it wants you to optimize your website for your visitors. Your keyword planner helps with every new post you write.
If the top results are not mobile-friendly and load slowly, it’s an indicator that Google has to rank a page that it doesn’t really want to. Unfortunately, there are no other pages of the same content quality, but that are also faster and responsive. You could fix that.
This is another manual check, so you won’t need to do it for every search result. But, it can be used as a final check before you make a decision to target or not target a keyword.
First, check if the page is mobile friendly, by using Google’s own mobile-friendly test:
Put in the URL of the page and click analyze. It’s a simple pass and fail test.
Next, check site speed, using a site speed tool like Gtmetrix. Paste in the URL and click “analyze.”
After a quick scan, you’ll get a performance report for the site. Pay special attention to the “Page Details” box:
In this case, the page isn’t slow, but it’s not fast either.
In general, a fast loading page loads in under 2 seconds and only has a few dozen requests. If you see a page that takes 4+ seconds to load, there’s a real opportunity to beat it.
Tools to help you do this faster
By now, you might realize that it would be next to impossible to do a large scale keyword competition analysis without the help of some automation, which is where tools come in.
There are hundreds of tools out there to do this specific job, some better than others.
Most of them work similarly, but I’ll walk through a few, so that you can understand how to use them and how they work.
Tool #1 – Term Explorer: This tool can be used to find keywords, but also to analyze competition, which is what I’ll focus on.
Your first option, once you create an account, is to run a bulk keyword job:
You enter one or more phrases from your keyword planner and it will give you a list of results of anywhere from 1,000 to 90,000, depending on your account type and choice.
Once you run the job, you will get keyword results, along with search volume data. You can easily filter the results according to keywords or search volume.
If you find a few keywords that you like (or many), you can check the box beside them and then click the blue button at the top to send them to the keyword analyzer:
Alternatively, you can enter in keywords from other sources directly into the keyword analyzer.
You will get a similar report this time, but you will see an overall “difficulty” score from 1 to 10. This is score is aggregated from the 3 categories to the right: relevancy, link strength and trust.
A higher difficulty means that it’s more difficult to rank for the term. With this particular tool, a score under 3.5 or so is “easy”, while 3.5-5.0 is “normal”, and above that is difficult.
Tool #2 – KWfinder: This is another tool similar to Term Explorer. To generate keyword ideas, enter a seed keyword into the box:
It will quickly generate a list of keywords (I believe based on Google’s Keyword Planner).
You’ll also note that each keyword will have an “SEO” score beside it to understand how it fares on a search query:
The scores range from 0-100 and are color coded (green is easy, red is hard). If a score isn’t showed by default, you will need to click the magnifying glass.
In addition, you can get a more detailed look at the competition behind any of the keywords.
If you click a keyword, the search results for that keyword will pop-up on the right. It shows data from Majestic (trust flow, citation flow) and also shows you an SEO score for each search result.
The average score is the one you initially saw.
This is important because average scores can be skewed by one or two results. If there are some very low scored pages at the bottom of the first page (maybe they’re temporary), they could make the keyword look much easier than it really is.
Tool #3 – Moz Keyword Difficulty tool: A third tool that you can use for keyword competition analysis comes from Moz.
It’s one of the many tools that comes with a Moz Pro subscription.
Type keywords that you’d like to analyze into the tool’s main text box.
It will bring up a quick report that shows you the keyword difficulty as a percent (maximum 100).
To get more detail, you can click the “view” button that’s under the basic SERP report column:
The bottom section shows the same information as the top section, just in a bit more detail.
Note that the page authority and domain authority here come from Moz’s Open Site Explorer, which isn’t as reliable or complete as the other link databases we’ve looked at.
Tool #4 – Ahrefs Toolbar: This tool isn’t an automated competition checker. However, it can be used to quickly assess the strength of an actual SERP.
When you search for a term in Google, the toolbar will load a small bar under each result:
It shows link information from the Ahrefs database on a page and domain level. In particular, you can look at the URL rank (“UR”) and the domain rank (“DR”).
Step #1: Gather your keywords
Now that you have a good idea of what competition analysis consists of and of some of the tools that you can use to simplify the process, I want to walk you through, step-by-step, how to actually do it.
Before you analyze the competition of keywords, you’ll need to gather a rather large list of them. It’s more efficient to do competition analysis in bulk.
Keyword research is an important skill by itself and it’s something that you should spend some time learning before doing competition analysis. Here are some guides that will help fill in any gaps in your knowledge:
- 5 Modern Keyword Research Methods to Uncover Hidden Gems
- The Definitive Guide to Keyword Research
- Keyword Research – The Advanced Guide to SEO
Try not to take the easy way out, when you’re doing keyword research.
If you just plug in a popular keyword like “content marketing,” you’re going to get the exact same keyword list of thousands of SEOs before you.
Using some of the creative methods in those guides above, you can find “hidden” keywords that fewer people are targeting.
When less people are targeting a set of keywords, you’re more likely to find some low competition keywords that are ripe for the picking.
Step #2: Start filtering out keywords
Once you have a large list of keywords, it’s time to start assessing the competition for them.
Unless you have an experienced eye (and even then it’s difficult), you’re not going to be able to reliably pick out low competition keywords without a thorough assessment.
This means you have 2 options:
- a manual assessment: You could manually review all of your keywords. However, unless you have a few weeks of free time, this isn’t really feasible.
- an automated/manual hybrid assessment (recommended): You can use tools to get started and find the highest competition keywords. Take these out of the results and then manually review the more promising keywords.
I would hope you’d pick the second option.
Run your keywords through a tool I showed you above (or similar tool), so that you can get an estimate of the competition:
It’s important to remember that this is just a quick estimate. There are many ways that the results can be skewed.
If you’re creating a ton of content, you might be able to just target all the lowest marked keywords and then see which pages are ranking the easiest. However, most people need to be more selective.
You can typically check the box beside a keyword and save the selected ones to a private list. If not, save the most promising ones manually to a spreadsheet.
The idea here is to take out any keywords that are obviously going to be too difficult to target.
Step #3: Dig in deeper
By the end of that process, you should have a much smaller list of keywords. I recommend aiming for between 10 and 20% of what you initially started with.
Give the list a quick review and take out any keywords that don’t make sense for your site.
Once you have a final list, the hard work begins.
Your tools have told you that every query you are left with is relatively easy to rank for based on the most common metrics, which is usually a combination of domain and trust authority.
Now, you get to see how easy they are to rank for from a user’s perspective.
For each keyword, search it in Google, but you need to make sure it isn’t personalized.
Go to “http://Google.com/ncr”, which is the global Google search (the “ncr” prevents redirection to your local Google). Make sure you’re in a private (or incognito) browser and you aren’t logged in to any Google account.
You can choose how many results to look at. I recommend starting with the top 3. If you’re not sure if the keyword is low enough competition, continue with the next 5 to 7.
Viewing the results for each keyword, you want to evaluate the following things:
- how many (and what quality) backlinks point to the page
- how many (and what quality) backlinks point to the domain
- if I was a user who searched this keyword, would I be fully satisfied with this result?
- does it load quickly?
- is it mobile friendly?
You can also add in any other Google ranking factors to your analysis as you’d like, but it will take you more time.
Step #4: Make a decision
After you analyze each keyword, you have to decide if it’s worth going after or not.
Unfortunately, there’s no magic metric or tool that will help you with this.
To make this decision, you need to first consider many factors:
- How strong is your domain? (How many links point to it, what’s its domain authority?)
- How easily can you get backlinks? (Do you have experience in SEO? Do you have a network of contacts to ask for links from?)
- What level of content can you produce? (Do you have the skills or financial situation to create the best content?)
At some point, the authority, trust and relevance of the page you create for a particular keyword needs to exceed all other results that you have examined.
The stronger your domain, the easier it is to rank.
The easier you can get quality backlinks, the easier it is to rank.
The bigger the budget you have for content, the easier it is to produce something that no one else can match. Not only does this make it easier to rank short term, it also makes it easier to stay there.
Based on these factors, you have to determine if a keyword is worth going after, during your manual review.
Keyword competition analysis is not a science.
While you can get some information from a tool, you’re also going to have to apply some SEO expertise of your own to find low competition keywords.
This is going to take some practice and time. The good news is that you’ll save that time, and so much more, if you pick your keywords carefully.
The easier the keywords that you target are to rank for, the more consistent your results will be.
Once you have a list of keywords to target, it’s time to plan your content strategy around them.
If you have any questions about how to do keyword research for your site, let me know in a comment below.