Keyword research for SEO: The ultimate guide

2024-10-01 00:00:07

Keyword research is a cornerstone of SEO success, but many SEOs do it incorrectly. Instead of focusing on keywords that drive revenue, they only chase traffic volume.

Instead of focusing on keywords that generate revenue, they often prioritize traffic volume alone.

This guide tackles a new approach to keyword research that boosts your online presence and directly contributes to your bottom line.

By the end, you’ll understand how to transform keywords into valuable assets that drive business growth and profitability.

Why most keyword research is wrong

Before writing this article, I searched for and read several keyword research guides.

They all suffered from the same problem. They only talked about how to acquire traffic from Google and not about why.

And the “why” is the most crucial part of keyword research.

As such, this article is split into two parts.

First, we’ll examine why we use keywords and how they lead to business success.
Then, we’ll examine the process for identifying business-relevant keywords.

Let’s start with the “why” behind keyword research.

Why are you doing keyword research?

The problem with keyword research is that people talk about how to do it but never talk about why you are doing it.

Most people, if asked this question, will talk about traffic.

But unless you are a publisher earning revenue from display advertising, this metric is too basic to be valuable for SEOs working for businesses that need leads, sales, profits and growth.

Others might use phrases like building brand awareness or talk about concepts like the ‘funnel’ and how they want to create top-of-mind awareness through content.

However, these are all essentially flawed and over-simplistic approaches that generally do not work or deliver value for clients.

To make life easier, I’ll explain why you are doing keyword research.

Revenue generation.

The goal of keyword research is to find keywords for which a brand can rank, leading to increased revenue.

So, for keyword research to be successful, we need to learn how revenue is earned online.

Dig deeper: Keyword research for SEO: 6 questions you must ask yourself

How brands make money from SEO: The fundamental rule of keyword research

Brands make money from SEO through leads and sales.

And here’s where SEO starts to unravel.

Traditionally, SEOs view their work as optimizing a website for all funnel stages. This is a strategic error. Here’s why.

The traditional marketing funnel looks like this:

This approach leads to using SEO to rank web pages, targeted at all stages of the funnel.

TOFU: Top-of-funnel content to build brand awareness.
MOFU: Middle-of-funnel content to reach those considering their purchase options.
BOFU: Bottom-of-funnel content where people buy products or services.

SEOs have sliced the funnel up in many ways, such as by renaming metrics as informational, commercial, etc.

However, the core goal has remained: to use SEO as an all-in-one marketing channel.

This has resulted in websites with huge traffic numbers and brands creating content for every keyword they can think of, believing it’s all advertising.

But most web traffic is useless.

Because it targets people at the wrong time.

SEO is product placement, not advertising: A tale of how marketing works

The argument by SEOs around keyword research is that “all traffic is good traffic.”

This makes the delivery of SEO more aligned with the “publisher model.”

SEOs will create content at scale to generate traffic and justify SEO.

This means that most SEO content is called “top of funnel.”

The argument used by SEOs is that they use SEO to create brand awareness. And through ebooks or other lead gen activities, you acquire marketing qualified leads (MQLs).

I’ve seen this justified using first-touch attribution data, where an email sign-up later becomes a customer, justifying the content.

Firstly, marketing metrics are rarely this granular. Yes, it will undoubtedly have first-touch attribution as “organic,” but so will all your organic traffic.

This includes your service and product pages.

However, significant studies show that we do not purchase or recall brands this way.

Dig deeper: 6 vital lenses for effective keyword research

We serve busy people with busy minds

When I think about keyword research strategy, I think about this line from Byron Sharp’s book “Marketing Theory, Evidence and Practice.”

“Our marketing and brand is ignored and forgotten by most people most days.”

The reason for this is the forgetting curve.

The forgetting curve is a well-established principle in memory function.

In short, we quickly lose what we don’t use regarding new information.

To demonstrate this, I often ask clients to name the last five articles they read and videos they watched online.

Followed quickly by asking them the brand or person behind them.

The answer is almost always met with difficulty.

I then ask the clients to name ten toothpaste brands. The struggle is intense, even for something as simple as toothpaste.

We have too much going on in our heads to recall the name of every brand we encounter online. We often forget things such as PINs and loved ones’ birthdays.

And yet, there is a romantic idea among SEOs and content marketers that the 3,000-word ultimate guide the prospects read 10 months ago.

They somehow embedded the brand in their memory architecture to such an extent that they can recall it and themselves easily.

This logic looks like this:

“We need to get some food for our new puppy. We’ll buy it from Pooch Love online. I read their 400 names for Daschunds article six months ago. They look great; I’ll search for them.”

OK, so that’s a little tongue-in-cheek. But this is the logic behind the vast majority of SEO strategies.

The idea is that all traffic leads to brand recall.

But it doesn’t. This is why advertising exists. To build and refresh memory structures around brands.

We need to ditch the idea of using SEO to build brand awareness through top-of-funnel logic.

The articles you’re writing that are growing your traffic are forgotten minutes after being read.

Your brand isn’t remembered and your traffic graph is meaningless.

So, what matters?

Tom Roach, VP of brand strategy at Jellyfish, wrote a compelling article about why the marketing funnel needs to change.

In his modified funnel, SEO sits as a connecting function. Connecting immediate prospects with brands.

And a bit of nudging as well.

We connect brands with buyers and show up on their purchase journey.

With this as our goal, keyword research will be more focused and revenue-driven.

Now that we’ve thoroughly covered the why of keyword research, we need to dive into the how.

Your new keyword process

The standard of keyword research will always depend on the amount of time you have to conduct it.

If you have a small retainer, the time you have to conduct keyword research will be limited.

But a good keyword research process looks like this:

Building your keyword strategy.
Customer and competitor keyword research.
Mapping your buyers’ keyword journey.
Keyword ideation – identify what your customers need.
Selecting your keyword assets.
Asset viability (manual keyword research).

Let’s break each section down.

Building your keyword strategy

When addressing keyword research, we must start with the premise that our brands are easily and quickly forgotten.

That’s the starting point for SEO.

And so, we come to the customer or buyer journey.

The buyer’s journey is a series of touchpoints from product awareness to purchase.

Many brands are discovered online with buyer intent keywords, which is no different from how many products are discovered in stores. We walk along physical aisles, where many brands are first discovered.

Search is the digital version of this.

Our first search terms are often broad and straightforward.

Often referred to as head terms. These keywords are broad and are usually expensive for paid search.

These keywords are also often competitive to rank for.

You need to start your keyword strategy with the client and their budget in mind.

There is no point in building a huge keyword strategy if the client lacks the resources to rank for the terms.

Before you take another step forward, it is useful to run through a SMART analysis at the beginning of the process to build your strategy.

Specific: What specific results are we trying to obtain here? Are we looking to grow revenue for a particular service or all services/ products?
Measurable: Where is the client now? What are their rankings?
Actionable and achievable: Can we achieve the desired result with the current budget? Do we have the resources and stakeholder buy-in?
Realistic and relevant: Are the desired results realistically achievable? Are they relevant to the business goals?
Time: How much time do we have? When are results likely to happen?

Much of the above can be gathered with a simple client call and a gut feeling by quickly checking some data and competitors.

The above is not designed to be perfect. It’s more of a guide as you move through the process.

We can now move to the next step.

Customer and competitor keyword research

To conduct your research into customer needs, you need data.

Some sources can include:

Customer data/ business data.
Search console data.
Paid search data.
Keyword research tools.
Google Analytics 4.

In-house teams will generally have access to more data and more time for the above, while agencies and freelancers will have less.

However, the aim of this initial stage is to gather as much data as possible about your buyer’s search journey.

What are they typing into search engines? What do they think when they are looking into buying what you sell?

Many keyword tools can assist with this. ChatGPT is, however, surprisingly good at giving you starting points for this information:

Throw in a simple prompt and you will gain some logical searches to consider.

If we look at what a simple prompt gave me if I was looking for “in the market” keywords for people buying fish tanks.

General fish tank searches.
Size-specific searches.
Type-specific searches.
Brand-specific searches.
Material-specific searches.

This isn’t exhaustive, but it is designed to get you thinking about a consumer.

The good thing about ChatGPT is that you can analyze markets for ideas, keyword formulas and more.

I have deliberately not used a specific search tool in this article because there are so many with their own guides.

What matters is that you gain the data from various sources. Our aim is not to study all the keywords but to make an SEO plan.

We are gathering as much data as possible about how our consumers might search for a brand like ours when they enter the market to buy.

Using this data, you can now map the buyer’s keyword journey.

Mapping your buyers’ keyword journey

To build your strategy, you need to identify the customers you want to target and the search terms they use. Then, we can map our keywords against the buyer’s journey.

For example, let’s say you are in the market for car insurance. Your 17-year-old daughter has just passed her driving test and wants a car.

You might start your car insurance journey with a term such as “cheap car insurance” or “new driver’s car insurance.”

You scroll and click the results, read the websites and leave because you must take your daughter to her fitness class.

A few weeks pass, and your daughter reminds you of the car insurance she needs for the car she has convinced you to buy.

You repeat the same search, but perhaps this time, you click a few results but see Google’s PAA (people also ask) section.

You click some more and new questions such as “How to reduce car insurance for new drivers” appear.

You are now in what Google calls the “messy middle of search.”

The messy middle is a world of exploration and evaluation, where brands can choose to show up and increase their chances of being selected and recalled by buyers.

We receive internal or external triggers to search.

We then explore subjects and vendors.

In the end, searchers will have a number of brands that enter their ‘consideration’ set of vendors or products they want to purchase from.

Every buyer search journey depends on the sector. Some take place in minutes, and others take over 12 months.

And this is the search journey we want to map out as best as possible.

We are trying to paint a possible picture of our consumer’s online activity in relation to purchasing.

In the car insurance example earlier, we saw how a prospect makes many searches over a period of time before building a consideration set of brands.

To map the keyword journey, it’s useful to split keywords into three categories:

Head terms
- Used for initial exploration.
- Used to evaluate brands.
- Used for buying.

Body or medium tail keywords
- Used for broad subject exploration. Often informational, when a prospect seeks more information on a subject to help them make an informed purchase decision.

Long-tail search
- Used for deeper subject exploration and unmet needs.
- A user is trying to find a precise solution and previous searches have yet to satisfy them.
- A user has complicated product, service or informational needs.

Once you gather your data in the next step, your keyword strategy will start to take shape.

Lower budgets might mean you can’t rank for competitive keywords, but you can reach people as they explore the subject with buyer intent.

Or you may focus on a more targeted and specific group of consumers.

Now, look at your data and add this to the three areas of heady, body and tail.

You can do this via a spreadsheet with three columns.

Dig deeper: How to find high-potential keywords for SEO

Keyword ideation: Identifying what your customers need

Once you have your data, you need to address what keywords match the customer’s needs.

These are keywords that a prospect will use to help them to choose a product, service or brand.

From your research, you’ll have a lot of data, but we need to consider how humans use the web and align our keyword research with that.

Kantar conducted research with Google to identify six needs that searchers have when they use Google.

These needs were identified as follows.

Educate me.
Thrill me.
Surprise me.
Impress me.
Help me.
Reassure me.

Let’s break down each one.

Thrill me

When people come to search engines with this need, they are often at the earliest part of their search journey.

“Search is a quick adventure to find new things. It is brief, with just a few words and minimal back-button use.”

Customers might be looking for ideas, inspiration, examples, entertainment, visualization success stories, case studies and more.

This is often where brands can use other media, such as social media, to grab the attention of consumers.

Surprise me

Searchers using Google with “surprise me” needs are looking for inspiration.

“Search is fun and entertaining. It is extensive with many unique iterations.”

It’s likely that these keywords are early on in a consumer search journey. This is similar to the “thrill me” search terms but usually more broad.

A lot of TikTok content will be consumed this way. With a consumer searching for trending products and information.

Educate me

People who are looking for education search for items such as reviews and comparisons.

“Search is about competence and control. It is thorough: reviews, ratings, comparisons, etc.”

They can seek out ratings and look for vendor experience, expertise, knowledge and authority. Which can come in the form of information and credentials.

It’s here that a brand should focus on displaying their E-E-A-T on their site and it’s likely that these keywords are used for customers who are placing brands in their consideration set.

Impress me

Here, consumers look for status, knowledge and experience.

“Search is about influencing and winning. It is laser-focused, using specific phrases.”

Many searches here are where a consumer knows exactly what they want. Luxury items will live in this sector, but consumers are also seeking out higher-end goods or services.

Help me

If a searcher needs help, they often seek advice, tools, templates and answers.

“Search is about connecting and practicality. It is to-the-point and more likely to mention family or location.”

The searchers have a definite problem and possibly an urgent need. SaaS platforms often win in this range of needs. So do local ‘emergency’ vendors such as plumbers and electricians.

But content can live here too. If you’re a SaaS platform that connects plumbers to locals in need, you could create content that teaches people how to solve their own problems.

By seeing what it entails and thinking, “This looks too much like hard work,” you have framed them for the next step: using your services.

Reassure me

Finally, “reassure me” keywords are related to support, answers and aftercare.

“Search is about simplicity, comfort and trust. It is uncomplicated and more likely to include questions.”

A huge number of medical and legal searches exist for this search need.

For many, it’s easy to dismiss this part of the buyer’s journey, as you might find in your research that these types of keywords are post-purchase.

However, these keywords can have a huge business impact if you can reduce support calls and even complaints.

At this stage of the journey, we are analyzing our search data and identifying our consumers’ search needs. You will probably see a pattern emerge.

For example, if you sell garden furniture, you’ll probably see consumers looking for inspiration and ideas more.

In the beauty industry, people might focus more on cost and finding treatment locations near them.

Regardless of your niche, you have now taken your consumer and search data and placed it in the context of search needs. My advice would be to build this out on a spreadsheet or other board tool.

Run through your head, medium and tail keywords earlier and assign them to each search needs.

Once you’ve done this, it’s time to look at the assets you can create to satisfy these search needs.

Dig deeper: Rethinking your keyword strategy: Why optimizing for search intent matters

Selecting your keyword assets

To identify and select your keyword assets, you will build two keyword lists:

Exploration keywords
- These are the keywords that a buyer will use at the beginning of their search journey.
Evaluation Keywords
- These are the keywords used to decided if your products or services and brand are for them.

We’ve established that:

The messy middle consists of two key stages where we can engage.
At each stage, users have distinct needs, which can be categorized into six main types.

Using the data we’ve gathered, we can begin developing solutions tailored to these consumer needs.

To start, we can organize these search needs into two groups: exploration and evaluation.

Keep in mind that there may be overlap between the two, and the categorization will vary depending on the specifics of each business.

Here’s an example of how this might look:

Exploration keywords
- “Educate me” keywords (list them under)
- “Thrill me” keywords
- “Surprise” me keywords
- “Help me” keywords
Evaluation keywords
- “Impress me” keywords
- “Educate me” keywords
- “Thrill me” keywords
- “Reassure me” keywords

Based on your data, you can start populating your buckets as you meet the search needs of your consumers.

For example, your data might show that your customers need education and you will create guides to match this search intent.

Or perhaps a dedicated review section of your website needs to be created to gather customer testimonials.

The aim here is consider the assets you could create to meet your consumer’s search needs based on your data.

Here are some ideas:

How to guides.
Knowledge centers.
Landing pages.
Tools.
Calculators.
Templates.

Each keyword should have a asset aligned with it.

Once you’ve done this, we move into the final stage.

The manual keyword research process

This is the part where most keyword research guides begin, and ours will end.

The reason is simple: as an SEO, you’ll use many different tools throughout your career.

It’s important to develop a keyword research process that works independently of any specific tool but can be applied to all of them.

What matters is the process and not the search tool.

Filter through your data to establish commercial viability for keywords and assets. Here’s how.

Establishing keyword viability

At this point, you should have two lists for Exploration and Evaluation.

Under each section, you will see a list of assets you have decided to create.

And for each asset, you’ll have a list of keywords that each asset targets.

Let’s say your research showed that people were searching for how much something costs.

You’ve identified that an online calculation tool would meet consumer search needs.

The purpose of your manual keyword research is to review the keyword to look from a business viability basis.

You’ll need to consider the search volume available, your internal resources and the time/ cost of ranking for that keyword. You should also consider the potential rewards for the business.

This will depend from business to business, but at the end of the process, you’ll have identified assets you can create for consumers that meet their search needs. And you’ll be better placed to gain buy in from stakeholders.

Conclusion

If you made it to the end of this article, you deserve a medal.

Establishing a revenue-focused keyword research process that aligns with consumer needs greatly stacks the odds in your favor.

Here’s a final recap of the process:

Build your keyword strategy: This involves setting SMART goals, considering the client’s budget and resources and understanding the competitive landscape.
Customer and competitor keyword research: Gather data from various sources, such as customer data, search console data, paid search data and keyword research tools, to understand customer search behavior and intent. Tools like ChatGPT can also be leveraged to generate keyword ideas and analyze market trends.
Map your buyers’ keyword journey: Understand the customer’s search journey from initial exploration to final purchase, categorizing keywords into head terms, body or medium-tail keywords and long-tail search terms.
Keyword ideation: Identify keywords that match customer needs, aligning them with the six needs identified by Kantar’s research.
Select your keyword assets: Create two keyword lists – exploration keywords and evaluation keywords – and identify the type of content or assets that best address these needs (e.g., guides, reviews, landing pages, tools, etc.).
Asset viability (manual keyword research): Evaluate the commercial viability of keywords and assets, considering factors like search volume, internal resources, time/cost of ranking and potential business rewards.

Do this, and you’ll have a great keyword research approach to help you generate ranking and traffic that drives revenue.

Dig deeper: B2B keyword research: A comprehensive guide