The Surround Sound Series: How We Built a Bespoke SERP Tracker and Measured the Results of Our New Content Program (Part 3 of 3)

This post is a part of Made @ HubSpot, an internal thought leadership series through which we extract lessons from experiments conducted by our very own HubSpotters.

As our three-part series comes to an end, there’s one last question (or two) we need to address:

Does the surround sound strategy really work? And what results have we seen since we started using it?

The TL;DR answer is yes. It’s working and we’re sharing our preliminary results in this article.

So if you’re looking to use surround sound in 2021, the two of us (that’s Irina Nica and Alex Birkett) thought we’d end this short series with a handful of resources: the formula HubSpot uses to measure progress, our proprietary surround sound tracker (aka the SERP Tracker), and a sample of results to hopefully inspire your own endeavor.

What is the surround sound strategy?

At HubSpot, “surround sound” is the name Alex Birkett gave to a content marketing approach that aims to make HubSpot products visible every time someone searches for a product-related keyword (e.g. “best help desk software” or “best live chat software”).

The goal is for potential users to see HubSpot mentioned in all or most of the pages that rank in Google top results for those keywords. Check out the first two articles in the series for a deeper-dive into what is the “surround sound” effect and how we scaled the program at HubSpot.

Surround Sound vs. SEO

Here’s where a lot of people get confused: The definition of surround sound says its goal is to improve visibility in search engines, but isn’t that what SEO does, too?

The answer is surround sound includes SEO results, but doesn’t stop there.

Here’s an example: say we want to increase our visibility in search engines for the keyword “free CRM.”

A first step would be to make sure our own domain ranks well for that keyword. That’s the “job” of the SEO function.

But visibility in SERP doesn’t stop there.

When potential customers are looking for a free CRM, they might not stop at the first results. Instead, they click through different suggestions of curated lists that they find in search results.

Our product awareness goal is to make sure HubSpot is visible in as many results as possible, especially on third-party domains. The level of SERP real estate that HubSpot is mentioned in (including owned and third-party domains) is what surround sound measures.

That’s why we say that in order to measure surround sound results, we need to measure our visibility in SERP, for a particular keyword or topic, beyond SEO.

But surround sound isn’t really a “mainstream” product awareness tactic — yet. So there are no methods or tools to measure visibility in SERP, beyond rankings (as far as we looked). That’s why, in order to scale our efforts, we had to build our own systems from scratch.

How HubSpot Measures Surround Sound Results

Let’s talk about scale.

If HubSpot were to have only one high intent keyword in our purview (say, “best marketing automation software”), it wouldn’t be insane to measure our visibility manually.

Once per month, you could simply open Google, search “best marketing automation software,” and count how many of the top 20 pages mention HubSpot.

This clearly becomes a problem when you add more than one keyword, however. Imagine the frustration of doing that for 600+ keywords per month.

In order to make this strategy “scalable” (a phrase I dislike, by the way, because it has been weaponized by curmudgeons who can all too easily shut down interesting ideas by saying ‘this won’t scale’), we needed to do a few things:

• Build custom tools and infrastructure to automate aspects of the strategy
• Educate others on the surround sound process and playbook
• Evangelize and excite others into buying into the idea

The latter was easiest; once you get some results, others can easily grasp the importance of an idea.

I’ll focus here on building custom tools. First, I want to explain the value of investing in custom tools, automation, and infrastructure, because I can already sense the objections:

• “We don’t have the technical resources to build our own tools and automations!”
• “Won’t building custom tools cost us a lot of money and time?”

To address the first point, there’s always a creative solution. If you don’t have internal resources to code up new tools, you can easily outsource this as long as you have good specifications. Sites like Code Mentor or even Upwork are great for this. Additionally, this is a good excuse for (mythical) non-technical marketers to build some new skills.

Second, building tools to reduce the marginal cost or effort in the long run increases the expected value of the program.

Lots of jargon in that sentence, so let me unpack that with an example from experimentation.

Why invest in tools, automation, and infrastructure?

Imagine the expected value (the “predicted value of a variable, calculated as the sum of all possible values each multiplied by the probability of its occurrence”) of an A/B test is $1,000.

Some proportion of A/B tests will fail, which means you only “lose” during the time of the experiment, some will do nothing, and some will win (and those that win will do so by varying levels). All in, if you averaged out these values, a given A/B test could be predicted at a $1,000 value.

Now imagine every time you wanted to run an A/B test, you needed to get two engineers, one designer, and an analyst not only to set up the test from scratch (you’ve got no testing tool here), but also, you have to do minimum one month of research, wait two weeks for approval after you submit your hypothesis, and then you have to sit through, on average, four meetings with this crew.

What’s the cost of all that? Over $1,000? Congratulations, the expected value of your A/B test is now negative.

But what if, on the other hand, you had a testing tool that made it easy to set up and analyze experiments (with trustworthy data), and also imagine you prioritize to focus on only high impact tests. Now you lower the upfront cost of each additional test you run (the marginal cost), which increases the expected value of each test (just slightly). On a program level, though, now you are rewarded for running more tests, since the expected value is positive, and you’ve got very low marginal costs. You spent a little time investing in tools and automation, and you’ve drastically increased the ROI of your program without even improving your win rate.

This is exactly what we needed to do to make the surround sound strategy economically viable.

The amount of effort to manually audit, measure, and store SERP visibility, as well as the manual cost of outreach and relationship management, would forever outweigh the positive benefits of the placements we could get on a SERP. That is, unless we could lower the costs of data collection and storage as well as outreach and relationship management.

Enter: the SERP Tracker

Now we have a functional tool that does all this for us! We call it the “SERP Tracker.”

Here’s the aesthetically-gorgeous interface:

(For any fellow nerds, it was written in R and it’s hosted and deployed with Shiny. We’re using serpapi.com to get Google’s search results, and we’re also using Bing’s API [free up to a certain number of credits]. I have an older version of the generic script here available for viewing — we’ve since updated it many times, but this version will still be functional.)

I’ll walk you through how it works, but you can also watch this Loom tutorial if you prefer video:

Here’s a quick tutorial on how you can use the tool and what kind of data it can give you.

First, you have to validate by entering a password. Since we use various APIs, we want to limit the usage to converse credits (again, we track some 600+ keywords, so it’s a weighty endeavor)

Then you can enter whatever keyword you’d like (like “best form builder” or “best help desk software”).

Then you can select a region and the number of search results you want to pull (up to 10 pages, or 100 results).

Finally, you need to enter the URL that you want to check the search results against.

In other words, if you want to check for the search keyword “best live chat software,” then you could enter “hubspot.com” or “drift.com” to see how many of the URLs that rank for the search keyword “best live chat software” mention that URL.

I have preset options for HubSpot URLs to check our product page, blog posts, or generic hubspot.com pages:

You can also enter a custom link by clicking “other.”

This allows for, among other things, competitor searches and custom HubSpot URLs (for example, if you just wanted to see who mentions our Forms product page or who mentions Academy pages – it’s also a workaround to check affiliate links, because they’re often structured like “affiliatesite.com/ref/hubspot” so you could just enter “hubspot” in the URL to check for those):

We use SEMRush’s API to pull keyword search volume, but there are times that SEMRush doesn’t have values for search volume. In this case, it just prints an error message:

If all information is entered correctly, click “Submit” and it will start computing. After some processing time, this is the report it will create:

You get a nice visual pie chart, beloved by analysts and data visualizers worldwide, representing how many ranking URLs contain the link (and how many don’t). This shows that, of the top 20 search results for the term “best live chat software,” our URL “hubspot.com” appears on 2 of them.

I also made it pull keyword level data as well (in the above example it shows that “best live chat software” gets 390 searches a month). This helps weigh the relative importance of a given keyword (ie if a keyword gets 10000 searches a month or 100, that matters strategically)

Then below that, you get a full data table that contains the domain and URL of all pages that rank for that keyword, whether or not the link you chose to check against is included (TRUE or FALSE), the keyword you searched (just for reference and if you want to pull the data to Excel), the number of links included, and the full link (so if you checked “hubspot.com” and the ranking page references “blog.hubspot.com/marketing/form-builder-tools,” it will pull that full link URL for you):

Like I mentioned above, you can use this to check other websites’ links as well. This one shows how many of the top 20 ranking pages for “mailchimp alternatives” mention appsumo.com (which makes SendFox, an email marketing tool):

And like I mentioned above, you can choose the region. Currently, I have options for Germany, France, Spain, and Japan, though we can add more regions fairly easily. This is going to be incredible for international expansion (more on that later)

This just changed the database to a different region (ie if you choose German, it will show what ranks in google.de). This report shows how many hubspot.com links appear on pages that rank for “marketing automation software” in Germany:

Next Level: Automating Report for 600+ Keywords with Google Data Studio and Scripts

This data from the SERP Tracker, on its own, is valuable. One can quickly audit the search results pages for product mentions, compare to other websites, and even do a bulk audit.

However, to truly get the wheels turning on the surround sound Strategy, it’s imperative to store and access data across time in order to track results and identify gaps and opportunities.

To do this, we extract data from the SERP Tracker, transform it and blend it, and load it to BigQuery so we can visualize it and create reports using Google Data Studio.

All of this is done automatically with an R script that runs once per month. It pulls all SERP data for our list of keywords, stores the data in Sheets (in the future we’ll write directly to BigQuery). BigQuery is connected to Sheets, so we filter, clean, and blend the data with a quick SQL script. This is then connected to Data Studio, where all filters are applied and we visualize our reports.

Our reports start with the way we map our target-keywords against the features, products and Hubs. This database is the source for the SERP tracker as well and the Google Data Studio reports which we use to track progress and communicate with stakeholders.

First, we use a birds-eye view of all of our product Hubs to understand overall progress and how each Hub stacks against the others. If you’re familiar with HubSpot’s products, it won’t come as a surprise that our Sales and Marketing Hubs lead the way when it comes to surround sound Visibility.

Our reports go one layer deeper and we can see in one chart how different features (e.g. HubSpot Email Marketing vs HubSpot Forms) perform against each other.

For a deeper analysis, we have feature-dedicated dashboards. In the example below, we see how one isolated feature (HubSpot CMS) evolved, as well as how our visibility changed for the most important keywords related to that feature.

1. Surround Sound: The First 6 Months

The main thing that you need to accept before starting a surround sound program is that you can never be fully in control of your visibility.

There are many factors that play a role into how your visibility evolves:

• Search engine updates cause the Top 20 results to fluctuate (sometimes quite dramatically)
• New content appears in Top 20 for the keywords that you’re focusing, maybe outranking you or results that mention your brand
• Competitors and other third-party pages appear in Top 20 results, but don’t mention your brand

Even despite these challenges, the results we’ve seen in the first 6 months of our surround sound program prove you can make a change in your visibility by partnering with other websites.

If you want to learn more about how to execute a surround sound program, check out the previous article in this series.

In the chart below you can see how HubSpot’s visibility in SERP changed since April. The Google May 2020 core update changed the SERP landscape quite dramatically, causing a lot of articles that used to mention HubSpot to disappear from the Top 20. After that, our visibility started climbing gradually, as we built more partnerships (e.g. more websites joined our Affiliate program).

If we only look at all the pages that mention HubSpot, for the 600+ keywords we’re monitoring for surround sound, about 50% of these are now influenced by our surround sound program.

Conclusion: Final Thoughts

There is still a lot of work and opportunities to tap into for our product awareness program. But the most important learning we had so far is that surround sound is a marathon, not a sprint.

It takes time and effort to build genuine partnerships and, as Brian Halligan always “add value > extract value”.

Also, having a strong brand name helps a lot, at least when first reaching out to a potential partner. In fact, it’s fair to say that if HubSpot were a startup, this progress wouldn’t have been possible in just 6 months. It takes time to build a brand and get others to recommend it.

As with everything, there’s no silver-bullet solution. While we share a lot of our process and tools, the way we do surround sound at HubSpot might not work exactly the same for other brands. The main thing we hope you take away from this is: there’s another way of looking at your visibility in SERP that’s not just SEO.

Have you thought about adding surround sound to your content marketing and acquisition strategies in 2021?

Want to learn more about the strategy and execution behind the surround sound strategy? Read part one and two of this series.

Originally published Nov 5, 2020 7:00:00 AM, updated November 05 2020