Bluerivermountains.com - Blog (EN)

7 Complexities of Implementing Meta's Conversions API

Fri, 24 Nov 2023 00:00:00 GMT

7 Complexities of Implementing Meta's Conversions API

Matthias Kupperschmidt

undefinedmin read

Welcome to the world of Meta's Conversions API, where the landscape is far more intricate than mere server-side data transmission.

I literally have people reaching out asking if we could implement this next Monday, hence this deep dive unveils the scope of implementing this tracking solution in reality.

I hope to reveal why it's an endeavor that goes beyond initial expectations.

1. The Deceptive Simplicity of the API

Many perceive Meta's Conversions API implementation as a straightforward process, but this perception is misleading.

The API's true power lies in its ability to attach detailed user data—like names, contact details, and location—to events, a feature demanding far more effort than initially anticipated.

Spooky? I know. Needless to say that a proper cookie management platform (CMP) has to collect tracking consent and have it be a requirement before leveraging marketing tags in the

tag management system

2. Technical Hurdles and Data Considerations

As a general requirement, Google Tag Manager server-side tagging should be set up. A custom sub-domain for the server is not a must, but strongly recommended.

When it comes to Meta's user data parameters, the devil is in the details.

Implementing the API involves navigating complex requirements like:

Phone number normalization
SHA256 encryption
ISO-formatting for countries, towns
Event de-duplication between front- & backend hits

This often leads to necessary post-implementation fixes due to:

Incorrect formatting
Mismatched data, especially with null or undefined data fields

3. Upgrading the Data Layer

A critical step in this journey is therefore to update the

data layer

to ensure user data is readily available in the right format when triggering Facebook events. This upgrade is not a must, but recommended.

Alternatively you can implement a logic that collects (scrapes) and persists user data throughout the user journey.

Providing the correctly formated data fields is essential for harnessing the full potential of the API, so it adds another layer of complexity to the implementation process.

4. Leveraging Google Analytics Tags to Use the FB Conversions API

While the current FB web tags fall short in leveraging the Conversions API, there's a workaround. You can utilize GA4 tags to send the necessary data to your own GTM Server, which includes:

Event data
Ecommerce data
User data

Consequently, it's imperative to update your current

Google Analytics Setup

in GTM. Ensure it runs through your GTM server and includes all required data fields for Facebook's API.

5. Timeframe and Project Scope Realities

If you're marking your calendar for a quick turnaround, think again. Contrary to the optimistic one-day projections, a realistic timeline spans around a few weeks, plus or minus a few days.

This duration is necessary, considering that feedback and validation of data fields occur within Facebook's Ads interface, which takes up to 48 hours to update warning and info notifications about your events.

6. Feedback and Corrections: A Time-Consuming Endeavor

Implementing the API is akin to a meticulous grind. Facebook's interface takes about 36 to 48 hours to reflect changes in their info warnings, turning the process into a cycle of make-change-wait-and-review.

This iterative process is crucial for ensuring data accuracy however.

7. Expanding the Scope Beyond Facebook

Once you've navigated the waters of Facebook's API, leveraging your efforts for other platforms like TikTok and Google Ads, which also support server-side user data transmission (TikTok Events API & G Ads Enhanced Conversions).

Data validation across multiple platforms significantly ramps up the complexity, requiring a keen eye on data formatting, error-handling and falsy values across all channels.

Conversion Rate Improvements and Business Impact

Despite these complexities, the payoff can be substantial. Implementing the Conversions API has shown to improve conversion rates by a minimum of 5%, with some cases soaring as high as 12-13% due to its advanced matching capabilities.

This improvement varies across different niches but underscores the tangible benefits of this technical undertaking.

Conclusion

Reflecting on the sheer scale of implementing Meta's Conversions API, it becomes clear that this is not a task to be underestimated.

It demands a thorough contemplation of the effort involved versus the potential gains. Make your decision wisely, considering both the complexities and the promising rewards.

Resources

Matthias Kupperschmidt

undefinedmin read

Track Google Form Views with GA4

Wed, 12 Apr 2023 00:00:00 GMT

Track Google Form Views with GA4 & Marketing Pixels

Matthias Kupperschmidt

undefinedmin read

Do you want to know how many visitors visit your Google Form? - In this tutorial I explain how to use Google Analytics to track visits to a Google Form to measure engagement and form views. After that, we'll learn how to use the visits for Google Ads campaigns.

I'll also show how, using the same principle, you can implement other Tracking Pixels from Facebook, Linkedin or Twitter on the form to target visitors through Retargeting Campaigns, for example.

As a

Google Analytics Consultant

I have set up tracking for Google forms quite a few times. So I will explain you why it is smart to track your form with additional tracking tools and what the advantages are.

Why is it important to track Google Form attendance with GA4?

Google Forms are an attractive solution to create online forms for websites. By March 2023, Google Forms is used by 0.26% of websites across the Internet as a solution for feedback forms. Popular websites using Google Forms include washingtonpost.com, techcrunch.com, and usatoday.com.

Why?

The advantages of Google Forms are:

Free with no limits
Forms can be created quickly and easily in minutes
Free upload feature
Many templates available
Compatible with desktop and mobile devices
Instant access to entered data in Google Sheets

Tracking the individual views of a Google Form with Google Analytics or Marketing Pixels from Facebook, Linkedin or Twitter is not possible without overcoming a few technical hurdles.

Yet it is worth it!

Tracking individual page views of Google forms has compelling advantages:

In Google Analytics 4, you can link your conversions to your Google Ads account, for example, to run retargeting campaigns. This means that if you configure the form visit as a conversion (!), the visitors will be added to a Google Ads segement for campaigns. Thus, you can later show promotional offers to the visitors. So, for marketing purposes, the visitors of the form are completely available.
You get to know how many visitors visit the form. If you subtract the number of completed forms from the visits, you know the potential of the campaign. GA4 engagement tracking additionally tells you how much time visitors spend on the form - which helps to improve the form in case visitors spend too much time filling it out.
By tracking with Facebook or the Linkedin tracking pixel, you can learn more about the interests or demographics of your visitors, thereby improving communication with your customer segments.

So if you combine the general advantages of Google Forms with the benefits of tracking with Marketing Pixels, Google Forms becomes a super efficient & free solution for digital campaigns with contact forms!

How to Track Google Form Visits with GA4

View Tracking for Google Forms in GA is achieved by embedding the Google Form Iframe together with the Google Analytics Tracking Code on the same page.

Since Google Analytics tracks every load of the website as a page view and the page contains the form, the views to the form can thus be tracked as well without having direct access to the form's code.

Simple, right?

Let me explain how you do this exactly, whether you have your own website or not.

Measure attendance of Google Forms through embedding

First of all, you should always embed Google Forms in a webpage, as this will allow you to insert your own Google Analytics tracking code into the page to measure the number of visitors to the form. I repeat: you should not send the direct link to the Google Form to your users.

If you use the Google form embed code on your own website and also add your Google Analytics snippet, every visit to the form will be recorded as a page view in Google Analytics. This gives you the advantage of being able to track the form attendance.

In addition, you can now also implement Tracking Pixel for Google Ads, Facebook, Twitter or Linkedin on the page - which would not be possible otherwise.

Don't have a website where you can embed the Google Form? That's no problem. You can create a hosted HTML page in seconds and embed the Google form in it.

For example, use the Basic Website starter from glitch.com, Github Pages or simply upload your HTML file to Netlify. Below I give an example of Glitch and Netlify.

1. Embedded Google Form with GA tracking on custom website

First, copy the embed code for your Google Form.

2. Create a single custom web page

If you already have your own website, create a new page with any address path and add the embed code. If you don't have your own website, follow one of the examples below.

First create an empty HTML page with a default boilerplate codeand add the Google Form snippet and the Google Analytics tracking code. Then upload the file to a web host.

Glitch Example

Start a new web page with the Basic Website Starter from glitch.com and add the embed code for the Google Form at the top of the <body>. Delete the rest of the code in the body tag.

Add Google Analytics 4 Snippet: Now add the Google Analytics snippet.

Alternatively, you could also add a GTM container snippet with a GA4 page view tag enabled.

Copy the new link to your Google Form: Next copy the link to your glitch page.

Under Preview -> Preview in a new Window you get the created page opened in a new browser tab. From now on use this link to invite visitors to your form.

Netlify Drop Example

Create a HTML page from a boilerplate template and add the copied embed code of the Google Form to the <body>. Then save the file as index.html in an empty folder.
Visit Netlify.com and log in with your preferred login service.
After that, go visit Netlify Drop and drag the folder with the index.html file into the drag & drop area.

3. Test if everything works

To test GA4 tracking you can simply visit the real-time report. On the events widget you can click the page_view event and then select the page_location parameter. This will show you all web addresses of the last minutes.

There you should see the URL to the page with the Google Form.

Analysis of form views in GA4

GA4 tracks engagement metrics like average engagement time by default.

When the form is called, the engagement tracking starts automatically, so you can look at the average engagement time (time tracking) of the visitors with the form later in Google Analytics.

To create a time tracking report you then build e.g. an Exploration Report with the values User Engagement and as dimension for the rows the Page Path.

Also add views to the values to show the total number of form visits (form attendance).

Track Google Form Visits with the Facebook Pixel and other Marketing Pixels (Twitter, Linkedin etc.)

Also, you can insert all sorts of marketing tracking pixels on the page as well. See here an example for the Facebook Pixel. The FB snippet is simply inserted after the GA4 snippet.

After you have implemented the Facebook, Twitter, Pinterest, Linkedin or any other pixel, you need to test if the implementation worked.

The easiest solution is to log in to the respective platform of the marketing pixel. After 24 hours at the latest you should see traffic there.

If you don't want to wait that long, there are extensions for most tracking pixel browsers that show information about the pixel when it is working.

For Facebook there is the Meta Pixel Helper.

After the extension is installed in the browser and we call the Google Form with the implemented FB Pixel, the Pixel Helper confirms that the tracking is working.

There are also similar Pixel Helper extensions for Linkedin and Twitter to check the functionality of the pixel.

Conclusion

I hope I could convince you of the importance of embedding the forms, so that you take the extra effort in the future instead of sharing the direct link.

Especially for YouTube or email campaigns, this way of quickly creating a landing page that is visually the same but offers many advantages in terms of marketing is a good idea.

We also looked at how to co-implement other tracking tools on Google Forms so that you have more metrics available for data analysis 😎.

Matthias Kupperschmidt

undefinedmin read

Blocking GA4 pageviews with URL parameters

Fri, 10 Mar 2023 00:00:00 GMT

Block page view with query parameter in Google Analytics

Matthias Kupperschmidt

undefined min read

Learn in this tutorial how to exclude pageviews from Google Analytics tracking when they have unwanted URL query parameters.

Any

query strings

in Page URLs or JavaScript redirects can cause tracking issues. For example, a CMS might embed user data in the URL, which can lead to limitations in analysis and expose user data to all services on the website. Therefore, you can simply exclude such pageviews from tracking by their URL parameters.

In my previous tutorial, we already discussed

how to filter query parameters from the URL

, so that the parameters no longer appear in Google Analytics reports.

If you don't want to have such pageviews in your analytics data at all, I will now show you how to completely block tracking for certain query parameters.

And - I will show you two different methods to achieve that!

The first method is easy and only requires the use of Google Tag Manager in combination with a special trigger configuration that checks for the presence of the parameter.

The second method achieves the same result in a different way, and therefore we will also look at it.

Both methods work with a single URL parameter or with a list of query parameters.

How to block Google Analytics pageviews with URL query parameters?

To illustrate the following methods, let's assume that we want to block all pageviews that contain the exclude=true parameter.

We can either add a condition to a pageview trigger or alternatively, we can create a blocking trigger. Let's look at both options 🤓.

Adding query parameter conditions to a pageview trigger

In Google Tag Manager, there is a built-in variable called Page URL. This variable contains the entire browser address as a string, which means that we can check it against any query parameters in the trigger configuration settings.

To do this, go to your triggers in the GTM container and create a new pageview trigger. Select that this trigger should only fire on some pageviews.

Next, select the Page URL variable from the dropdown menu and add the "does not contain" rule, along with the example parameter mentioned above, exclude=true.

If you click save now, you have created a trigger that fires on all pageviews, except those that contain the query parameter 💪.

Next, use this trigger for the Google Analytics 4 tag that sends the event for pageviews. In most cases, this is the GA4 Configuration tag, if the checkbox for "Send a page view event when this configuration loads" is checked there.

Done.

How to Stop GA Tracking for a List of Query Parameters?

If you have a list of query strings for which tracking should be suspended every time they appear in the web address, then set the trigger with a Regex matching rule. Don't worry, you don't have to learn Regex specifically for this.

Regex (short for Regular Expressions) is a sequence of characters that describe a pattern in a string, which can be used to search text and perform an action when there is a match.

Creating Regex Rule for URL Parameters

Let's assume we want to exclude all page views that contain the following query strings from web tracking:

exclude=true
debug=4387969876
userName=hans.hansen
email=hans@hansens.com

Multiple strings can be concatenated in Regex using the pipe symbol.

Since the values for the variables debug, userName, and email are dynamic, we also include a wildcard in the Regex rule:

Regex matching rule: exclude=true|debug=.*|userName=.*|email=.*

Using GTM Trigger with Regex Rule

Next, we create a new trigger in Google Tag Manager in which the Page URL does not match the regular expression.

Afterwards save the trigger and apply it to your page view tag. Finished.

Create blocking trigger with URL parameter

The second method is to add a blocking trigger to an event or page view tag.

The difference is that with the first method, you change the general trigger for page views by specifying the URL parameters. However, this change can lead to complications due to effects on other tags with the same trigger 🙄.

Therefore, it is usually easier to create a single blocking trigger and then add it to the tag as a trigger exception.

So in the second method, we leave the page load trigger as is and just add our blocking trigger to the tags where we want the query strings to stop the tag.

The advantage of this method is that in a GTM container with many tags, you can easily add an additional block trigger to any tag without affecting other tags. This is better than with the first method, where we make changes to the general page view tag and all other tags with the same trigger are also affected.

To create the blocking trigger, follow the same steps as described above for the page view trigger.

After that, visit the tag that should not be executed on page views with parameters.

Then click on Exceptions in the trigger section and add the trigger as an exception.

You can hence continue to use e.g. a general page view trigger ("All Pages") as the regular trigger for the tag in combination with the exception. The tag will always be executed unless the condition with the query parameters in the exception trigger applies.

After that, click save and publish the GTM container.

How to block events other than page views based on query strings

It is also possible to block events other than page loads based on query parameters in the URL.

To do this, the type of the blocking trigger must be modified to respond to any type of event. Then, the modified blocking trigger can be used for all possible events in Google Tag Manager.

If the associated URL sent with the event contains the defined query parameter, the event will be blocked as stated.

To do this, change the previously created trigger and set the trigger type to "Custom event". As event name write .* and check the box "Use regular expression match".

We use a Regex wildcard for the event name, because this way any event name will be accepted.

Now set this trigger on any tag if you don't want it to fire when the defined query parameters are in the URL.

Conclusion

In this article, you have now learned how to stop page views in Google Analytics through multiple methods.

The most important thing in this tutorial is to learn how to use negative conditions in GTM triggers or trigger exceptions (blocking triggers).

If Regex was new to you, I recommend taking a closer look at it, as it is a great help for data filtering and is supported in a large number of tools 🤓.

FAQ

How do I exclude a URL from Google Analytics 4?

To exclude a URL from GA4, utilize Google Tag Manager (GTM) and set up a trigger that checks for specific query parameters in the URL. If the URL contains unwanted parameters, the trigger prevents the pageview from being tracked.

How do I block a URL in Google Analytics?

In GTM, create a pageview trigger that fires only when certain conditions are met. Use the "Page URL" variable and set a rule like "does not contain" followed by the undesired parameter, such as "exclude=true". This ensures the trigger doesn't fire for URLs with that parameter.

Can you exclude pages from Google Analytics?

Absolutely! By leveraging GTM and its trigger configurations, you can exclude specific pages from being tracked in Google Analytics based on their URL parameters.

How do I exclude a landing page in Google Analytics?

To exclude a landing page, set up a trigger in GTM that checks the "Page URL" variable. If the URL matches the landing page's address or contains specific query parameters, the trigger won't fire, thus excluding the landing page from tracking.

How do you exclude page views in Google Analytics?

There are two primary methods: a. Add conditions to a pageview trigger in GTM, ensuring it doesn't fire for URLs with certain parameters. b. Create a blocking trigger that acts as an exception for specific pageview tags, preventing them from firing when certain URL parameters are present.

How do I disable page views in GA4?

To disable pageviews in GA4, use GTM to set up triggers that check for specific URL parameters. If these parameters are detected, the triggers prevent the GA4 tag responsible for sending pageview events from firing.

What are the issues caused by any query strings in Page URLs or JavaScript redirects?

Query strings in Page URLs or JavaScript redirects can cause tracking issues. For instance, a CMS might embed user data in the URL, which could result in limitations in analysis and expose user data to all services on the webpage.

How can I exclude pageviews from tracking by their URL parameters?

You can exclude such pageviews from tracking by their URL parameters. This process involves adding certain conditions or creating a blocking trigger in Google Tag Manager.

How can I completely block tracking for certain query parameters?

You can block tracking for certain query parameters by either adding a condition to a pageview trigger or creating a blocking trigger. These two methods allow you to block all pageviews that contain a specified parameter.

What is the first method to block tracking for certain query parameters?

The first method requires the use of Google Tag Manager in combination with a special trigger configuration that checks for the presence of the parameter. This method involves creating a new pageview trigger and setting it up to fire only on some pageviews that do not contain the specified parameter.

What is the second method to block tracking for certain query parameters?

The second method achieves the same result as the first but in a different way. Instead of changing the general trigger for page views, you create a single blocking trigger and then add it to the tag as a trigger exception. This method is usually easier and does not affect other tags with the same trigger.

How does the first method work?

The first method uses a built-in variable in Google Tag Manager called Page URL. This variable contains the entire browser address as a string, allowing you to check it against any query parameters in the trigger configuration settings. By selecting the "does not contain" rule, you can create a trigger that fires on all pageviews, except those that contain the specified query parameter.

How to block Google Analytics pageviews with URL query parameters?

To block Google Analytics pageviews with URL query parameters, you can either add a condition to a pageview trigger or create a blocking trigger. For the first option, you create a new pageview trigger in Google Tag Manager that does not contain the specified parameter. For the second option, you create a blocking trigger and add it as an exception to the tag.

How do you stop GA tracking for a list of query parameters?

To stop GA tracking for a list of query parameters, you need to set the trigger with a Regex matching rule. Regex (short for Regular Expressions) is a sequence of characters that describe a pattern in a string, which can be used to search text and perform an action when there is a match. This allows you to create a rule for multiple query strings and apply it to your page view tag.

What is the second method to add a blocking trigger to an event or page view tag?

The second method involves creating a blocking trigger and adding it as an exception to the tag. This method allows you to block the tracking of certain query parameters without affecting other tags in the Google Tag Manager container.

How do you block events other than page views based on query strings?

It is possible to block events other than page loads based on query parameters in the URL. To do this, the type of the blocking trigger must be modified to respond to any type of event. This involves setting the trigger type to "Custom event" and using a Regex wildcard for the event name to accept any event name.

What is the most important thing to learn from this tutorial?

The most important thing in this tutorial is to learn how to use negative conditions in GTM triggers or trigger exceptions (blocking triggers). This skill is essential for data filtering and is supported in a large number of tools.

What is the advantage of the second method (creating a blocking trigger)?

The advantage of the second method is that it allows for more flexibility. In a GTM container with many tags, you can easily add an additional block trigger to any tag without affecting other tags. This is a more efficient approach as compared to the first method where changes to the general page view tag could potentially affect all other tags with the same trigger.

How to create a blocking trigger with a URL parameter?

To create a blocking trigger with a URL parameter, you follow the same steps as for creating a page view trigger. Then, you visit the tag that should not be executed on page views with parameters, click on Exceptions in the trigger section, and add the trigger as an exception. This allows you to continue using a general page view trigger as the regular trigger for the tag in combination with the exception.

Matthias Kupperschmidt

undefinedmin read

Exclude URL Query Parameters in Google Analytics

Sat, 22 Oct 2022 00:00:00 GMT

Exclude Query Parameters in Google Analytics 4

Matthias Kupperschmidt

undefinedmin read

In this tutorial you will learn how to exclude query parameters from web addresses in Google Analytics 4.

However, I have another tutorial in case you want to completely

block page views from GA tracking because of their URL parameters

You can also use this tutorial to clean other URLs (e.g. video or link addresses) from query strings.

To do this we will create a filter functionality with Google Tag Manager (GTM). It will filter any parameter from a URL string that we define.

If you are looking for a solution to not send personal data via query parameters to your GA4 property, then read further down below under Redact Personal Information in GA4.

This guide requires that you use GTM for your tracking with GA4.

Before we start solving the problem let’s first define what query parameters are to be aligned.

What is a query parameter?

Query parameters are part of the web address for a web page.

A query parameter starts with ? behind the usual web address and assigns values to variables. Several query parameters result in a query string.

Programmers sometimes use query strings so a server can tell from the parameters that it should serve a modified version of the requested web page.

For example, a query string might look like this:

?site=bluerivermountains.com&referrer=google.com

The entire web address including query parameters then looks like this:

https://bluerivermountains.com?site=bluerivermountains.com&referrer=google.com

With that cleared up, let's now learn how to create a filter function for query parameters in GA4 with the help of Google Tag Manager (GTM).

What causes query parameters in the Google Analytics reports?

User input

Every time a web page is loaded, the GA4 library sends the URL of the page to Google's server with an event.

That means that any visitor can theoretically insert a parameter into the URL and thus send it to your GA reports.

Try it. Add the following query parameter to the end of any web address in the browser and press Enter:

?testParameter=true

The page will load without problems in most cases. And so the parameter was also sent to Google Analytics when the page was loaded.

Tracking Services

Many tools and services use parameters in URLs to track clicks on links.

Examples are e.g. Google Ads, where the parameter ?gclid= is added to the link when clicking on an advertising link.

Another example are

UTM parameters

, which are used for campaign tracking in Google Analytics.

The only reason these parameters don't show up in Google Analytics reports is that they are automatically filtered out as the parameters are part of Google's tracking system.

So all other non-Google parameters of Tools & Services are not automatically filtered out. There are countless examples, since parameters for tracking are a common solution.

Here is a short list of tracking services and the associated query parameters:

Tracking Service	Query-Parameter
Bing Ads	?msclkid=
Facebook Ads	?fbclid=
Google DoubleClick	?gclsrc=
Adobe Analytics	?s_kwcid=
Klaviyo	?_ke=
Hubspot	?hsa_cam=
Ebay	?mkcid=

So if any service adds its custom tracking parameter to a link to your site, you will later find it in the Google Analytics reports.

Why are query parameters a problem?

Parameters are not necessarily a problem. But in some scenarios they generate one. For example:

Data privacy issues

Many website systems use parameters during the registration process to send user data to the backend.

If such websites are tracked with Google Analytics or the Facebook pixel, you are now automatically breaking Google's and Facebook's terms of use, because you send private data to their servers via the query parameters in the web addresses.

Afterwards you either get warnings or, in the worst case, you have to expect the your account to be blocked. On top of that you are also breaking EU data protection rules (GDPR).

Problems with Facebook event matching

If you operate Facebook tracking via the browser and via the server using Facebook's Conversion API, tracking data must be deduplicated. Among other things, for the deduplication the web addresses are used. Query parameters that contain personal data are often only filtered out in one of the two data sources, i.e. either in the browser or on the server. Thus, the event matching scores on Facebook plummet.

Problems with data analysis

Tracking tools mostly treat URL's as ordinary strings. This means that parameters are not filtered out automatically. This creates problems in data analysis since data for the same page path is not grouped.

See the following table as an example:

Video URL	Views
https://myvideos/coolSpring?kjh1249nnj=1	3
https://myvideos/hotSpring?kasd1249nnj=12	1
https://myvideos/hotSpring?123456=true	9
https://myvideos/coolSpring?kjdkj49nnj=asasjhb328	43
https://myvideos/hotSpring?k123nj=false	2
https://myvideos/coolSpring?asf45nj	15

As you can hopefully see in the table above, it's difficult to calculate the sum of views for a video when the video URL's contain parameters.

How to exclude query parameters in GA4

How can we filter out query parameters from web addresses in Google Analytics 4?

First we will create a JavaScript variable that will clear the entire query parameter if there is a previously defined parameter in the URL. We then send the URL to Google Analytics without query parameters.

Here we go!

1. Delete query parameters in GTM

To remove the question mark ? at the end of the web address and the rest of the query string, the first thing we will do in Google Tag Manager is create a new custom variable of type "Custom JavaScript" called Page Location - Custom

Next, let's add the following custom JS code:

function() {
    // define parameters to exclude
    var excludeStrings = [
        "hsa_acc",
        "fbclid",
        "wbraid",
        "hsa_cam",
        "hsa_grp",
        "hsa_ad",
        "hsCtaTracking",
        "submissionGuid",
        "hsa_src",
        "hsa_tgt",
        "hsa_kw",
        "hsa_mt",
        "hsa_net",
        "hsa_ver",
        "li_fat_id",
        "q",
        "msclkid",
        "ref",
        "cache",
        "_x_tr_sl",
        "_sm_nck"
    ];
    var addressString = new URL(document.location);
    var queryString = addressString.search;

    // check if query string holds any parameters, otherwise just return the url without them
    if (queryString.indexOf("?") != -1) {
        // https://stackoverflow.com/questions/901115/how-can-i-get-query-string-values-in-javascript
        var getQueryParamsFromURL = function getQueryParamsFromURL() {
            var match,
                search = /([^&=]+)=?([^&]*)/g,
                decode = function decode(s) {
                    return decodeURIComponent(s);
                },
                query = addressString.search.substring(1);

            var urlParams = {};

            while ((match = search.exec(query))) {
                urlParams[decode(match[1])] = decode(match[2]);
            }

            return urlParams;
        };
        // create param object from query string
        var urlParams = getQueryParamsFromURL();

        // if it holds any of the defined parameters, remove the key and keep the rest
        Object.keys(urlParams).map(function (key) {
            if (excludeStrings.includes(key)) delete urlParams[key];
        });

        // Create filtered query string
        var queryString = new URLSearchParams(urlParams).toString();

        // add ? to querystring unless it's empty
        if (queryString != "") queryString = "?" + queryString;
    }

    // return cleaned URL
    return addressString.origin + addressString.pathname + queryString;
}

Now look at the third line of code, which defines the excludeStrings variable:

An array with a list of parameters for filtering is defined. Each of these strings represents the name of a query parameter. If one of the parameters appears in the web address, it will be deleted.

The rest of the URL and the query string remain intact so that important parameters such as gclid parameters (Google Ads) or UTM parameters (campaign tracking) are not accidentally deleted.

The above parameter list from the code snippet resulted over time from Hubspot and Facebook parameters. If you want, delete all parameters and then add your own parameters. However, note the syntax: "parameter1", "parameter2", "parameter3"etc.

For example, if you wanted to add a personal parameter named myPersonalParam, the array would look like this (see end):

    // define parameters to exclude
    var excludeStrings = [
        "hsa_acc",
        "fbclid",
        "wbraid",
        "hsa_cam",
        "hsa_grp",
        "hsa_ad",
        "hsCtaTracking",
        "submissionGuid",
        "hsa_src",
        "hsa_tgt",
        "hsa_kw",
        "hsa_mt",
        "hsa_net",
        "hsa_ver",
        "li_fat_id",
        "q",
        "msclkid",
        "ref",
        "cache",
        "_x_tr_sl",
        "_sm_nck",
        "myPersonalParam"
    ];

We are nearly finished. Go on.

2. Set GA4 configuration

Next, let's go to the tags in our Google Tag Manager container and open the GA4 configuration tag.

Now add the field page_location to the fields to be defined and define our created JavaScript Page Location - Custom as the value:

This setting overrides the web address we send to Google Analytics with our own custom web address.

That means, if the query parameter name was specified in the code snippet, this custom web address no longer contains the parameter.

Finished.

Filter out all query parameters in GA4

The previous solution is based on the assumption that we have no way of knowing which parameters will be added to the URLs and whether or not they should be filtered out.

The analyst must first notice the parameters in the GA4 reports, then decide if they need to be removed and finally expand the parameter list in the JS variable.

On the one hand, the solution gives the user control, since each filtration is considered at least once; on the other hand, updating the list can be cumbersome.

An alternative solution is to filter out all query parameters, except for the one required by Google Analytics (namely gclid- and utm- campaign parameters).

However, the disadvantage of this approach is that control over the filtered out parameters is lost. Actually, you only know which parameters are retained (gclid and utm parameters) and nothing more.

Such a solution ensures consistent web addresses in Google Analytics without manual effort. Therefore, if you are willing to give up some control, you can put the following JS code in the Page Location - Custom variable instead of the above script:

function() {
    // define parameters to keep if available
    var includeStrings = [
        "gclid",
        "utm_",
        "gtm_debug"
    ];
    var addressString = new URL(document.location);
    var queryString = addressString.search;

    // check if query string holds any parameters, otherwise just return the url without them
    if (queryString.indexOf("?") != -1) {
        // transpile ES2016 => ES2015
        var _defineProperty = function (obj, key, value) {
            if (key in obj) {
                Object.defineProperty(obj, key, {
                    value: value,
                    enumerable: true,
                    configurable: true,
                    writable: true
                });
            } else {
                obj[key] = value;
            }
            return obj;
        };

        // https://stackoverflow.com/questions/901115/how-can-i-get-query-string-values-in-javascript
        var getQueryParamsFromURL = function getQueryParamsFromURL() {
            var match,
                search = /([^&=]+)=?([^&]*)/g,
                decode = function decode(s) {
                    return decodeURIComponent(s);
                },
                query = addressString.search.substring(1);

            var urlParams = {};

            while ((match = search.exec(query))) {
                urlParams[decode(match[1])] = decode(match[2]);
            }

            return urlParams;
        };

        var filterParamsFromList = function filterParamsFromList(obj, list) {
            var urlParamKeysFinal = [];
            var urlParamKeys = Object.keys(obj);

            // test each param for availability and create array with final keys
            for (var i = 0; i < list.length; i++) {
                urlParamKeysFinal.push(
                    urlParamKeys.filter(function (key) {
                        return key.includes(list[i]);
                    })
                );
            }

            // merge all keys into one list
            // https://stackoverflow.com/questions/10865025/merge-flatten-an-array-of-arrays
            urlParamKeysFinal = [].concat.apply([], urlParamKeysFinal);
            return urlParamKeysFinal.reduce(function (cur, key) {
                return Object.assign(cur, _defineProperty({}, key, obj[key]));
            }, {});
        };

        // create param object from query string
        var urlParams = getQueryParamsFromURL(); // Create filtered query string

        queryString = new URLSearchParams(
            // remove any non-matching keys from param object
            filterParamsFromList(urlParams, includeStrings)
        ).toString();

        // add ? to querystring unless it's empty
        if (queryString != "") queryString = "?" + queryString;
    }

    // return cleaned URL
    return addressString.origin + addressString.pathname + queryString;
}

Note that at the beginning of the script, the variable includeStrings is defined with all the parameter names that definitely should always be kept in the URL: gclid and utm parameters.

If there are other parameters you want to be ignored, just add them to the array.

I also added the gtm_debug parameter. It signals Google Analytics when a page is visited in GTM debug mode. As a result, page views are filtered out of the GA reports during debugging.

Filter query strings from other event parameters in GA4

You can also use the above scripts for other event parameters in GA4. For example for Video URLs or URLs of external links.

Almost at the beginning of my code I define the variable addressString.

var addressString = new URL(document.location);

Now instead generate the variable from the GTM variable that outputs the video URL:

var addressString = new URL({{Video URL}});

The script will from now on remove the query strings from the video URL.

Then you can replace the {{Video URL}} in the GA4 event tag with the new JavaScript variable.

Be careful when filtering gclid parameters and UTM parameters

Gclid parameters are query parameters that Google Ads adds to the web address of the landing page when an ad is clicked. For attribution in Google Analytics, it is important that these parameters remain in the URL so that the click can be attributed to the paid search channel.

UTM parameters are campaign parameters that Google Analytics users add to links to your website. Using the utm parameter, the user can later see in the Google Analytics reports exactly which website or campaign a visitor came from.

Gclid parameters and UTM parameters are automatically filtered out by Google Analytics during data processing and are not visible in the reports. These parameters therefore do not have to be filtered out manually with Google Analytics.

Redact Personal Information in GA4

GA4 has recently released a new Gtag-feature: Data redaction for query parameters.

Meaning, if your website is picking up personal information or simply unwanted data in query parameters, you now have an easy way to tell GA4 to ignore the values of those query parameters.

Head to GA4's Admin Panel and find the new feature here:

Admin -> Data Streams -> Choose your stream -> Redact Data

This is a quick and easy way to make sure that you won't be sending personal data to Google Analytics servers.

However, it won't help you with other services like Facebook. For third-party platforms, you will still the above solution to clean URL strings before sending.

This is how the URL will look liked after GA4 redacted the values:

FAQ

How do you redact data in GA4?

In GA4, use the "Data redaction for query parameters" feature. Head to Admin -> Data Streams -> Choose your stream -> Redact Data. Define which query parameters to redact.

How do you exclude query parameters in GA4?

Use Google Tag Manager. Create a JavaScript variable that clears specific query parameters from URLs before sending to GA4.

How to strip query parameters out of URL GA4?

With Google Tag Manager, set up a custom JavaScript variable. This will remove specific or all query parameters, except essentials like gclid and utm.

Can we remove query params from URL in GA4?

Yes, using Google Tag Manager. Set up a custom JavaScript variable to filter out query parameters before sending the URL to GA4.

Why exclude URL query parameters?

Main reasons: Address data privacy concerns, avoid Facebook event matching issues, and ensure accurate data analysis.

What is the difference between query and params in URL?

Query parameters follow a "?" in a URL, assigning values to variables. They help servers deliver modified web pages based on these parameters.

What is the meaning of a query parameter?

Query parameters are part of a web address. They begin with a "?" and assign values to variables. When you have multiple, they form a query string.

Does Google Analytics track URL parameters?

Absolutely. Every time a webpage loads, GA4 sends the URL, including its parameters, to Google's servers.

What is an example of a URL with query params?

Here's an example: `https://bluerivermountains.com?site=bluerivermountains.com&referrer=google.com`

What is an example with multiple query params?

Here's that same example: `https://bluerivermountains.com?site=bluerivermountains.com&referrer=google.com`

How can a URL have multiple query parameter values?

Multiple query parameters in a URL are separated using the "&" symbol: `https://bluerivermountains.com?site=bluerivermountains.com&referrer=google.com`

How would you define URL parameters?

Query parameters are parts of a web address that start with a "?" and assign values to different variables.

Why do UTM parameters not show up in the URLs in Google Analytics?

UTM parameters don't appear in Google Analytics reports because they're automatically filtered out, being part of Google's tracking system.

How do you exclude query parameters in GA4?

In GA4, to exclude query parameters, you'd use Google Tag Manager. You'd set up a custom JavaScript variable that filters out specific or all query parameters from the URL before it's sent to GA4.

What is a query parameter?

Query parameters are part of the web address for a web page. A query parameter starts with `?` behind the usual web address and assigns values to variables. Several query parameters result in a query string.

What causes query parameters in the Google Analytics reports?

The causes are twofold: User input and Tracking Services. Every time a web page is loaded, the GA4 library sends the URL of the page to Google's server with an event, allowing any visitor to theoretically insert a parameter into the URL. Additionally, many tools and services use parameters in URLs to track clicks on links.

What should you be cautious about when filtering gclid parameters and UTM parameters?

Gclid parameters are query parameters added by Google Ads to the web address of the landing page when an ad is clicked. UTM parameters are campaign parameters added by Google Analytics users to links. Both are automatically filtered out by Google Analytics during data processing and do not need to be filtered out manually.

How does the URL look after GA4 redacts the values?

Your query params will still show up in the GA4 interface, but values are redacted.

Why are query parameters a problem?

Query parameters can cause data privacy issues as many website systems use parameters during the registration process to send user data to the backend. If such websites are tracked with Google Analytics or the Facebook pixel, you are automatically breaking Google's and Facebook's terms of use. They can also cause problems with Facebook event matching and data analysis.

How to exclude query parameters in GA4?

You can exclude query parameters in GA4 by creating a JavaScript variable that will clear the entire query parameter if there is a previously defined parameter in the URL, and then sending the URL to Google Analytics without query parameters.

How to filter out all query parameters in GA4?

To filter out all query parameters, except for the ones required by Google Analytics (namely gclid- and utm- campaign parameters), you can use a JavaScript code provided in the article.

How to redact personal information in GA4?

GA4 has a new Gtag-feature: Data redaction for query parameters. You can define the query parameters to let GA4 redact their values in the Admin Panel.

What is the purpose of query parameters in a web address?

Query parameters are part of the web address for a web page. They start with `?` behind the usual web address and assign values to variables, allowing a server to serve a modified version of the requested web page based on these parameters.

How does Google Analytics end up with query parameters in its reports?

Every time a web page is loaded, the GA4 library sends the URL of the page to Google's server with an event. This means that any visitor can theoretically insert a parameter into the URL and send it to your GA reports. Tools and services also use parameters in URLs to track clicks on links, which can end up in the reports.

What problems can arise from using query parameters?

Query parameters can lead to data privacy issues, problems with Facebook event matching, and difficulties in data analysis. For instance, if websites use parameters to send user data during the registration process, and if these sites are tracked with Google Analytics or the Facebook pixel, it results in a breach of Google's and Facebook's terms of use.

How can one exclude query parameters from web addresses in Google Analytics 4?

To exclude query parameters from web addresses in Google Analytics 4, one can create a JavaScript variable that will clear the entire query parameter if there is a previously defined parameter in the URL. Then, the URL can be sent to Google Analytics without query parameters.

What is an alternative solution for filtering out all query parameters in Google Analytics 4?

An alternative solution is to filter out all query parameters, except for the ones required by Google Analytics (namely gclid- and utm- campaign parameters). However, this approach sacrifices some control over the filtered out parameters.

How can one redact personal information in Google Analytics 4?

GA4 has a feature called Data redaction for query parameters. Users can define the query parameters to let GA4 redact their values. This feature can be found in the GA4's Admin Panel under Data Streams.

How can one filter query strings from other event parameters in GA4?

The same scripts used for excluding and filtering out all query parameters in URLs can be used for other event parameters in GA4, such as Video URLs or URLs of external links.

What is the importance of gclid parameters and UTM parameters in Google Analytics?

Gclid parameters are query parameters that Google Ads adds to the web address of the landing page when an ad is clicked. They are important for attribution in Google Analytics. UTM parameters are campaign parameters that Google Analytics users add to links to your website, helping the user to see exactly which website or campaign a visitor came from.

References

What are UTM parameters?
Google's best practices to avoid sending PII
Read more about Google Analytics view on PII data in their terms of service in Section 7, "Privacy"

Matthias Kupperschmidt

undefinedmin read

Trigger a tag only on the first page view with GTM

Mon, 18 Oct 2021 00:00:00 GMT

Trigger a tag only on the first page view with GTM

Matthias Kupperschmidt

undefinedmin read

The default page view triggers in a

Google Tag Manager setup

are not enough to cover all tag management use-cases.

Example: Let's say you want to show a pop-up to each website visitor at the beginning of their session.

For that you would need the tag to only fire on the first page view.

So in this tutorial, we will create a custom event trigger that fires only once on the first page view.

The code will be simple and we won't use any

to store the data.

How to trigger a tag only on the first page view?

We'll need a custom HTML tag to evaluate on each page view if it is the first of the session or not.

We'll store that information in the browser storage. Then we'll need a JavaScript variable to check against that value in a trigger.

Let's get started.

1. Create a custom HTML tag in GTM

First, create a new Custom HTML tag with the following code:

<script>
// check if item has been set on a previous hit
if  (window.sessionStorage)  {
    if  (sessionStorage.getItem("firstHit"))  {
        sessionStorage.setItem('firstHit',  'false');
    }  else  {
// otherwise set this hit as first page view
        sessionStorage.setItem('firstHit',  'true');
    }
}
</script>

Set the trigger type to be the all pages trigger and hit save.

Note: If your website is a single-page application then this tag should be fired on each history change event.

On the first page view of the session the code creates an object in the session storage with firstHit set to true. And on any following page views it will set the variable to false.

So when we check this value with a custom JavaScript variable in GTM, it will return true on the first page view and false on any other page view.

Let's go and do that.

2. Create a custom JavaScript Variable in GTM

In Google Tag Manager go to variables and create a new variable of type custom JavaScript and name it “First page view”.

Add the following JavaScript code:

  function() {
      return sessionStorage.getItem("firstHit");
  }

This variable returns true or false indicating if the current page view is the first page view of the session.

Now we can use this variable as a condition in our custom trigger.

3. Create custom GTM Trigger

Next, go to triggers and create a new Window Loaded Trigger that fires only on Some window loaded events.

As a condition, the First page view variable shall equal true (see below).

Click save. And we are finished.

With this custom GTM trigger you can now fire any tag or code snippet on the first page view only.

Session Storage API

For the trigger configuration in Google Tag Manager we need a data storage that tells us if the current page view is the first of the session.

Since the data shall only be persisted for the session we can use the session storage API of the browser.

Using the session Storage API to detect the first page view of a session is the more developer & marketer-friendly solution.

It requires less code than using cookies and stored data is erased automatically at the close of the browser.

In addition, the API is very simple to use.

You can inspect the session storage in your browser’s developer tools if you go to application tab > storage > session storage.

The session storage API is supported in 96.83% of browsers globally. Meaning in about 3.2% of browsers worldwide the variable will return undefined instead and hinder tag execution on the first page view.

The same solution with cookies instead of session storage has a negligible better global browser support of 0.31%.

Final words

Thats it. The code is lean and stored data is deleted automatically at the close of the browser which makes it a neat and light-weight solution.

Matthias Kupperschmidt

undefinedmin read

How to Outline a Blog Post FAST

Mon, 16 Nov 2020 00:00:00 GMT

How to outline a blog post fast

Matthias Kupperschmidt

undefinedmin read

To write a blog post outline you need to determine all relevant subtopics that cover your main-topic thoroughly.

For each subtopic you then note down a headline until all important aspects for your target audience are covered. So the outline structure will only contain headlines.

This is an example of a draft outline:

If you like, add a few content notes for a paragraph neatly indented under a heading with bullet points to give each section more depth:

But there is still one step missing during this process.

Keep search engine optimization in mind

You probably want your article to rank for a certain target keyword that is close to your main topic. Headlines are critical for search engines to understand what a blog post is about.

So when writing headlines during your outlining process, you want to strike a balance between making them engaging and descriptive, so search engines understand them and deem the topic relevant to the article.

If you neglect this step you risk not showing up in your keywords' search results at all.

But how can you make a high quality, topically relevant blog post that is SEO friendly without spending too much time?

Write a Great Blogging Outline very fast

A nifty way to determine what information to include in your blog outline is looking at the subtopics of articles that are already ranking for your keyword.

Following this approach, you save time planning subtopics and researching which to include in the article. Plus, you leverage your competitors header optimization.

If those competing articles are ranking at the top, clearly they must be doing something right about their headlines, so it makes sense to leverage their optimization.

Step-by-step the process goes like this:

Search for your target keyword
Open the best ranking results
Compare their subheading structure with each other
Draft your blog outline based on their headings
Give it your personal twist and add value with your own ideas

If you compare the subheaders side-by-side in Google Sheets or Excel, you can scan them at once and determine:

common subtopics across all blog posts
special topics that are only covered few times

The general rule to follow is, take what's commonly covered by others and choose from the special topics if you consider them to be valuable.

This is how it looks like when you put all headlines of the top-ranking blog posts for the keyword “best backlink tool” next to each other for comparison:

You see how much more structured this approach is, than visiting each piece of content yourself (and maybe getting distracted by videos, images and advertising 😂)?

Create summarizing headings

To create your own article outline, tint subheaders that cover the same information with one color and work your way through all of them.

At the end, the colors represent potential topics for the blog content and you can simply add a heading as a section to your draft by choosing from the same color.

This is how it looks like when I'm putting it together:

So by scraping the headings from the top of the search results and comparing them in a spreadsheet, you can:

plan your blog post outline FAST
make sure you cover the most important subtopics
ensure headlines are topically relevant (SEO)
find opportunities for insights others don't cover yet (since you now have an easy overview)

This is all great - except it’s a pain to manually extract subheadlines from the top search results and put them in a spreadsheet.

Since I started freelance consulting with Blue River Mountains, I automated as much as I could, since it is one of the few ways to scale as a freelancer.

That’s why I automated the process and below I’m sharing my head ripper tool with you so you can extract all subheaders in just seconds.

Competitor headline extractor tool for batch-analysis in Google sheets

Here is the link to the head ripper tool.

The tool runs in Google Colab, so let me quickly show you how it works:

This is what you need to run it:

Google account
URL's of competing pages

Step-by-Step guide to run the Tool

First, go to the head ripper tool in Google Colab and Save a copy in Drive.
Execute the first code block by pressing the play button.
Click the link in the output to get the authentication key, then paste it into the empty field and press enter. This allows Google Colab to create a Google sheet.
Search for your target keyword on Google, then copy and paste the URLs of the search result pages into the form.
Give the Google Sheet a name.
Execute the second code block by pressing the play button.
Execute the third code block by pressing the play button.
Done. Go to Google Sheets and open your newly created sheet holding all of your competitor's headlines.

Nice. Now that you have all blog sections in a spreadsheet, you can easily find commonalities and start building your own outline.

Afterwards you can get into the details and start the actual writing process.

How many subtopics should you include in the content for SEO

Group the sections of your competitors by tinting them with a color in the Google sheet. Once you did that with all of them, you know what they most-commonly talk about.

Then add the most common blocks to your own content outline to create the basic structure.

To exceed competing content, add 1-2 additional subtopics that you find relevant to add.

But how can I find additional relevant points that nobody is talking about yet?

Simpletools.io. helps you find related questions to your keyword. Answer them the ones you deem most relevant to your visitor.
Another method is to look at what Google deems highly relevant to your topic. StreamSuggest scrapes all auto-suggest keywords for you and enters each one into another Google search to scrape all of their keyword suggestions again. You get all related auto-suggestions in seconds.

Let’s wrap this up.

Takeaway

Let's take a step back a second, ignore the tool and dissect all important details to learn what the key message is.

You can achieve topical relevance often by just analyzing the sections (H1, H2, H3...) of top-ranking search result pages.
My tool allows you to scan and create a content outline fast, because you get all important subsections distraction-free to start your analysis by grouping and selecting them.
To exceed competitors blog posts, add 1-2 additional points based on your own expert advice, e.g. answer a relevant question from your personal perspective or explain another point that Google finds relevant.

That's it. Let me know on Twitter if you liked this and if it helped you to plan SEO-friendly content. ✌️

Matthias Kupperschmidt

undefinedmin read

How to log all JavaScript events to the console for debugging

Thu, 24 Sep 2020 00:00:00 GMT

How to log all JavaScript events to the console for debugging

Matthias Kupperschmidt

undefinedmin read

Let's look at how to debug all JavaScript events and log them to the console with a nifty trick.

It's for example important when debugging a web page to see how many submit events a form element sends out.

With this debugging method you prevent sending and tracking browser events multiple times.

Listen for all events in JavaScript

There is a handy API in Chrome Developer Tools that makes this easy.

The monitorEvents API allows you to log any occuring JavaScript event to the console in Chrome. You just have to give the API an element and optionally the event you want to listen to:

  //syntax
  //monitorEvents(element,event)
  
  //listen to all events
  monitorEvents(window);
  
  //stop listening
  unmonitorEvents(window);

To listen for all submit events the code would look like this:

  //start listening
  monitorEvents(window,"submit");
  
  //stop listening
  unmonitorEvents(window,"submit");

Note that if you want to stop listening for events again, you simply use unmonitorEvents.

Example: Log all click events of the window object to the console

Now with this helpful debugging tool in our toolbox, we can for example select a form element in Chrome Developer Tools to start monitoring submit events.

After you select the element in the elements tab, Chrome Dev Tools will show a little $0 sign next to it.

If you keep the HTML-Element selected, you can now use the $0 to quickly reference the element in the console - without needing a CSS selector.

Now together with the monitorEvents API you can quickly and easily listen to all ongoing JavaScript events and print them to the console for debugging:

This is helpful for all kinds of

event tracking

because you can debug every JavaScript event in the console without having to run the GTM Debugger.

So there you have it. By using Chrome Dev Tools' API, it is the quickest way to monitor all JavaScript events that I know of.

Let me know on Twitter if it worked for you and if you liked this. ✌️

Matthias Kupperschmidt

undefinedmin read

Check SERP rankings instantly with this Python script

Sun, 12 Jul 2020 00:00:00 GMT

Google SERP checker Python script

Matthias Kupperschmidt

undefinedmin read

What's really cool about on-page SEO is its feedback loop. Because, as soon as your changes are indexed, you can see an effect in Google's search results.

You either moved up a bit or down. So you learn if Google liked your changes or not.

With this learning you can apply more changes, get them indexed again and work your way steadily up the SERPs.

Nice - but there is one challenge.

For keyword optimization in a foreign country, you will need a VPN to see the local SERP exactly the way it would look like in that country.

So I did that for a while, but it turned out to be too cumbersome.

I had to change my IP manually each time and then count through the results. It took a lot of time, so I needed a better solution.

My keyword rank tracker would only give me data that was from 12 to 48 hours ago, so that was no solution either. I wanted to know my rankings now and if possible for free, because I'm a

freelance SEO

and want to keep costs in control.

Recently, I had a look at some SEO API services and noticed that many had a free tier included.

Shortly after, I also stumbled upon Google Colab to write and easily share Python scripts and enjoyed the development experience right from the beginning.

Hence, I built an SEO rank checker script in Python that would instantly check a keyword ranking. It works directly from Google Colab, so no setup-hassle.

This is how it looks like, when you fetch a SERP for a keyword in the UK:

You can use the code too if you follow below steps.

How to get started

Sign up for the free tier on Serpstack.com - The free tier allows for 100 requests per month. It’s an affiliate link. If you ever decide to move to the paid plan, I'll get a commission.
Copy the API key from the Serpstack dashboard.
Make a copy of my Google Colab Python script

Open your copy and fill out the form:

api_key - enter the key you copied from Serpstack's dashboard
target_query - your keyword
target_domain - the domain you want to find, like example.com
google_domain - the Google domain you want to search from, like google.co.uk
country_code - country code. Here the complete list following the ISO 3166 standard and a few examples:
Country Country Code
Germany de
USA us
UK gb
Denmark dk
Sweden se
language - Language code. Here the complete list following ISO 639-1 standard and a few examples:
Country Language Code
Germany de
USA en
UK en
Denmark da
Sweden sv
results - number of results to scrape, from 10 to max. 100
device - the search device (mobile, desktop, tablet)

Country	Country Code
Germany	de
USA	us
UK	gb
Denmark	dk
Sweden	se

Country	Language Code
Germany	de
USA	en
UK	en
Denmark	da
Sweden	sv

Go to the navigation bar and click on Runtime > Run all

After a few seconds a result should be logged to the bottom of the screen, that looks something like this:

How to use the script to check your website rankings?

Don't get me wrong. The idea is not to replace rank tracker tools. Among the most important SEO tools, it is necessary to have a tool that checks Google rankings accurately and immediately.

Why?

Because keyword trackers only give you the search results with a 24h delay. You don't know your website rankings now.

Why do you have to know your SERP results on-demand?

As soon as search engines index changes on your website, the page rankings update. So you want to check keyword rankings right before you publish on-page optimizations and right after - for your custom reporting.

The new organic results will indicate if your update was helpful to rank higher for your search query or not.

So this SERP checker tool is intended to be used to quickly check rank updates, and not to track rankings over time.

In my experience, this is a very rewarding methodology to work your way up into the top 10 Google results. You get feedback after each update and use the Google results themselves to tell you what works and whatnot.

Get instant SERP data for specific locations

You may have had the problem, that you work for international clients and have to check their organic results from a different city or country.

You can not simply search from your location, due to different SERPs based on the country you search from.

So the great solution to the problem is to use an API with a script that takes care of getting a local IP for the Google SERP.

You can just select the Google region you need and then let the tool check the data.

Additionally, you also get data about the related keywords, ad rankings and local pack results. And you can search with a mobile browser or get a SERP overview for desktop.

So the idea here is to make use of such services because they already solve many little problems you would otherwise encounter doing it yourself.

So there you have it. You can instantly check your keyword rankings anywhere for free.

The code is also available in this Github repository.

Let me know on Twitter how it worked for you. ✌️

Matthias Kupperschmidt

undefinedmin read