Research 101 Series
Part 3: Understanding Benchmarking
Part 1 (Distance Decay) of our Research 101 series, focused on how to use the Distance Decay report to identify the number and percentage of Locals (those who live within 0-40km from the point of interest) versus Visitors (those who live 40km+ away) coming to your location. In Part 2 (Ranking Area), you learned to identify which markets to target by ranking the top areas your guests are travelling from. The article also touched upon terms such as ‘Base %’, ‘% Pen’, ‘Index’ and ‘Benchmark’, which can be confusing if you aren’t immersed in it all of the time. Today’s article, Part 3 of this Research 101 Series, will take a closer look at these key terms.
As a quick reference/recap of these terms, included below are their definitions. By continuing to look at a Ranking Area report, we will provide a deeper understanding of these terms using various examples on why they are important for your marketing campaigns and how they are used.
Let’s review the Ranking Area report:
Name:
Will tell you where your visitors are coming from. The above example uses City (Census Subdivision) as the ranking metric.
Code:
Don’t worry about it. It is simply a code assigned by Stats Canada.
Count:
This is the number of unique households within the specific geographic area that visited the location we want information on. The table is ranked in descending order based on the highest value in this column. In the above table, the highest number of people came from Toronto.
%:
This is the percentage of the total unique visitors to your location from each area. 13.46% of total unique visitors to the location above came from Toronto.
Base Count:
This is the TOTAL number of households within each specific geographic area.
Base %:
This column represents the percentage of households in that specific geographic area (row) against the entire benchmark (the total households in Ontario which is 12,782,786). In the example above, the benchmark is Ontario. It tells us that there are 2,611,953 households in Toronto and the ‘Base %’ represents 20.43% of total households in Ontario.
% Pen(etration):
This is the percentage of total households within each geographic area that visited the location we want information on, essentially, what % of the ‘Count’ of your unique visitors are penetrating the ‘Base Count’ (‘Count’ ÷ ‘Base Count’) of households within each geographic area.
Note: There are times when % Pen is greater than 100% and that is because the Base Count is the number of households and the Count is the number of unique visitors. If four people from one household visited, that is a Count of four from one household.
Index:
Measures if the households in the specific geographic area are more or less likely to visit the location of interest when compared to the benchmark. An Index of 100 is average. Indices above 100 are above average or over-represented. Indices below 100 are below average or under-represented.
Benchmark:
A term used to describe the entire geographic area that we are analyzing the data from.
Why Benchmarks Matter?
Benchmarks allow you to quickly understand where and how to target your limited marketing and communications budget. That is, of course, once you learn how to read and understand the numbers.
Today we are going to keep it as simple as possible by focusing only on where visitors are coming from. Benchmarking identifies and ranks the cities/towns your visitors are from, allowing you to make data-driven marketing decisions to improve your ROI.
Everything you learn today can also be used to benchmark the “type” of visitors you are tracking, which allows you quickly see where more of them live in bigger numbers so that you can determine where to get the biggest bang for your marketing spend.
Setting the benchmark
Central Counties Tourism typically defaults the benchmark to all households in Ontario because it captures 96-99% of all Canadian visitors to area we are measuring. However, you can also reduce the benchmark by:
- Comparing against all Households between 40km and 100km (where the majority of tourists came from in our example)
- Comparing against all Households within 100km
- Comparing against all Households within 40km (locals)
If, once you have run your numbers, it turns out that 96+% of your visitors are from within 100kms, a more granular benchmark may be more appropriate. When you are working with Tom to create your reports, talk with him about what the best benchmark is for you.
Why CCT uses Ontario as the benchmark
There is a limitation in the data collection methodology that can skew the benchmark numbers for smaller benchmark areas. If a postal code from a town/city is present in the count it uses the entire town/city base count – even if just one or two of the postal codes are within the catchment area of the benchmark. You can see this in action in the 0-40km and 40-100km examples at the end of this article.
If you know it’s there, you can see through it. However, to a newbie, it can be confusing.
How to Read and Understand Benchmark Data
This is a real-life geofence of an area in Orangeville, which will become obvious when looking at the numbers below. We are using Ontario as the benchmark to walk you through how to read and understand the data. However, if you are curious about more granular benchmarks, we have placed them at the end of the article.
Ontario as the benchmark
What we know
Toronto has 2,611,953 total households (Base Count), which represents 20.43% of all households in Ontario (Base %). The location being researched saw 43,247 unique visitors from Toronto (Count). Those 43,247 people represented 13.65% of total unique visitors from Ontario to the location being researched.
By doing some quick math, we know that 1.66% of all Toronto Households visited the location being researched (43,247 ÷ 2,611,953). This is the % penetration value. What this tells us is that while Torontonians may represent the most visits, the location is barely scratching the surface when it comes to the total number of households in Toronto.
From the ‘%’ column, we also know that 13.65% of total unique visitors to the location being researched came from Toronto, but Toronto represents 20.43% of total households in Ontario. This leads to a low Index which is calculated: ‘%’ ÷ ‘Base %’ (13.65% ÷ 20.43% = 67).
If 20.43% of total unique visitors to the location being researched were from Toronto (rather than 13.65%), the Index would be 100. So, when you are reviewing your data, now you know to look to see if the % of unique visitors from a city (or FSA – first three postal code digits) is greater than Base % of the total households in that city as a percent of the total households in the benchmark area. If it is, that city will be over-indexed (over 100) and if it isn’t, that number will be under-indexed (under 100). When you look at the Index for Orangeville, it becomes pretty obvious that the area being studied is within Orangeville.
The number of unique visitors from Toronto is very high, however, when compared to the total number of households in Toronto (Index), it is low. This is exactly why it is important to always look at the % AND the Index. The % tells you where your visitors are coming from and how many, validating which cities/FSAs you should be marketing to. Whereas the Index will tell you the likelihood of where your visitors are coming from. In our example, Toronto is where the most visitors are coming from; however, because Toronto represents 20.43% of total households in Ontario and we only received 13.65% of unique visitors from Toronto (Index 67), there is potential to increase visitation from Toronto due to its large household population.
You can do the same for the other three cities listed here. What is interesting is that there were more unique visitors from Orangeville than there are households, meaning that more than one person from many of the Orangeville households visited the location being researched.
The billion-dollar question
Now that we thoroughly reviewed each of the terms, you should be asking yourself, “To grow my business, should I focus on % or Index?”
A high % in a Ranking Area report will tell you where the most of your visitors are coming from and indicates that you should be marketing to that area as they are your biggest supporters/cheerleaders/ambassadors.
A high Index in a Ranking Area report will show you markets with an over-representation of your visitors, which means it should be easy to attract more of them from that area.
Ideally, you want both a high % and high Index.
Here is a quadrant to help you decide when to select % or Index:
| Low Index | High Index | |
| Low % | No It will be extremely difficult to attract this Visitor | No/Maybe Easy to attract with little effort, but volume of visitation may be limited. This might be a secondary market but not main audience |
| High % | Yes/Maybe Harder to attract and requires effort (maybe lots), but visitation will be high (E.g., Toronto example above) | Yes Easy to attract with little effort with high visitation. This is the ideal visitor that connects with your target market |
Custom Benchmarks – Going Granular (if needed)
As mentioned above, going granular has a limitation in the data; the system used for calculating these reports cannot adjust the ‘Base Count’ and ‘Base %’ values to the custom benchmark, only the ‘Count’ and ‘%’ values adjust to the custom benchmark. Custom benchmarks should be layered on top of the Ontario benchmark report as supplementary. Custom benchmarks are used when the stakeholder wants to focus on a specific area and wants to know the count and % of visitors within that area (e.g., if 96% of visits are within 100km).
| Ontario as the benchmark captures the complete values for each row (city in our example): ‘Count’, ‘%’, ‘Base Count’, ‘Base %’, ‘% Pen’, and ‘Index’. Custom benchmarks capture only your ‘Count’ and ‘%’ values within the selected area. |
Like in all research, it really depends on what research question(s) you are trying to answer.
- There is a lot of great data and insights stakeholders can gain from using Ontario as the benchmark to make actionable decisions on where to market: ranking by city, counts, %, complete base counts and %, % penetration and Index.
- However, if you are interested in knowing additional information about a specific area (e.g., 0-100km, 0-40km, 40-100km, or other), then a custom benchmark would be applicable and would provide you with only the number and % of how many visitors are travelling from where within that custom benchmark.
We will now look at understanding how to read and interpret custom benchmarks below, using the same geofence in Orangeville as above. Please note that the range of benchmarks go from large (Provincial) to small (0-100km, 0-40km, 40-100km, …) and the ‘Count’ and ‘%’ values reflect only those within the selected benchmark.
Benchmarking just 0-100kms
What we know
All of Toronto is within 100kms of the location being researched because the Count value remains the same in the previous benchmark. However, now Toronto represents 30.36% of total households that fall within a 100km circle around the location being researched. Because of that, it is indexing even lower: % ÷ Base % (14.95% ÷ 30.36% = 49).
Benchmarking at 40kms
What we know
Most (or all) of Toronto are further than 40km away because it is no longer listed in the top four. Let’s turn our attention to Brampton.
A number of households in Brampton are further than 40kms away. We know this because the Count from Brampton at 40kms is lower than the Count from Brampton at 100kms. Here is where the data limitation takes effect. Even though some of the households in Brampton fall outside of 40kms, the Base Count for Brampton is still all of the households in Brampton. This falsely increases Brampton’s Base %, which then lowers the Index.
40-100 KM from point of interest as the benchmark
What we know
We are now focusing on tourists with a benchmark of 40-100km from the center of our location, and the corresponding ‘%’ represents a percentage of how many households from tourists living 40-100km away. E.g., Toronto here represents 29.61% of visitation from tourists in our 40-100km ‘doughnut’.
It also demonstrates that if you were to focus on growing visitation from 40-100km, Toronto is now a prime market because it has an almost neutral Index but also represents the largest number of households and has the largest % of current visitors.
Did you notice that the count for Brampton is now lower than the first three benchmarks (34,397 vs 3,067)? The difference in the Count and % values represent the number of visitors from Brampton at each benchmark level. This means there were 34,397 visitors from all of Brampton, but 31,330 of them were locals (Benchmark 0-40km) and 3,067 were tourists (Benchmark 40-100km).
This once again demonstrates the limitation in the data because the total number of Brampton Households that are greater than 40kms away is much smaller than the total number of Households in Brampton. However, now the 3,067 visits is being measured against the total number of Households in Brampton rather than the total number of Brampton Households farther than 40kms away, which makes the index very low.
Hopefully these terms now make a little more sense, and you can start making the connections yourself.
However, if you do have any questions about these reports, other research, or data in general, feel free to contact Tom.
Written by:
Tom Guerquin
