3.4 Measuring Non-Digital Marketing Channel Metrics

Some people seemingly default to easy-to-measure marketing channels such as pay-per-click advertisements rather than choosing other channels such as print and television advertisements, whose effectiveness can be more difficult to track. However, you should not let convenience drive you to waste money on less effective channels when, with a little effort, you can also measure, test, and refine more impactful forms of marketing. It’s worth spending some time to discuss a few tactics that will allow you to measure non-digital marketing campaigns.

The key thing to remember when testing different channels is to keep your marketing message the same, so the only thing that you’re varying is that channel itself. Otherwise, you won’t know if the difference in outcomes is due to one channel being more effective than the other, the message resonating more with the audience, or a combination of the two.

Ways to Measure the Effectiveness of Non-Digital Marketing Channels

3.4.1 Simple Survey

There are many opportunities to survey customers, including immediately when they visit your website, open your app, contact your sales team, or complete a purchase. The timing of the survey greatly depends on what you intend to measure. For example, one great way to ascertain what marketing channel most effectively drove new customers is simply to ask them in a survey at the time of purchase.

As mentioned in the first chapter, marketing not only determines the number of prospective customers a product attracts, but also the likelihood of purchase (acquisition) and retention. For example, we could launch an online marketing campaign that falsely exaggerates the capabilities of a product. This campaign could drive lots of prospective customers to our website. Depending on the effectiveness of the website, we will then convert a fraction of these people. Unfortunately, due to the nature of misinformation that brought these customers in, we immediately see large drop-off rates. This example demonstrates three critical stages: initial interest, purchase conversion, and post-purchase retention. We could implement surveys at all three stages to measure how well the marketing campaign drives prospective customers to your product or website, how effective it is at targeting the right customers and conveying accurate information leading to conversion as well as retention.

If the key question that you are trying to answer relates specifically to the effectiveness of driving potential customers to your product then you will want to survey them at the very beginning of the sales cycle when they come to your website, when they contact your sales team, and so on. On the other hand, it is sometimes much more interesting to consider both how well different marketing methods convert interest to actual usage or sales. In this case, you need to survey people after they have used your product, registered for your service, or completed a sale. Finally, it is also useful to have a sense of how well different campaigns attract the right customers and retaining them. In that case, you might want to survey customers some time after they have bought or started using your product. It’s important to acknowledge that customers may not recall the marketing campaign that drove them to try the product, and the data that you get from surveying individuals a long time after they have been exposed to the campaign is likely to be less precise.

Imagine that you are interested in learning how many people that bought your product were driven by your radio advertisement versus an online advertisement. Let’s assume that you are spending $10,000 on online advertising and an equal amount on the radio campaign. After surveying customers post-purchase you find that roughly twice as many people bought your product after hearing the radio advertisement versus those that saw the online advertisement. You compare the two channels based on your survey data and estimate the cost per acquisition as in the table below.

Example Comparison of Radio vs Online Advertising

It seems that the more traditional medium, radio, is a much more effective channel for attracting and converting customers, but how confident should you be in this analysis. You were very careful to keep the messaging the same for both channels, so that should not have affected the difference. But you are concerned that maybe there is a sampling bias where those customers that bought your product after hearing a radio advertisement are more likely to take the survey. This is a legitimate worry, and might have been hugely important if the numbers were closer. However, it is not necessary to be perfectly scientific about this kind of analysis when running growth experiments. It is certainly better to do a more thorough statistical analysis and take greater pains to design a more accurate survey, but such a level of rigor might not be worthwhile in many cases.

3.4.2 Special Landing Pages, Emails, or Phone Numbers

Another very clever way to measure the effectiveness of marketing channels is to tie them to unique website pages, telephone numbers, or even email addresses. For example, you might run a radio advertisement for your product, Sergio’s Widget, and direct folks to navigate to “www.sergioswidget.com/radio.” You might also run a television advertisement which directs people to “www.sergioswidget.com/tv.” The great thing is that you can be quite certain that folks that navigated to the landing page associated with www.sergioswidet.com/radio heard your radio advertisement and not your television advertisement.

This method has some advantages over surveys. First, you do not have to interrupt customers while they are in the process of evaluating or buying your product. Instead, you send them down discrete paths from the very beginning. Second, people might misreport in a survey what piece of marketing led them to discover your product, whereas providing them with a unique link or phone number for each marketing channel forces them to remember more accurately. Third, and most importantly, providing them with a special destination opens up many opportunities to track them throughout the entire acquisition process. In other words, you can track how many people that started at www.sergioswidget.com/radio went on to successfully complete a purchase. This is hugely valuable information.

Tracking marketing channels with unique web addresses and phone numbers also has some disadvantages of which you should be aware. Just as was true with surveys, this measurement technique will give you relative measures of performance. In other words, you might learn which marketing channel is likely to perform best. It’s also very likely that users will navigate to www.sergioswidget.com even though the radio advertisement mentioned www.sergioswidget.com/radio. This might not be a big issue and will not severely skew your relative measures of channel performance if we can assume that this problem occurs uniformly across the various channels. However, if one destination is much easier to remember than the other, you’ll likely have an inaccurate outcome. For example, if the telephone number that you provide in the radio advertisement is 1-800-888-888 while the television advertisement mentions 1-800-289-0973, it will seem that the radio channel is a lot more effective, relatively speaking, than it really is because the phone number is a lot easier to remember. What this all means is that you need to carefully select the unique identifiers that you use so that they don’t end up skewing your results.

3.4.3 Promotional Codes

Another great method for keeping track of acquisition across various marketing channels is providing promotional codes that are specific to each channel. This method is very similar to providing distinct web addresses, phone numbers, or emails, but it is different in the sense that people do not have to go to unique destinations. This can be a big advantage for prospective customers since they don’t have to remember a special web address or phone number. The main disadvantage is that the promotional code must be tied to some kind of discount or reward, otherwise people do not have any incentive to enter it, and this can be costly.


This post is part of the Growthzilla Book series, which is an online draft of the print edition that will be available in 2018. Be sure to check back next Thursday to learn about evaluation methods that you can use for growth. New sections of Growthzilla are published every week.

3.3 Retention Metrics for Growth

Example Average Lifetime Calculation

Imagine that you are really great at attracting new customers. However, the vast majority of the folks that you convince to try your product use it only a small handful of times and drop off. You don’t want to be losing customers as soon as you get them since your acquisition efforts will be squandered. This means that you have to focus on improving how well your product and business retains customers. But how do you know what tactics improve retention? You will need some way of measuring customer retention, so you can discern what changes are helping your business to keep existing customers. Below are three common measures that are useful to track.

3.3.1 Customer Retention Rate

The most common retention metric that businesses track is the customer retention rate (CRR). The customer retention rate is the percentage of customers that you kept during a period of time as compared to all the ones that you had at the beginning of the same period, excluding new customers that you gained. Mathematically, this percentage can be represented as follows:

The closer the number is to 1, the better your product and business are at retaining customers.

3.3.2 Average Customer Lifetime

Directly related to customer retention rate is the average customer lifetime. This is a fairly simple measure that captures how long your customers stay active, on average. To calculate this figure, one would take all of the customers that became inactive and average the period between when they first became customers and when they ceased to be customers. A simple example is shown in the table below.

Example Average Lifetime Calculation

There are a couple of things to note from the above example. First, it’s clear that this is a contrived example because we’d probably have many more customers that joined and fell off in the implied time frame. In your analysis, you would want to include all those customers that you acquired up to the present date. Second, those customers that are presently active should not be included in the calculation since they might skew the average if your acquisition rate is accelerating. Third, it can be difficult to identify which customers are active and which are not. For example, if your product is a real estate app just because a user has not logged in for a while might not mean that they have abandoned it.On the other hand, if your product has a monthly subscription fee, your customers will cancel it when they no longer want to use your product.. It is important to carefully pick the criteria to use for designating a customer as inactive as well as to consistently use those criteria. The final point is that it’s possible to compare the average lifetimes of cohorts of customers. In fact, that is the main way that your team could determine if the changes that they make increase retention by a longer average lifetime.

3.3.3 Dollar Retention Rate

Another very useful measure of retention (and perhaps a more accurate one) is the dollar retention rate, or DRR for short. Here is why many consider it to be a more accurate measure of the health of your business: imagine that you have a hundred customers and each of them pay you a hundred dollars a month for your amazing project management software. Your revenue is $10,000 per month. Now let’s say that you raised your monthly price to two hundred dollars per month. However, many of your current customers got upset and quit using your product. Specifically, thirty customers stopped using your project management app. Also, during that same month you got ten new customers paying the higher fee. Was raising the monthly fee a sound business decision?

The customer retention rate would have clearly fallen, but perhaps your company is making more money. The dollar retention rate is precisely the measure that could help answer the above question. Mathematically, DRR can be represented as:

Dollar Retention Rate DRR Calculation

Your customer retention rate given that thirty customers left and ten were added comes out to 0.7.

Customer Retention Rate CRR Example

At the beginning of the month you were making $10,000 and at the end of the month you are making $16,000 (70 X $200 + 10 X $200) with $2,000 in new revenue, which gives you a DRR of 1.4.

DRR Example

At the end of the day, your business is about revenue not about the total number of customers, so DRR might be the more authoritative gauge of retention. However, that does not mean that you should neglect your CRR. If you keep alienating customers by raising prices, you are probably not creating a viable long-term business model.


This post is part of the Growthzilla Book series, which is an online draft of the print edition that will be available in 2018. Be sure to check back next Thursday to learn how to measure performance of non-digitial marketing channels. New sections of Growthzilla are published every week.