Alex Loveless and Pete Hodge discuss the value of test & learn strategies for creating value and mitigating risk when deploying commercial and operations strategy.

Transcript

[00:00:00] Alex: we’ve done our exploration, some evaluation. We’ve got some insight and really it’s about either checking that insight or we’re actually into the action stage. You know, we’re evolving and we want to, we want to look and change something. And it’s following. the similar cycle in terms of Agile that we do during exploration.

[00:00:24] Pete: So it’s an iterative process. And what we want to do is pick off small parts to do, and we are learning in that. And the reason why you do small bits to test, take action, and then learn from the results is you want to minimize your risk. There’s always going to be a risk in any change. And you want to minimise that risk.

At the same time you want to maximise your learning. So you want to not only make the change, but there’s no guarantee that any change you make is going to work. So you want to minimise impacts of the change, do enough so that you can get a measure from it, and get your learnings, and then rinse and repeat.

[00:01:04] Alex: So have you got any examples in mind?

[00:01:06] Pete: This is probably something you’ve done before, you know, and you’ve got a number of choices of how to do it. And what we would advocate here is choosing a segment of your customer base and, you know, a variety of prices that you might want to charge. And you’d break those out and test each one individually against, a small segment as opposed to just putting your prices up for everybody and see what sticks.

There’s obviously always risks with putting prices up. And if you can segment your customers in a way that, you can separate them out, then you can start to see which ones are going to work.

[00:01:44] Alex: So the risk is that you put the prices up for everyone, which is going to be quicker.

And obviously in that case, you’re getting more money coming in, right?

[00:01:56] Pete: you’ve got the risk element, so straight away take a bit more time and you’re reducing your risk. The flip side is the learn side.

If you put your prices up and everything seems tickety boo, that’s great. What you’ve learned is that your market can sustain a higher price. Okay, could it have sustained an even higher price? We don’t know, you know, if you’re, if you were ticketing something, for example, and you increase your your prices by two pound a ticket, and there was absolutely no drop off in the number of tickets you were selling, you would go great.

But could you have sustained three pounds? Could it have been four pounds? Will the market sustain six pounds, ten pounds? You don’t know. The only way you’re going to know. Is you’ve got to do another 2 and what you keep doing that until you get to the point where people drop off

so, yes it’s quicker but you don’t learn as much. With a smaller test and learn, you would look to maybe even make the increment smaller and certainly a smaller number of customers. . Now you’re not affecting your whole customer base. Equally, you might then say, well, it worked for this segment of customers. Now try it for these ones. And because you’ve segmented out and you’ve got smaller chunks, you’re seeing and you’re learning whether each of them will sustain it, or what you might find is a small group, did move away.

So you get to learn a lot more, which of course is great for when you want to put prices up again in a year, two years time, you’ve got a much better starting point for which part of your customer base will, will sustain that, that price increase.

[00:03:31] Alex: Yeah. So I think by putting your price up across the board you might have by doing that have changed the, the, makeup of your, your market, your customer base. And therefore, because you put the price up, you’ve, you’ve sort of nudged into a different, demographic perhaps.

Is that what you wanted? And by pricing the lower bracket out Did you want to do that? And, and could you not have both audiences? And is there a medium price point that you could, you could hit that would , please, , everybody and increase your overall base while improving your income?

Pricing for markets is way more complicated than this, but I think that the, the core point remains that if you stratify your test and you target it properly, you get to use it as a method of data generation that will allow you then to apply much more nuanced pricing strategies going forward.

[00:04:31] Pete: So I think the stratification is the real key point. You can start to make, you know, Very targeted tests, and you’re going to learn very targeted things. And as you say, you start to focus on that and do some analysis around that. And that’s going to get you thinking about some of the other risks, which might not be so apparent.

[00:04:50] Alex: The vast majority of the things you’re saying actually relate less to the, in this case, the price increases themselves and a certain business strategy and more about data and information generation and and learning from that.

So it it’s not just about testing and learning, what you’re actually doing is generating more data about your business and your customers that you would not otherwise have had. And, that should be considered gold dust. It should be handled properly, analyzed properly, and understood properly, both in the context of the test that you’re running, i. e. higher prices, but what else have you learned from that? And what can you infer From what you’ve learned from that that you may be tested on a certain segment You know people who like football, right? And they were much less tolerant of the price increase than people who like snooker right Do you act on that?

Right, do you say, but I’m gonna charge snooker fans more, right? Maybe, I don’t know, but maybe not. But the point is you’ve lost nothing. What you’ve gained is information and if you’re a betting company, perhaps, that’s really valuable.

So the implication is that businesses are testing and learning all the time, right? Or rather they are taking actions and through either active analysis or through general Consumption of of information they are learning things,

And really what we’re not talking about here is changing fundamentally, , the way that your business operates. It’s by taking processes that are already . happening to some degree and putting them in a an environment and an analytical framework that will allow them to be meaningfully and accurately measured. So we create an environment whereby we can solicit a behaviour. But we may never repeat that action again, even if it was successful, right? We might just do things to , the customer, that are, are simply to generate information.

There’s all sorts of things you can do just to solicit behavior from the customer that will allow us to learn something that will ratchet us up to a much bigger goal.

[00:07:09] Alex: So sort of a a hierarchy of hypotheses as it were.

Thank You!