Episode 17
Signal Vs. Noise with Matt Shenton
So you feel like you have a different take on this perspective. I wanna understand that. So please expand on this.
Matt Shenton (00:47)
As a search marketer, search 10 years ago, you could very easily draw a line around it. It was Google search. Over the years, other things have crept into the ecosystem, retail search, social search, UGC, and you won't find it too hard to predict the demise of legacy search. Everyone thought that everyone's going to search on TikTok and no longer Google, et cetera, et cetera. And the same thing happened with AI Search. And what has happened with AI Search, is they've just introduced new, interesting ways for people to search and discover on the internet. It has led to an increasing ecosystem of how people search. One hasn't replaced the other. It's a really complex, fragmented ecosystem. And then providing a view of it so that as a brand you could look at it and go, okay, this is where we're showing up, this is where we're not, this is the kind of content or the areas where authorities and we're trusted, here's where the gaps are.
So I founded Brand Schema and the first thing we try and do for brands is understand the ecosystem. And then we help out with execution. But right now, the focus is to simplify the complex, give people a visual understanding of the ecosystem and then a way forward.
Doug Logan (01:52)
Gosh, I miss what, ten, fifteen years ago? The golden era where it was like everything could be tracked and everything was so much easier. And now it's either we have too much information or not enough information because of privacy. So how do you make judgment calls? How do you know where to spend your time, where to put your money, put your ad dollars at work? And same thing for search. It is arguably where everyone wants to be, because it's the bottom of the funnel. It's so close to the sale.
You even touched on sentiment too. Are you there because people are searching and liking your brand? Or are you there and people are saying, like, don't ever buy from this company? 'Cause you could be ranking really well for all the negative things too.
Matt Shenton (02:33)
Well, this ecosystem is so complex now, it would be foolish to think we could track every single thing within it because we can't. So what is the middle ground right now? And I think part of it is a bit of like ideology philosophy on like what is going on to help fill those gaps. Because as you know, like a lot of marketing campaign strategies get voted down not because of data, but because of someone's belief about what it is and how it works and the ideology behind it. Like, you've got to be a good storyteller in marketing to get things through. It's not just about hard data. And so I think this idea that the storytelling and the ideology and being able to simplify the complex is a skill, especially in this AI world, I think is a skill that's really needed for marketers slightly more than before, because you are having to deal with situations where you can't point to a load of data to say this is factual. You have to have a way of describing that people buy into and believe. You need measurements still, obviously.
And I'm on a journey, same as everybody, to try and figure that out. I think my hope is that we do move away from this very granular touch point approach into something that is a level or two higher so that we're able to try and test and move into things that do prove their value of a slightly longer timeframe. As this world evolves, more and more we have to do harder things and those harder things are usually things that take longer for the payback.
Doug Logan (03:52)
Yeah, you're right. We are going back to the golden era. I remember when Google Analytics rolled out and it was you know amazing because we finally had something that was really powerful and that could work the majority of the time and nobody was blocking it and there was no privacy concerns or anything. We were living in an era where even if we say, Look, nobody's saving that Google search ad, maybe they clipped out the full page print ad you did and pinned it up on their wall or still have the magazine sitting on their coffee table or something. But we could at least say, definitively this publication versus that publication versus that print campaign or this direct mail campaign versus this search ad you can look at that mix and really start to dial in to say what's working and and what's not, at least not having to rely completely on your gut.
And now I think there was a point just a couple of years ago where it was sixty, seventy percent analytics 30% gut and I feel like it's quite possibly flip-flopped because the analytics is just getting so muddy and there's definitely a huge amount of data. And I did this study a couple of years ago to try to understand, specifically how much analytics and how much traffic we weren't able to track because it was just completely unknown to us. And That was for some of our clients it was fifty percent. It's probably gotten only worse. And and it's hard because it's if someone truly, has everything locked down from a privacy standpoint, we're never gonna track them. We're never going to see that. It will never add up.
Matt Shenton (05:24)
What do do? How do you approach that?
Doug Logan (05:27)
I think what it really comes down to is that we use the data as a guide, but we still have to use our gut just as much. And so what used to say almost factually, I could tell a client, we've got A and we've got B and we're gonna test them both and and see what happens and we measure it and we see what the outcome is of that particular campaign and that test. I just don't think you can do that anymore. If you do that within one particular channel and you test, a Facebook ad versus another Facebook ad, you can you can say that that probably worked. But what happens if that ad leads them off network and onto your website? Then all of a sudden we get a little fuzzy. I mean, yes, we can track it to a certain degree, but what if that website's then tied into your CRM? Like that's a huge funnel. Every time you add that a next step in there, you're adding a whole lot of complexity.
It's not Yeah, it's not so black and white anymore.
Matt Shenton (06:22)
Yeah. Where my mind goes is all that is, and this is a skill that's becoming more important, is being able to understand what's a signal and what's a noise. Like there's so much data there and to your point, some of that data is incomplete or you can't trust it. So being able to look at that and say, this is the reason why I think this is a signal and it validates or invalidates a hypothesis or a test and this is just noise. And maybe that means that the test you have to do a bigger swings because that's the only way you get enough signal and move away from the noise. It's having that ability to recognise signal and noise, I think is really crucial.
And I love talking about this stuff because that for me is like just another example of a skill you need in the AI world. And I think more needs to be said about those sort of things because we get so stuck in this narrative around replacement. It's like… Well, no, actually, there's all these things that we're going to have to be good at and learn within this new world. In most cases, I think, so far, they're pretty interesting. Like, it's pretty intellectually challenging to be like, how do I recognize signal and noise? Like, there's lot goes behind that. And I think that's fun. And I guess it plays into, the optimist mentality as well.
Doug Logan (07:29)
Yeah. I I love that. I don't know that I've labeled it signal versus noise, but it is. And I would I would even go further into defining what is signal and I I look at it from the perspective of is we've got no lack of metrics. But if the metrics that we're trying to use doesn't actually affect any change or cause our clients to do something different, then it's the the wrong metric. So, we can use data to to back up our story whichever way we want. We can we can say, well, this is this and Roas says this and whatever. And it's like great. Does it actually matter to the client? At the end of the day, are they changing something because of that metric? Does their business change because of that metric? Because those are the metrics that matter. We can look at all of the ones that ladder up to that metric and make those decisions.
But when we sit in front of a client, I don't want to have to educate them. I don't want to give them a marking degree so they understand what we're saying. I I want them to look at a number and for us to say that number means success. And I'm doing this because of that number. We're doing a good job. We're not doing a good job. Something's changed. Something needs to change. those are the different things that I look at in some cases we're just coming up with new metrics that are unique to the client because that's what matters, right? And and there's not necessarily a a dictionary term for the metric that maybe we're creating, but that metric is what really matters to them and it's based upon all the data that we can possibly get.
Matt Shenton (08:55)
Yeah, it's a fun game.
Doug Logan (08:58)
Ha ha ha.
I think it's very easy to overwhelm people with with too much information and get extremely intellectual and go over somebody's head. I find that as a bad example of leadership. And I think also just a bad example of being an entrepreneur too. I don't try to be the smartest person in the room. But there are a lot of times where you can come into a room and you can sound like you're the smartest person, but are you really creating any value, or are you just slowing down the the perception of everyone? You they might leave feeling like, Man, he had a lot of really good buzzwords. I don't know what half of them were. I jotted down a bunch of them, I need to look them up later on. But that we just sacrificed clarity, we sacrificed any kind of progress in that conversation because we made it too hard for everyone to be speaking that same language. And as marketers and especially data scientists as we are, if you would say. it's we've got no lack of acronyms for sure. So how do you how do you approach I mean how how are you thinking about that? Because obviously you're you're pulling together some very complex things. Are you gonna be adding to the dictionary or simplifying?
Matt Shenton (10:04)
Definitely don't want to be adding to it. I very much like to simplify. But it is, it's a really important point because I think there's, I think you ideally want to try and operate across the full spectrum because sometimes people want to start off with what is your futuristic view or kind of what's a high level view or what's the common theme across all of these different things. And that helps ground people. But you've got to be able to come all the way down to, well, these are the three things you do tactically to improve ROAS, increase volume. And it's hard to do that succinctly all the way through. you kind of have to in order to really have what I think is a winning proposition. I have a view of the commonalities across them. And I think I...
I think the thing I am probably still building out is making sure that there's not too much education required because I think there are certain elements of the world today or way we market that do need to change because we are applying like old thinking to an evolving world. So there is a bit of education there, but you don't want get to the point where you're having to reinvent everything. And so it's just find that balance. And I think the other framing I'd put is like, all I care about is driving better performance. Like to your point, all that matters is that last week's last month's performance is better than the previous month. So if I'm wrong in thinking this stuff, then I won't be around for very long.
Doug Logan (11:24)
I don't think you're wrong, but I think you've you've definitely picked a very big challenge. I think the balance between what we agencies do for our clients and what clients trust us to do is kind of that in between. With some clients yeah, let's put all of it into one big bucket. Let's look at how many leads we generated across all the different channels, paid and organic for a given month and see how that works. And that becomes our number. And then we can look at channel by channel specifics, which one was performing better. For some clients when social does play a role, the challenge, is that typically social isn't a big push or they don't have a big enough follower or any kind of content like that where it's like worth really looking at that and measuring I agree with you conceptually. I have no idea how you're going to tie it all together in a way that tells a client, when we spend money here, it also has impact over here and be able to do that confidently to be able to show them that.
Matt Shenton (12:25)
Yeah, I mean, the first point to make is that from measurement perspective, you can't go into this thinking you can measure it the same way as the old stuff. And I think that's most clearly evidenced by how people are talking about AI search and chat GPT. AI has highlighted how interconnected these platforms are. So the fact that AI cites and uses sources that are social, that is UGC, shows the importance of it. Because the reason it cites those is because they are trusted authority platforms, authority influencers, creators, content. Because we don't have the measurement, we first have to show how this is all connected and how it's hard to measure. I can't go into a room and say, okay, here's the entire ecosystem. Here's where I'm to measure all of it. Here's how you go and drive more results. Cause if I could do that, then everyone would be doing it.
As a paid search guy, I technically could just push and pull levers and not really know anything about marketing and drive sales. Maybe that's being a bit unfair. You've still got to write ad copy and all that kind of stuff. But targeting was handled by the system. Biddings handled by the system Deliveries you didn't really have to be a fundamentals expert. Because marketing, the push and pull is always like brand and performance, art and science. I think what technology has done recently is it has leveled up what we can do from a science perspective and maybe people think that's taken away from the brand, but actually what it's done is it's also levelled up the brand perspective, like you need to be even better at that. So it's kind of levelled up everything. And so within that, the fundamentals are really important. The fundamentals are now more important than they were, 10 years ago. And when I say those opportunities to push and pull levers don't exist anymore, they do, but not as much as they used to.
Doug Logan (14:05)
Immediately what I start with every client is like figuring out what's what is truth to you, right? What does success look like in three months from now, six months, twelve months from now, I wanna be sitting across from you at a table looking at a piece of paper.
We're both nodding our heads in agreement that this was a success, or shaking our heads in agreement, that it was a failure. But we, you know, we have to understand that and unpack that because if if we're if we're measuring the wrong things, we can it might make us feel great, but if we're being judged by something that we're not measuring, then Shame on us because we're gonna find out eventually. So might as well find it out right at the beginning.
