Growth Myths: Data-Driven Truths & Fallacies in Modern Marketing With Christian Limon & Eric Seufert

Christian Limon is the former Chief Growth Officer at Wish, which was the top spending advertiser on Google and Facebook. He was also the Chief Growth Officer at Tubi and Gemini. Throughout his career, Christian has achieved five exits and $28 billion in IPO and M&A proceeds. He has launched 20 apps and led 33 more apps on growth and monetization.

Eric Seufert is the General Partner at Heracles Capital, a pre-seed venture capital fund focused on the mobile technology ecosystem. After beginning his career at Skype, he held a marketing leadership role at Rovio, where he launched Angry Birds 2. Eric also founded Agamemnon, a mobile marketing analytics startup acquired in 2017. He is the author of Freemium Economics and manages Mobile Dev Memo, a blog on mobile advertising and monetization.

Here’s a glimpse of what you’ll learn:

[3:03] How marketers can discern value from vanity metrics despite fraud concerns
[9:14] The importance of prioritizing profitability metrics
[15:07] Christian Limon and Eric Seufert examine the pros and cons of geo-holdouts in fast-scaling companies
[17:33] Why slow, controlled optimization leads to more durable ad campaigns
[21:08] Tips for improving creative ad testing
[31:50] Leveraging AI for hypothesis testing and idea generation in marketing
[39:40] Workflow and LTV use cases of AI
[48:24] How Apple and Google's control over attribution impacts marketing strategies
[53:06] Should marketers move beyond attribution?
[1:01:51] Eric and Christian weigh in on the future of growth and debunk myths

In this episode…

Today’s marketers face a challenging paradox: the more data they have, the harder it is to identify what's valuable. Between conflicting attribution reports, algorithm-driven campaign shifts, and pressure to scale fast, many teams optimize for metrics that don't move the needle. How can growth leaders cut through the noise to build scalable and realistic strategies?

Seasoned mobile growth strategist Christian Limon emphasizes the need for broad, strategic creative testing that breaks out of traditional methods like UGC. He recommends marketers tap into unconventional sources and avoid over-controlling creative input. Leading economic and digital marketing strategist Eric Seufert urges brands to prioritize commercial outcomes like profit and ROAS rather than exclusive platform metrics. Marketers can also use AI to enhance workflows and generate ideas for optimizing LTV.

Join William Harris in today’s episode of the Up Arrow Podcast as he chats with Christian Limon, growth strategist, and Eric Seufert, General Partner at Heracles Capital, about optimizing growth marketing. Together, they discuss how to identify meaningful marketing metrics, how to build systematic, creative-first campaigns, and the dangers of over-diversifying channels.

Resources mentioned in this episode

Quotable Moments

"You want to use AI in ways that magnify and amplify your efforts, not replace them."
"The more chaotic it is, usually the less sure you can be of any metrics you see."
"I try to think about how different and wide the search space can be for creative."
"The bar is so low to be systematic in growth, and most teams still miss it."
"You need to earn that first marketing hire with more work than the CEO could feasibly do."

Action Steps

Orient marketing teams around commercial outcomes like profit, not vanity metrics: Focusing on real business outcomes ties efforts directly to revenue, not misleading intermediate metrics. This alignment prevents wasted spend and drives sustainable growth.
Expand creative testing with unconventional sources: Sourcing creative content from outside traditional demographics generates fresh, unexpected ideas that can outperform predictable campaigns. It broadens the search space and removes internal biases.
Systematize campaign management processes: Building consistent frameworks for launching, editing, and evaluating campaigns eliminates guesswork and improves scalability. A systematic approach leads to greater stability and repeatable success.
Use AI to accelerate workflows, not replace decision-making: AI can automate repetitive tasks and surface insights faster, but humans should still drive strategic decisions. Maintaining human oversight aligns outputs with business goals and brand identity.
Limit channel diversification until core platforms are maximized: Over-diversifying too early dilutes focus and adds unnecessary operational complexity. Mastering one or two primary channels first creates efficiency and stronger attribution clarity.

Sponsor for this episode

This episode is brought to you by Elumynt. Elumynt is a performance-driven e-commerce marketing agency focused on finding the best opportunities for you to grow and scale your business.

Our paid search, social, and programmatic services have proven to increase traffic and ROAS, allowing you to make more money efficiently.

To learn more, visit www.elumynt.com.

Episode Transcript

Intro: 00:03

Welcome to the Up Arrow Podcast with William Harris, featuring top business leaders sharing strategies and resources to get to the next level. Now let's get started with the show.

William Harris: 00:15

Hey everyone, I'm William Harris, I'm the founder and CEO of Elumynt and the host of the Up Arrow Podcast, where I feature the best minds in e-commerce to help you scale from 10 million to 100 million and beyond as you up arrow your business and your personal life. Today's episode is a freaking masterclass in modern marketing, served up by two of the most brilliant and battle tested minds in the growth world. These aren't guys who talk just about theory. They've shaped the strategies behind some of the most explosive growth stories in tech, media and consumer brands. First up, Christian Limon.

Previous guest on the podcast and dare I say, friend? He's led mobile growth for some incredible companies, including Wish.com, Tubi and Gemini. He's also one of the most sought after growth advisors in the space, known for cutting through the noise and bringing clarity to chaos. And alongside him, Eric Seufert. Eric is the founder and general partner at Heracles Capital and was the mad scientist who helped grow Angry Birds.

He's one of the sharpest economic thinkers in digital advertising. If you've read his work, you know he's the guy who called the post Idfa world before most marketers even knew what a privacy framework was. His writing has reshaped how the industry thinks about attribution, platform power, and performance marketing. Today, we're diving deep into what I like to call growth myths. Those things we say share an optimized for and modern marketing that might not actually be true.

We'll cover why data being data driven might be the most dangerous myth in your marketing strategy. What AI is actually good at, and where marketers are getting it wildly wrong. Why your attribution model is probably lying to you, and whether the next evolution of growth will require more math, creativity, or philosophy. And yes, this conversation might get a little nerdy. We might reference Plato or even the Heisenberg Uncertainty Principle.

But don't worry, we'll talk about how to actually grow your business faster, smarter, and with fewer blind spots. So buckle up. This is going to be a fun, fast, and frankly, essential ride through the myths and truths that define modern growth. Christian and Eric, welcome to the show.

Christian Limon & Eric Seufert: 02:06

Thanks for having us. Hey, great to be here.

William Harris: 02:08

Cool. I'll give a quick shout out to our sponsor and then we'll dig right in here. This episode is brought to you by Elumynt. Elumynt is an award winning advertising agency optimizing e-commerce campaigns around profit. In fact, we've helped 13 of our customers get acquired, with the largest one selling for nearly 800,000,001 that ipo'd.

You can learn more on our website at Elumynt, which is spelled elumynt.com. That said, on to the good stuff. I want to start off with the illusion of being data driven. So my first question is for Eric here in Christian's podcast appearance here, he mentioned that being data driven is often a myth.

I loved it, it's a really good take. Similarly, Eric, in your article, yes, some metrics are fake, but performance marketers don't care. You discussed the prevalence of misleading metrics in performance marketing. How can marketers discern between meaningful data from vanity metrics in today's landscape?

Eric Seufert: 03:03

Yeah, so that's a great question. So so when I wrote that piece, that was a guest piece that I wrote for Adexchanger, and the reason I wrote it was that there had been this kind of viral thread on Twitter that was basically making the case that you essentially can't trust any metrics that you get from ad platforms, like essentially making the claim that like, everything is falsified or everything is misrepresented or mischaracterized or, or just, you know, just utterly fraudulent. And I was saying like, yeah, sure. Of course that's the case, you know, that that fraud exists, that, that, you know, not everything you buy has value. Right.

But but ultimately, like if you're a performance marketer, that's that's not that in and of itself shouldn't scare you away from doing marketing. I don't really know what the alternative is. You know, just throwing your hands up and saying, well, I guess we're never going to market anything again. The point I was making there was like, if you can discern value from waste, you can optimize your campaign spend. You may not be able to trust all the intermediate intermediary metrics all the time.

But like if you can build a framework to do measurement, like fundamentally do measurement, then you can sort of tune your ad spend. And if you're, you know, if you are doing it in such a way that you, you, you are sort of like demonstrably, demonstrably producing workable, you know, profitable ROAS return on ad spend then, okay, if there's some wastage, there's some slippage, whatever. Like that's unfortunate. But ultimately your your marketing activities are delivering more value than they cost. Right.

And so that was the point there. But I think like you know where people get hung up. I mean so there's there's a there are a couple of things that can happen that can sort of derail a marketing org, right? One is just bad incentives. Right.

And you've seen this historically with like famous cases. like like that of Uber. Right. So like you might not be familiar with this, I want to say it's like 2019, 2020. Uber sued a bunch of the agencies it was working with and they said, what was that Christian?

Christian Limon: 05:06

Is it fix you?

Eric Seufert: 05:08

It was it was a number of them. It was it was more than no, no. You know, they sued their agent. There was one agency that they sued. It might have been more than one.

But then they settled with them and then sued the demand channels that the agencies were buying from. Yeah. And, you know, the thing was though, like they had a chart that showed, you know, ad spend and then like new registrations, right? It's like two lines. And then they, they, they thought that like okay, well this ad spend is not effective.

Right. This ad spend is not incremental. And then they cut the ad spend. So you see the ad spend line just drop to basically zero. And the new registration line stayed the same.

And the new registration line was composed of paid and organic registrations. Right. And so like you saw okay. Well when, when they cut spend, the paid registrations line went to zero and the organics just went to the same level as the previous total. Right.

And so essentially they were buying users that would have downloaded the app anyway. And it turned out that, you know, there's an incentive problem. They were they had an intermediary that was doing the traffic acquisition. Right. These these agencies.

And they were buying from channels that I think and I made this point at the time they should have known were probably not delivering legitimate, high quality traffic. But everyone's incentives there were broken. Right. You you had the people in the marketing team who were just paying the agency that, you know, had this line that looked like it was delivering profitable metrics. They were not really incentivized to dig deeper.

You had the agency that's getting paid on a percentage of media basis, who's not really incentivized to dig deeper, because if they discover fraud, it's going to reduce the amount of money they're spending is going to reduce their cut. And so you just had like bad incentives throughout that system. So that's one way that, you know, your marketing can just be broken. You just have bad incentives. The other the other way is to just like, not really orient it around commercial outcomes, right?

To think that, like any given point along the funnel is as valuable as any other. Right? And so then you're doing, you know, cost per click or you know, you're doing even CPM buying or you're doing cost per registration or all these kind of things. And, you know, the reality is that those each of those has a different value and they have a different sort of like they possess a different sort of of amount of conversion potential to the thing that you actually do care about, which is a purchase. And then there's all different ways to like, manipulate LTV metrics to to project them out to far to calculate ROAS on too long of a timeline.

And so, like, you know, I think the way to fix that is to just orient everyone around true commercial outcomes, right? And to make sure that when you're modeling things that they're reflective of reality and sort of everyone is aligned around what that reality is.

William Harris: 07:44

I want to give Christian a chance on this, but I want to say this real quick. I said this on Kurt Elster's podcast, the Unofficial Shopify podcast. I want to do it with you guys too. Do you do you like cake? Do you enjoy cake?

Eric Seufert: 07:56

Yeah, sure.

William Harris: 07:57

Okay. Do you like real cake? As much or more than you like a picture of a cake?

Eric Seufert: 08:03

Probably more.

William Harris: 08:04

Right? Right. For the simple reason that it exists, right? Like to a point. Like there's a benefit to it existing in the world being better than just this picture of it.

That's kind of how I looked at ROAS. And one of the biggest reasons why we talk about, even at the beginning, talking about optimizing towards EBITDA, at being a big part of how we focus things. ROAS is completely made up. It's a it's a made up number that doesn't really exist within the ads platforms the way that I see it. Because that depends on what kind of data you're bringing in and out of this.

That could just be based on, like their way of how they're calculating attribution. There's a lot of ways in which it's not really it's not actually the real return that you have from that ad spend. And so that's where to your point, we align towards how do we make sure that what we're doing is driving towards EBITDA. If I don't have a client that's willing to at least share something that's analogous to their PNL with me. There's the likelihood of me being able to align to their actual goal goes down significantly, because I'm not going to hit that real goal.

Christian, you've done this significantly at Wish.com, I know especially, but as well as others to be, etc. but like I remember there was a campaign that we talked about that you did with, you know, the Lakers. And it's like, well, how do you measure the attribution of what this is? But like, what do you think about what Eric's saying here?

Christian Limon: 09:14

He, he said a lot. I think there's nothing he said that's wrong. I think a lot of it is like nuance lost in I think there's like a, almost like a segment of, of folks who would prefer the world to be like a lot simpler or like two dimensional as opposed to like eight dimensional. And, you know, a lot of the arguments, they, they, they. are underneath.

Underpinning all of them will be something related to like a binary argument or a is this valuable or not as opposed to really there's, there's I'll track like, you know, 30 metrics or 20 metrics and there's a hierarchy to them. And the highest one is like profit and then in revenue and paying users and you know, but you can doesn't mean that that's what you want to necessarily buy, because that might be the most expensive way to to acquire that. So there's, you know, there's like a lot of strategy and mechanics to figuring out how to make it profitable and how to how to like, scale it without impacting cost too much.

William Harris: 10:40

Yeah. I think one of the things that I have recognized is that a lot of times, it's easy to point the finger at the agency. At least I run an agency a little bit biased here. But more often than not, Eric, kind of what you called out is that even the internal marketing team usually doesn't even know whether or not what they were doing was driving towards profitability or not. They are also given, you know, oftentimes a a goal that maybe wasn't as deeply tied to the knowledge of how effective or impactful this was to the business.

Christian Limon: 11:13

So, to be fair.

William Harris: 11:14

To.

Christian Limon: 11:14

Be fair to Uber. So one, the person who was doing performance marketing like he had been in gaming before, he knew that that that the web was all fraud basically, and the way he was buying it or he should have known. But Uber is one of the only companies that's ever been in a race. You know, where like the time component was very valuable, where they were in a race to like capture a lot of the US and a raised in China, and they were basically like paying to speed things up. I don't know if that's how they actually justified it, but I would assume I've never been in a situation I feel like where, where?

Like they were requiring putting a premium on on time to that degree.

William Harris: 12:04

Eric, who's responsible in a situation like this, like is.

Christian Limon: 12:08

It.

William Harris: 12:09

Leadership?

Eric Seufert: 12:11

Well, yeah, I mean, everyone I would say. But yeah, I mean, that's that's a fair point. Like they were there was a there's they're putting a premium on speed for sure. And to be to be clear, like I don't have any details about the incentives that were, you know, made available within the team. All I know is what I read in the, the filing.

The court filing. I just I imagine that was the case. Right. And they did investigate it internally. Right.

Obviously. But yeah, I mean, like, who's I think when you're spending those sums of money, I mean, this, this, this is a, an imperative that starts at the very top. I mean, I think the CEO should, should, should get updates on like, you know, this is this this is like sort of like ROAS profile of our ad spend. And the thing like that I find really interesting is, you know, a lot of digital, a lot of digital first businesses. They don't really see marketing as like this critical structural component to their business.

Right? Like they know that they need it. They need it to for, to, to, you know, as an influx of revenue. But like, you might have a management team that's like totally divorced from the mechanics of doing, you know, performance acquisition like and at that scale, that is your business. That's essentially the entirety of your business, right?

Like if you cut it off, you know, the effective UA that you're doing, the effective performance marketing, the effective customer acquisition, if you cut that off, your revenue is going to go to zero. Your new customer acquisition will go to zero. Right. And it's also like generally the largest line item expense for the company. And so like just it's weird that there is a gap there, like a knowledge gap or a domain expertise gap.

Now I'm not I'm not saying that like the CEO of a very large digital first company should know how to, you know what the different types of bidding on Facebook. That's that's too low altitude. But they should understand like the mechanics in terms of like how this money that we spend flows through into, you know, a PNL and understand like where they can identify, you know, red flags there. And they should be sort of like regularly auditing that and working in partnership with like the CMO or the marketing leadership to do that on like a very sort of like, you know, kind of quick turnaround basis.

William Harris: 14:22

So this is where I feel like the Plato's allegory of the cave comes up, right? This idea that it's like we're looking at shadows, we're chasing shadows versus the real thing. How do we how do how can leadership CEOs have the knowledge you're looking for? an example would be Geo holdouts love geo holdouts. Do you think they're good or do you think it's too slow?

Especially if you're, you know, a company like Uber, you're trying to move fast. Is it effective to be able to get that incrementality or. No.

Christian Limon: 14:50

Well, the nice thing about having that many resources also of Uber is they don't have to do things sequentially. They can do everything in parallel.

William Harris: 14:57

Totally.

Christian Limon: 14:58

All at the same time. So they could do geo holdouts as well as everything else.

William Harris: 15:05

That's fair. Yeah.

Eric Seufert: 15:07

Geo holdouts are a tool, right? I mean, they you know, there's in like the sort of effectiveness is context dependent I think. Like if you were a company that was very quickly acquiring market share, geo holdouts might not be effective in certain cases. Right. Because first of all, you're probably predominantly investing in the US.

So what do you hold out on right now. You could do state level holdouts, but those are a lot harder to instrument right. And then you know, you are there is an opportunity cost there. Right. So like let's say okay.

Well like let's turn off UA and or customer acquisition in the US to see what happens. Well, if I'm growing extremely quickly and I have a competitor nipping at my heels, that's going to be a very difficult proposition to sell to your CEO. We're going to cut off ad spend in the most important country just to see just to just to just to measure the incremental effect. Right. And and then and if you are disproportionately investing in the United States, well, then if you go and cut it off somewhere else to see the incremental uplift there, it's not really that reflective of maybe what's happening in the biggest, most important geo.

Right. So like you've got to you have to do this kind of experimentation. It's really important. I think, you know, geo holdouts, they're a very effective tool in a lot of cases. But you also have to kind of consider the context and you have to come up with like bespoke solutions that match the situation.

Christian Limon: 16:25

Yeah. And they they don't tell you what or like why things change. They, they show you that there is some type of change Likely. And, you know, ideally you designed it in a way so that whatever you structured in there and baked in there was, was different. Tells you something, but it's what you see is like some type of outcome, not a, you know, a like second order like rationale as to like why everything was different or why x, y, z chose something else.

William Harris: 17:00

So I really like diagnostic action as one of my favorite ways to do something. You go to the doctor, and the doctor says 80% of the time that you have these symptoms, you have this thing, take this medicine if you have it. I was right, like, if you get better, then I was right. If you don't get better, then you know you were falling into this other 20%. But they don't say it that way.

But that's essentially what they're doing. Similarly, in an ads thing, it's like, well, just do the thing. If if things go up, likely you did the right thing. How do you feel about diagnostic action as relates to growth?

Christian Limon: 17:33

Would. The way I've done things has always been very iterative and slow and controlled, as opposed to anything bursting or any hard on and offs. It takes a little bit because you to scale, you need to. It's like I'm optimizing 100 times as opposed to a few times that like moves the needle up and down. It just it takes much longer.

But once you get it there, it's it's like quite stable and quite healthy because you, you know, the in between points have controlled for it. And that actually has like a lot of benefits to the to the actual campaign from like not introducing a ton of like volatility and shortening the life cycle of the campaign. That isn't like a fact, but I observed that as from like super durable two year campaigns being being live as opposed to other people's like one month campaigns or, you know, month and a half.

William Harris: 18:42

Yeah. That's fair.

Eric Seufert: 18:45

Yeah. I mean, I tend to have the same approach, right? I mean, you want to try to isolate as much change as you can and like, I kind of have always likened, you know, kind of like modern like call it like the last few years especially like post digital marketing to, like managing a Jenga tower, right? I mean, you want to pull one thing out and be just very cautious. And then, you know, you pull another thing out, you make a change, you watch, you wait.

You sort of like, hold your breath. Like, I think my approach, you know, and, you know, to Christian's point, like this is supposed to be, like, slow and methodical and systematic, right? It's not supposed to be, like, frenzied and chaotic. And the more chaotic it is usually like the less sure you can be of any metrics that you see, right? I mean, you can't just chase short term volatility all the time.

It's too expensive. And and, you know, all of these metrics tend to be like mean reverting. So you need a process and you need a dependable process. And that means just running it many times and being certain of the results. But I think like what what I, you know, think is really important, particularly now, is like managing the creative side of things in a way that is very systematic.

And that's that's typically what I see people doing wrong. Like if I, you know, get exposure to an organization that where their marketing, you know, is not delivering the results that they would they would want. It's that there's no real rhyme or reason to the creative. They're just pushing stuff out, and they're pushing a lot of stuff out, and they don't really have a system. They don't have a system for doing analysis, they don't have a system for doing production.

They don't really understand what volume of creative produces, like reliable results. And that probably is like the single biggest lever. And so it's just like disproportionately focusing on that and building an actual, repeatable system usually solves a lot of problems for like digital, you know, especially for like digital first companies. But but even just generally with digital marketing.

William Harris: 20:38

I like where you're going with this. I want to jump into this creativity side then. So there's one school of thought out there where somebody says, just throw it all, you know, it's the spaghetti against the wall, throw it all at the into the algorithm and just see what happens. And they're typically saying you need to be testing, you know, a thousand new ads a week or whatever. They're throwing out some number and they're just saying this.

You need to be testing.

Christian Limon: 20:59

This contained by only what you can think of, though.

William Harris: 21:03

Sure. Okay. So what's the alternative? What's what's beyond what you can think of.

Christian Limon: 21:08

Is you develop a strategy that takes advantage of more of the search space, is you find out how to remove yourself from being the limiter of things you've tried. I did that for UGC video I it took me forever to find something, find things that worked because I kept I had to I have to first develop a strategy for how was I going to try things. That was so that was like not biased by me and that were different enough from each other and from everything that I've done, including how these groups or people think and what they do for a living, that that I wasn't going to be, you know, the four of the six reasons why things, why we still why there wasn't a working like e-commerce video creative.

William Harris: 21:56

So how did you do it? And I'm going to give you a scenario. One of the things.

Christian Limon: 22:00

We did instructions.

William Harris: 22:02

Say that again. Oh it didn't give anybody instructions. Just let them do it.

Christian Limon: 22:05

I didn't let them do it, I gave them I told them what they what they wanted to do and the how I, I left it up to, to them, gave them a budget, but I had I did the simultaneous approach I and these these groups and people weren't necessarily all They didn't make creative for a living. Some of them did, you know, apps kind of like whatnot and stuff like that, where they might be able to make creative on the side to like, pay for some bills because they have they're making a lot of video in-house, or it's like a stand up comedy type of sketch something or a modeling something. But I just try to think about like how different and wide can the search space be? And like, how can I how can I make sure I'm covering like very different points that can have like viable signal for like, is there potentially is this somewhere where the working ad could live?

William Harris: 23:12

So I like that one of the ways that we've tried to get outside of the echo chamber, if you would, of even UGC can start sounding exactly the same too, is giving it to somebody who is not even in your ideal customer profile. And an example that I have is there was a beauty device that we were working on selling, and so we were kind of in a rut and we said, what about something completely outside of it? And I don't remember the exact niche that we went after. I think it was like biking camping type thing. It was like, how about how about, you know, a female who's 35, who's more into biking and camping?

And then it gave us some completely different things, but let's.

Christian Limon: 23:47

Say B2B things like that a lot, where they're constantly they're they're thinking about who specifically they want or who they want to exclude, as opposed to just enabling.

William Harris: 23:59

Sure. So let's say that you do this though, let's say that you get 10,000 ads from completely different people. How do you still turn that into a learning experience that you can lean into certain things that are working versus just wasting it? Because in theory, every ad that you test is a little bit of a waste small.

Christian Limon: 24:16

It's really not right. So to to to get signal on whether something works. That's what you're dependent on. Is a sample size like not a it isn't dependent on some other like variable. So like regardless, everything that you try, you're the first like learning period.

You know, like X conversion amounts, you're going to pay for all of them equally just to find out what the result is. But then from there, it's up to you whether you want to like, take it to like $1,000 or increase it by, you know, 15%. And it's it's everyone. It's like everyone else is jumps up. Even small advertisers, they jump up to like really big amounts really quickly.

William Harris: 25:08

Yeah. Eric, what do you think? What's your process here.

Eric Seufert: 25:12

Well, so so I mean the number of variants I mean I think that's probably a vanity metric in a way. Right? I mean, unless those variants truly capture fundamentally different ideas. Now, now you want variants. A lot of times because you just want to display you want to you want to convey some idea, you want to convey some sort of like, you know, commercial narrative with different properties because like, those properties might perform better, they might not perform.

But like if you're talking about truly optimizing performance, I mean, you kind of have to think about this like a gradient descent problem. Like, you know, the variants are going to like sort of, you know, find like the kind of like local best, you know, the the local sort of like maximum. But if you want to find the global maximum, you need just radically experiment with concepts. And so you might have you could have 200 variants or 10,000 variants. And if they're all, you know, just capturing the same three concepts, you're not really doing experimentation.

You're just sort of like modifying variants.

Christian Limon: 26:07

My critique of of the data driven thing was that it's still constrained by like, you know what, you're allowed to experiment in your organization or your team and your resources, or your CEO, or, you know, by your brand constraints.

Eric Seufert: 26:23

Yeah. Or just production constraints. Right. And I think that that's that's the kind of that's the kind of, you know, suboptimal workflow that teams get stuck in. It's like, well, okay, we'll just increase the number of variants.

So like that just means okay, instead of making, you know, 20 variants for every concept, we're gonna make 200 variance for every concept. You're still conveying the same concepts right. And you know, the way that you introduce like true you know true difference is, is by with randomness. Right. And so you could think about this like as the concept of like temperature and machine learning.

Like you got to turn the temperature up and you got to get like truly random ideas and or just different. And so like a lot of times there's a couple ways you can sort of engineer that to happen. One is just finding different sources of inputs. Right. So like you've got a marketing production team.

Well maybe let's get ten outsourcers and we'll give them sort of like very broad, vague instructions and just see what they come up with. Right. And then like we'll cycle through, let's say we have like this, this, this web of influencers, this network of not influencers of freelancers. And we'll just have a we'll cycle through like a different ten of them every week or something. That's a way to introduce essentially not not true randomness, but just just like alter the sort of the sort of input sources to get different ideas, like different fundamental ideas.

Christian Limon: 27:40

How do you reconcile? So I had a button at Wish, and I would duplicate a lot of campaigns. I would just duplicate them like 100 times, 50 times something like that. And since since the learning period is is actually pretty finite and the campaigns are path dependent, meaning it it judges who it finds next based on who it found before in a decently like, like finite pool that can be biased with enough of those duplications that we could create and or manufacture outliers. Do you think that maybe that was an accident?

Or how do you reconcile that, that piece?

Eric Seufert: 28:20

I think that that would have been I think that would have been like a really effective strategy. But pre advantage. Plus I think now they try to consolidate everything to the point where like they want to engineer that. And I think you wouldn't be able to do that with like just a large number of I see the logic. Right.

And that does make sense. But I think now it's the way they've, they way they've sort of like instituted the consolidation through advantage. Plus I think it'd be hard to do that with just creating a bunch of campaigns.

Christian Limon: 28:45

Yeah I never understood that. So there's do this so that it performs better. And then there's the does it perform better. And we had like 30,000 campaigns and we never consolidated it. And it was our suspicion that, like the only rationale that made sense that they could communicate was that they were just annoyed by it.

Versus a the only thing that matters is, is, is it Does this method? Can we find a signal that tells us that this method is like works better as far as like the returns.

Eric Seufert: 29:20

Was they being Facebook? They were annoyed that you had so many campaigns. That's a lot of campaigns.

Christian Limon: 29:25

Yeah, but also they couldn't see them because we would buy through the API.

Eric Seufert: 29:30

Okay.

Christian Limon: 29:31

So they were annoyed that basically that they're just like very in the dark.

Eric Seufert: 29:35

Yeah.

Christian Limon: 29:35

Or they would be annoyed by the just you being an anomaly to a best practice.

William Harris: 29:42

So you're the reason why they have ASD and they decided they wanted to control all of this.

Christian Limon: 29:47

Yeah. We never I never just didn't listen. Yeah. And then they would like we would get carved out of like any having to do anything like that with the you know UAC was like that for a long time. We just stayed in all this time or, or AdWords for a really long time with multiple campaigns instead of that universal app just because it's like if we disrupt everything, do you really want to lose, like, half a billion bucks?

William Harris: 30:17

Sure. You don't want to disrupt something that's working?

Christian Limon: 30:20

Yeah.

William Harris: 30:20

Since we can't likely do that as much anymore. Like you said, based on what? What are other ways that you can inject that randomness? I'll give you an idea of what we do and what I think. And what I think works very well.

But I'd love to know what you guys think about this. We. Okay, so for testing the different concepts, let me start by by pulling back to that, testing different concepts versus just 200 iterations of the same thing. I do like to look at behavioral science tactics. So I'll just say like which like is this the sagnac effect.

Like which which thing is it that we're actually testing within this versus just simply saying, oh, all 200 ads that we just tested are the exact same behavioral science tactic. Can we branch out from that as one of the different ways we're looking at this, even this, these were all UGC from 200 different people. They people. They could all be still trying to rely on the same thing, maybe because they watched other people write ads for this or whatever. So how do I break that out?

How do I see I want to test this other concept out, try and do that a little bit more. What I haven't done, but you inspired me just today. To try to think through is on the randomness side. Is using maybe ChatGPT to say, okay, maybe I'm going after 30 year old girls for this particular product, this beauty device. Maybe I say write an ad as if you're writing this to a 60 year old male.

Like maybe that would also be this idea of like completely different and off script, but just seeing like, like, how could I branch this out beyond what it already thinks it should be writing towards? Good idea, bad idea.

Eric Seufert: 31:50

Yeah. I mean, it's a good idea. There's actually a lot of research going into using Llms for hypothesis testing. Now, there's an interesting paper. I'm blanking on the guy's name, but it's his it's it's actually his job market paper.

He just finished his PhD at University of Chicago. And he's looking for like an academic position. And what he wrote about was they used Llms for hypothesis testing and they tested them on headlines. Right. So there's this this data set, this Upworthy.

Right. So I don't know if I don't think the website exists anymore, but it's Upworthy. They had it was kind of like a just like a pop news website. But they ab tested all of their headlines. And so there's this large data bank of all the headline variants that they tested.

And so what this guy did was he fed these into ChatGPT and he said, like, come up with hypotheses around why these might have worked. And then he consolidated those like deduped them and then used those to create new headlines and then tested those and just to see which ones worked. I think there's a lot of interesting research going into using Llms to kind of introduce that kind of randomness is like almost like a stochastic mechanism. I think that's that's really interesting. I mean, I think like, you know, but even if you didn't want to build that kind of framework, I think it's there's there's sort of there are ways to just say, look, we need to we need to come up with like totally new things.

And, you know, if we just all the same people sit around the same table and say, let's throw some ideas out there, you're probably not going to get that right. So some recognition that like the, the, the, you know, the process that got you to this point is, is is going to continue delivering like very similar results. Right. And you have to shake things up in some material way.

Christian Limon: 33:27

How I notice a lot of people get stuck up on just the concept of they need to test, but test means this specific thing, like the way you were describing, I need to test this or that, or or how the mechanics make it so that it's a test as opposed to, you know, like spending the. Like, why do you want to test. Right. Is it to like, inform something or is it to discover like healthy, marketable campaigns? If the ultimate goal is marketable campaigns, then you could just, like, launch everything.

Live small. And that's your. That's your test. You'll. As long as you don't spend, you know more than you would a test.

Like whatever you consider to be. Efficient. That's like a.

Eric Seufert: 34:22

I think there's also there's two surface areas, right. It's not just the ads, it's the product. And so, you know, I was we were talking about like well creative is a is a is a very powerful lever for driving performance. But if you can actually make the product work better, the same performance on the creative side and the click side on the top of the funnel that could, could actually, you know, would deliver better ROAS, right? So they have two surface areas that you can sort of you can optimize not simultaneously, but like in a kind of back and forth process to, to get to like the sort of optimal point.

And so some of those things can be just introducing radical variation to the product itself. Right. And if you can just get people to, to check out at a higher rate, I mean, this is also like really easy to do on the web. That's one advantage that you have when you're doing sort of like any sort of like web based marketing, it's like, well, you're pushing someone to a website. You can introduce variations to like a checkout write process or a registration process really easily in a website.

Christian Limon: 35:16

That sounds like a cmo's nightmare. The a like variance introduced into their site or into the brand. Like their I think they they did a variance would be like a negative within within them or they would they would want to say okay variance but within this. But then when you really look at what this is, it's like quite tight.

William Harris: 35:38

I mean this is very similar to what what's her name is doing Sheen. Right. I mean, it's just they're literally just testing product.

Christian Limon: 35:46

Yeah.

William Harris: 35:49

Yeah, right. They're just testing product just to get it out as many different types of products out there. I mean, they're not testing variants on that product, but in essence, they are.

Christian Limon: 35:57

Yeah. It's just a it's a variant of the same school as like a wish, which is just a and more is more of everything. You know, more is better.

William Harris: 36:08

When you're.

Christian Limon: 36:09

If you're gonna do one of those strategies, like the stuff that I talk about is like, you either commit to that type that extreme, or it doesn't work.

William Harris: 36:19

Sure, it's not going to work if you have one product kind of thing, as well as if you have many products.

Christian Limon: 36:24

That doesn't work for a number of reasons, like, yeah, like for mechanically, for one, creative works a lot like worse than say, you know, dynamic like a 40 SKU, 40,000 SKU creative. But also Tam was yeah.

William Harris: 36:44

When when you talked about making the product better. Eric where my mind went was not like the idea of like Wish or Temu or whatever, right. Like where there's you're testing out significantly different products. I mean, even though they're, they're iterations of those products. It it doesn't feel like you're testing out like a, like an improvement to the product as much as just just different versions.

Where my mind went was like, let's say you have product A and you maybe only have five products. You just, you know, this is your niche, this is what you have. You have your product and you're testing out, how do I make that product better? So in my mind, I was looking at it from that perspective. So it's less of an acquisition play.

Now to me that sounds like something that you would see more on the LTV than you would on the kayak, right? Like in theory then you have people sticking around longer. They're referring people. You're decreasing kayak because of word of mouth. But the actual advertising.

Kayak might not change from that. But you would see that on an LTV because more people are sticking around, they're buying more often, etc..

Eric Seufert: 37:38

Yeah, but but LTV goes up, the ROAS goes up. So this is all marketing. Like I don't I don't I don't distinguish I don't distinguish between these two tactics. Right. So like but I'm not necessarily talking about like let's make the product pink.

You know, we sell soccer shoes and they're going to be pink with like you know, red stripes, like changing the actual fundamental product I'm talking about. More like the customer journey. Right. And so just like, like finding ways to optimize that. But like if you're talking about digital products, I mean, like everything can be dynamic.

I mean, like dynamic pricing, you know, there's the registration flow. Like all of this stuff can be dynamic. And so I'm talking more about like, you know, make optimizing that flow. And in so doing the advertising is more efficient. And therefore your ROAS is better.

And then probably you're increasing your bids. And like this this is just this this endless cycle. I mean that's that's the job, right? It's like this, this, this constant recurrence of optimization, increased bids grow, spend, you know, and then on and on.

Christian Limon: 38:33

What do you tell folks with like smaller sample sizes or smaller, you know, target audience where they don't they can't iterate as quickly because they don't they don't have that much volume. Like B2B is in this boat.

Eric Seufert: 38:48

Yeah. Well, that's I mean, that's just a constraint, right? I mean, you just gotta you can't have you're talking like kind of very slow sequential building up enough sample making really big changes, right? Like it's not. Well, I'm going to change the ordering of the check box.

Like that's not if you have a million registrations a day. Okay. That might tell you something. You'd have to make sort of like big, bold conceptual changes.

William Harris: 39:13

Yeah. How would we think about this from the perspective of people using AI to optimize and improve these workflows? Where do you think AI is best utilized the way it stands today to improve the amount of testing that we're doing and optimizing creativity because it's wildly creative. But to your point, if you're not testing something intentionally, like, where are the best ways that we could use this? And that could be for either.

Christian Limon: 39:40

One of you mentioned, you mentioned headlines. That's a good one.

William Harris: 39:42

Yeah.

Christian Limon: 39:42

Headlines text things in text.

Eric Seufert: 39:47

I think like my where I've seen companies, you know, utilize AI to like great effect or. Our just in workflow improvements, right? So like I think a lot of people think like, okay, we're going to just, you know, replace a lot of the core components of our products with like API calls to ChatGPT. And that's going to ten X our business. Like this thing that I've been you know, this this I wrote an article a while back, but like it's kind of like a theme that that comes up a lot when I, you know, talk to companies is like, you look, you should look at AI as like an output accelerant, right.

Or an output magnifier like, and not something that replaces the actual effort. Right. As I've called this like exoskeletons, not cyborgs. Right. Like you want to, you want to you want to use AI in ways that like magnify and amplify your efforts, but not that replace them completely.

And so like there's a lot of a lot of workflow improvements that you can use AI to do. Like for instance, I mean, just just one that I implemented with the company I work with. You know, we we have a tool that just scans all of like chart movements, like top downloaded app is an app company, right? All the top downloaded chart movements, looking for the biggest rank jumps, and then going and trying to find those pages on Facebook, and then pulling out the ads from the Facebook ads library and then surfacing those in slack. Right.

And like, that's pretty easy to do with AI agent. Right? But that's still ultimately bringing those to us. And then we're using our human intuition to understand like, well, why do we think they jumped so much? Do we think it's the advertising and can we learn anything from the advertising?

Right. That's still putting a human at the center of that workflow. But it's just it's cutting out a lot of the manual effort that you would need to do to build a process around that without the AI.

Christian Limon: 41:31

Yeah, I noticed that last mile is where all the air I think is like built in is the your interpretation of like why sometimes.

William Harris: 41:42

Yeah, that makes sense. I think you even hinted at the idea of last mile. And again I go back to for me, I still look at things very differently between kayak and LTV because to me, when I'm looking at this, especially on the e-commerce side of things, what I do to acquire customers is very different than what I'm gonna do to make sure that I continue to them being in the company, although it can be different. But I will say that one of the things that I've noticed with a lot of creative testing, let's say that you are testing out, you know, thousands of different creatives. It complicates understanding whether you've acquired the right customer or not.

And so what I'm looking at this from an LTV standpoint and I think.

Christian Limon: 42:16

About the right customer.

William Harris: 42:18

Well okay. So that's where I'll get to like the right customer. Could be maybe one ad creative does a better job of acquiring a certain demographic. And that demographic has, let's say a $20 kayak but a $200 LTV. Another one might have a $40 kayak, but a $400 LTV.

And so to me, sometimes I might be over optimizing now for the lowest acquisition cost instead of the higher LTV by looking too quickly. When I'm doing too many ad creatives.

Christian Limon: 42:48

The what time frame are you optimizing well.

William Harris: 42:51

And so that's what I'm saying. So like I think that let's just say.

Christian Limon: 42:54

There's only a few variables in this. So it's like quite easy to to compare them.

William Harris: 42:59

But I think this is what I let me just say. This is where I think a lot of people forget to look at when they're looking at creative testing, is that they will look at which one is doing a better job immediately, and they're making these decisions. They're making these calls within a week where they're not seeing what's the impact on that creative at driving the right customers over somebody who has a higher lifetime value. So I would say that, like there's the initial analysis, but then there's the later analysis. And I like to go back and revisit some of the creative tests that we did.

I like to revisit them six months or 12 months down the line to say, did my hypothesis, my assumption hold true? Yes, it was a better Cacc. Can I look at this to say, has that impacted negatively or positively the LTV of those customers?

Christian Limon: 43:42

Yeah, I don't think that there's a like right or wrong. I mean in B2B there definitely is. But you know, when you have a large audience, it's like more on a spectrum that there is only more expensive customers and less expensive customers and more valuable customers and less valuable customers. And, you know, other than that, you're trying to get them as efficiently as possible and make them as happy as possible to, like, spend money. And there is no, you know, and also, if you call something right or wrong, like you have determined that you know, why like that, that result will like reproduce or it's like a it's it's projecting a lot more than, than what is being given to you, which is just this performed x, y, z.

And that's all it told you.

William Harris: 44:36

Sure. That's fair. Eric, any other thoughts on that?

Eric Seufert: 44:40

Yeah, LTV is just such a fraught like topic. I think, you know, it's really easy to get wrong? And you do need some. Like where I see companies make mistakes with LTV, it's like they're making all these changes and then they're constantly trying to adapt the model to that. Right.

And like you've got so you essentially have like different data sets. I mean, if you make every sort of like fundamental change you make is you have to kind of throw out a lot of assumptions about how they're going to behave over some amount of time. And it's like the more the more sort of like volatile the product is, or the marketing is like the sort of shorter the LTV calculation has to be, just by definition. Right? Because you're only able to really track these people that are similar in those kind of in those along those features that were used to target them.

So, so my sense is like, you've got to be mindful of that if you're just sort of like pushing all the data through this LTV model that has been trained on, you know, a year's worth of cohorts. But you're making like radical changes to your marketing mix, like every day or your, your the campaign structure or like the creative you're using like that LTV model is not really reliable.

Christian Limon: 45:46

I haven't worked somewhere where most of the places that I that I've ran marketing they the value was more front loaded. Sure. And there is like a long term LTV. There was like a year and a half two year LTV. But you know, most of the value is like behind you versus in front of you.

So even when you're off, like you, you're within a pretty tight range in terms of like where it's going to end up, you're fairly confident as opposed to, you know, there are some products, I guess, where like in the future, it's possible that you will be a really like a zeroed out outcome or a, you know, ten x outcome.

William Harris: 46:27

That's fair. We have customers that are on the other spectrum, so that's why I bring that up so often. One of them sold let's say saltwater fish supplies. And so there was some front load that the tank and the setup and whatever. But the lifetime value I mean we're talking three, four, five, six years later.

I mean, it just grew significantly, and we could see that very much in the cohort analyses.

Christian Limon: 46:50

Six years later. Well, it makes sense for some products, but I think Eric is implying that, you know, you're messing with the math to make it work for you.

William Harris: 47:00

That's fair.

Eric Seufert: 47:00

Six years is pretty extreme. I think that's, you know, my sense. Like, like actually the example that always comes to mind is like Casper, you know, like, okay, you're buying a mattress, you're really going to buy another one in another year. You know what I mean? Like, you might buy the pillowcase, you might buy the pillow, but you're not that that that that LTV is pretty front loaded to like that one purchase, right?

That's like one of the nice things about ecom is a lot of that stuff is is pretty front loaded. And and you have to there's common sense here, right? I mean, you could test a bunch of stuff and but you know, like with ecom, I mean, there tends to be like a pretty common sense understanding or delineation between, okay, when this purchase was driven by an ad and when it was driven by the previous purchase. And I think a lot of times it's just driven by the ad. It's like, oh yeah, I remember that brand.

I bought some stuff from them six months ago. Let me buy another thing. And so then that probably just belongs to that new ad spend. It's not really like long term value in that sense.

William Harris: 47:53

Yeah that's fair. Let's switch over to platform power and attribution changes. Eric, you have written about the control that Apple and Google exert over advertising attribution. And I might be stretching it a little bit, but to me at least, it feels like the Heisenberg's uncertainty principle, where it feels like the more that I know about one thing, the less I seem to know about something else. How does Google and Apple's control over attribution influence marketers ability to measure and optimize campaigns effectively?

Eric Seufert: 48:24

Well, what's funny is I think a lot of that might be reversing course, right? So, I mean, you know, Google just announced two days ago that they're not deprecating cookies now, right? They wanted to move everybody again. Again? Yeah, again.

They wanted to move everyone on to Privacy Sandbox, which is, you know, is not going to be as effective as probably as cookie based measurement, but that's that's an attribution methodology that they own. Right. And Apple has skadnetwork and now the ad attribution kit. There's a lot of value in being able to control how people attribute ad spend, especially if you operate a scaled ad network. And so there's there's a lot of incentive to want to do that.

But I think that I my sense is like they both kind of pushed the limit in different directions, but they both pushed the limit. And they're having to like step back those ambitions.

Christian Limon: 49:10

Why would Google do that if their their business model is predicated on on advertising?

Eric Seufert: 49:15

Well, it's predicated on advertising but not open web advertising. So they make way more money on search and YouTube. I mean, just on a, on a, on a, on a margin basis than they make a network. But now also on an absolute basis, a network is a much smaller proportion of the Ads business now than it was five years ago. It's 11%.

Now YouTube makes more money. And so I think they they see that as a relic. They just want to drive people into the O and O. And so if they diminish the power of open web attribution, that's one way to do that. But again, now they're preserving cookies.

They're not going away. But I still think they have another lever to prevent people from using open web advertising, which is search as a distribution mechanic. And you see what happens has happened to Google search. When you go to Google and you do a search, you don't get a big list of links anymore.

Christian Limon: 49:59

Yeah, they're AdMob piece has gotten deteriorates as a result.

Eric Seufert: 50:05

Yeah. I mean but that I mean, I don't know how how big that was ever. I mean, my sense is like they kind of capitulated with that two years ago when they started routing the Google demand into other mediation platforms. Right. So like max and level play, I think once they gave up that real value proposition because that was that was historically that was exclusive to AdMob.

On with real time bidding, you had the only way to get access to Google demand in in was to use AdMob as your SSP and they, they they gave that up, I think a year and a half ago, two years ago. And so that kind of, to me made it seem like they weren't really focusing on that anymore.

William Harris: 50:45

Christian, you were at one time the number one spender on both Google and Meta, or at least rumored to be. I don't know if I can confirm that one way or another. Yeah for sure.

Christian Limon: 50:57

Google top five overall?

William Harris: 50:58

Sure. Very close.

Christian Limon: 51:00

At least 1 or 2 mobile advertisers.

William Harris: 51:02

How how did you get around the idea of like the walled garden of attribution by.

Christian Limon: 51:12

Doing one thing at a time. So I mean.

William Harris: 51:15

But you're also doing thousands of things. How could you do one.

Christian Limon: 51:17

On the same channel?

William Harris: 51:19

Okay.

Christian Limon: 51:20

I we didn't get to Android. We had gotten to become a $3 billion GMV business just on Facebook. And it's not because, like, wow, imagine if you would have gone elsewhere. We wouldn't have been able to get that big if we would have spent on like ten places because we wouldn't have known, you know, we would have introduced so much fraud.

William Harris: 51:38

That's fair.

Christian Limon: 51:39

Like, we only got good because we did one thing.

William Harris: 51:43

I feel like a lot of people fight against that, right? They have so much of a desire to be everywhere. How do you was this top down or how were you able to convince, you know, the CEO that you're going to go still all in on one channel and you're like, no, we're going to ignore Google for right now.

Christian Limon: 52:01

It was the it is like thesis driven. It was like we we just had like influenced coming up with the DPA. So and nobody else was like really able to tap that because no one else had like a product feed. And even if they did, they didn't know that that was so beneficial for ads. So they were just uploading like 20.

You know, they were doing like things that are not beneficial for, for ads because they didn't understand that it was very beneficial.

William Harris: 52:33

At what point do you guys think that we just say? Attribution is good enough. Move on to better strategy.

Christian Limon: 52:40

For. What do you mean.

William Harris: 52:41

Just in general? Like marketers. Like, I mean, marketers are looking at, you know, their ga4 results. They're looking in Shopify. They're looking at North Beam and Triple Whale, and they're looking at all these different.

They're running the holdout studies at some point in time. I do feel like there's like an analysis paralysis where nobody wants to make the call and actually just do something. Like, at what point do we say, look, the attribution is good enough. Just take action, do something.

Eric Seufert: 53:06

I think they should be doing that earlier than most do. But I think like Christian makes a good point. And, you know, you kind of highlighted it. It's like you can engineer that by focusing on the core channels and not diversifying too early. I mean, I think people make a mistake when they conclude that they need to diversify, like diversifying will, in and of itself inherently deliver a lot of a lot of efficiency.

And that's not that's not that's usually not the case for a number of reasons. One is you develop specialization in the platform that unlocks efficiency. Two, you add a lot of overhead. When you onboard new platforms. You've got to do data integrations.

You've got to learn how to make, yeah, different creative styles. That's overhead. That's a that's a drag on efficiency right. And three like there are a handful of channels that you could rely on exclusively. And if you only rely on those then your attribution is almost a solved problem.

Like you don't even really need a lot of sophisticated modeling. You know, where the traffic is coming from. And if you're only on one, which is Facebook, you're probably covering 90% of your Tam anyway.

Christian Limon: 54:13

Yeah. So you can make it the whole distance with that.

William Harris: 54:17

That's fair.

Christian Limon: 54:17

That was that's my party. If I'm somewhere that can scale very large is. I don't want to get good at something that cannot scale extremely large March. It's like venture. It's like all these things are, you know, barely work.

So when it does work, it needs to work massively big.

William Harris: 54:40

Yeah. With stability. Again, a lot of efficiency can be found on TikTok for a lot of brands in the e-commerce space especially, I can't speak outside of that. Right. But it could be very high or low.

Like it could be very good one week, very bad the next week. And it doesn't feel as stable of a platform from an acquisition standpoint as a lot of like meta would be or YouTube would be for a lot of things. And so your point, it can scale potentially, but maybe not as stable.

Christian Limon: 55:03

It's hard to to control for that piece. The DPA or the performance plus like volume kind of helps to control for that. But even when you have volume, say in a game, as a game publisher, every new title you don't know necessarily you can estimate how it's supposed to do, but you don't know how it's going to do. You can't you can't predict very well.

William Harris: 55:29

Do you think marketers are too reactive now to all of the different nuances in platform changes, as opposed to maybe just leading with strategy, and that maybe sometimes we're trying to fine tune things too much instead of just saying the issue is not the 1% incremental that you can get from this little tweak. As much as your product sucks or your landing page sucks, like this is the issue you need to actually focus on.

Christian Limon: 55:57

They can't control the like, recreate the product though.

William Harris: 56:00

Sure.

Christian Limon: 56:00

Like if they're an enterprise company or, you know, games like most like a junior UA manager is not going to have any. Sure. You know, hey guys, let's like, like create another product.

Eric Seufert: 56:15

That's why I generally advise like startup companies. Like, I think the CEO should be running Facebook ads up to like, some amount of money. I think you need to prove out the product market fit before you bring a marketing person in. Because like, first of all, it's just really beneficial for the CEO to understand, like the mechanics of growth, which, you know, customer acquisition is and Facebook will be particularly for like digital first like app companies. But second, I mean you need to earn that first marketing hire.

You need to earn that first marketing hire with like more work than the CEO could feasibly do, because there's so much to optimize on the marketing side because you're growing so much.

Christian Limon: 56:50

I like it, yeah, that has a ton of benefits in like being able to communicate to your future growth people to, you know, and your discussions with like investors in the future. I think that's like a, a massively underexplored thing is the, the attributing the the associating the right amount of value with like being like having some kind of proficiency in that. Like, yeah, like just yesterday actually all the time I'll see people, you know, comment on LinkedIn saying, oh, we got this many users with no marketing. And you know, I'll even reply just because it's it's a fact that, well, you're going to need to market if you want to get bigger than some like alternative in the future, right? If you get like more ambitious, only like the way you get better at it is through volume and repetition.

Intentionally so. Like the longer you like, if you don't start, it's just going to take you longer. You're going to be pretty bad for a longer period of time, or it's going to take you. It's going to take you're not going to get good for a while.

William Harris: 57:59

That's true to everything, right? Like you have to take the shots in order to get better. Gary Vee says this even with, you know, running a podcast, for instance, and it's like your first 100 are going to suck, but you don't get to 101 until you do the first 100.

Christian Limon: 58:13

Why are people so excited to not do the 100 thing?

William Harris: 58:16

Yeah, you have to get over that, right? You have to say like, okay, well then I'm excited to do these first 100 bad things in order to get to the 101st one, that's going to be good or better.

Eric Seufert: 58:28

Yeah. I mean, I remember when I launched my blog, I wrote like ten articles that I just never published. I just wanted to, like, get the muscle going. And when I wrote my book, I actually saved the first chapter for last, because I knew that that was going to be the probably the most important chapter in terms of like, you know, convincing people to keep reading. And I knew that, like after I wrote ten chapters, I'd be better, you know, than I was going to be having written zero.

And I mean, like, it's just it's just it's just life, right? That's that's that's the, the, the human growth process, like, you know, and and just kind of you have to that's why it's really good to do stuff that you like. Because if it's fun the whole time, then at least you're getting something out of it.

William Harris: 59:09

I've never heard anybody write the first chapter last, but I really like that idea and your rationale for doing so.

Christian Limon: 59:14

That's what I think most people I think people should start their careers in consumer because that's what you'll that's where you'll get the volume and the, the, you know, pattern recognition and build the muscle and how to do things. And then you switch over to whatever you are, you know. Because you'll, you'll have seen like have some framework for how funnels work and how channels work and what an a B test is and why it's helpful and and like what it what you shouldn't count on it solving.

William Harris: 59:46

And it's more fun because you get the dopamine hit faster.

Christian Limon: 59:49

I mean it's not fun to like have really high failure rates. Sure.

William Harris: 59:54

That's fair.

Eric Seufert: 59:55

I mean, consumers is hard and it can be like heartbreaking and defeating, right? But like when you're working on something that people are using and, you know, like the coolest thing for me, I mean, I'm sure you experienced this more often than I did, Christian, but like going on a plane and, you know, walking down the aisle and seeing someone playing a game that your company makes, like that was the coolest feeling. Or sitting next to somebody, you know, and they're playing the game that you're coming. That was just a really, really cool feeling. And you know next level.

Yeah that's that's true. Rovio. I had that sensation more but but like you don't really get that with B2B SAS. I mean I guess you'll be sitting next to somebody using notion or something. But like it was just it was just a really fulfilling feeling.

It is. That's that's the nice thing about working in consumer. It's like you're working on stuff that you yourself might use or like your friends might use, and someone says, oh, you work there. Like you work there. That's so cool.

William Harris: 1:00:51

It's true for your kids, too. I recently had, you know, I've had a lot of really fun guests on the show. But I tell my kids about some of the guests that I have, and they're like, who? Like, what is that? I don't know anything.

But I had I had Brian Olson on here a couple weeks ago, CEO of PacSun. And my kids are like, I've got, you know, teenage girls like Pac son really. Like, I want to watch that episode. I finally got them interested in maybe watching the podcast.

Christian Limon: 1:01:14

I used to love people at the airport. Those like the people who work there would recognize all the wish stuff and talk to me about it. And I used to I used to like that. Or cab drivers.

William Harris: 1:01:25

Barbers that recognition. We're running close to time here. The I want to end with, let's just say, the future of growth going beyond like the normal metrics and things like that that we're looking at. Christian, specifically, you have talked about how growth leaders need to think more like portfolio managers. How do you evaluate, let's just say what does that mean?

Like to think more like a portfolio manager as a growth marketer.

Christian Limon: 1:01:51

I don't know about portfolio manager. I, I think that. Teams need to be like the results that I, that I've done has only been from being cross domain and from like developing a case that's like cross domain and collaborating with like multiple skill sets. And like if I'm pigeonholed to, to like only one channel or just one thing, that's like really going to limit what I'm what I can do. Like, Eric's talking about funnels.

He's talking about, you know, ChatGPT copy in creative and like the the the wider you can, you can pull resources into the, the more effective you can be. But the more the more the narrower you are. Then you're only going to find whatever's possible there.

William Harris: 1:02:44

What? Eric, what do you think about the future? Like what emerging systems or metrics? Or like how do we think about growth in the future, knowing that a lot of let's just even say, like a lot of what we do is being automated by AI, right? Even a lot of the analyses and things like that, and even the ads platforms, within five years, a lot of the ads platforms are just going to do significantly more of that without you even having the ability to have manipulation or control over it.

Eric Seufert: 1:03:12

Yeah, but I think most of that's a good thing, right? I think people tend to overestimate their ability to sort of. Deliver better performance in the platforms. Right. It's like, you know, it's kind of like in the man versus machine paradigm.

You tend to want to root for the man. My sense, though, is I want to lean into the things that the platform is going to do better than me and free up that time to do the things that I can do better than a machine. Right. Which are probably going to be like the more creative endeavors, like the more creative tasks, right? So I think that's kind of the future of growth.

And there's not like a one, there's not like some tactic. Right? It's just a sense of like, how can I use these tools to the best effect, to free me up, to go think about things that really drive like value in a fundamental high concept way.

Christian Limon: 1:03:57

There's also a difference in what what something does and what it's supposed to do. Like.

William Harris: 1:04:05

Sure.

Christian Limon: 1:04:06

I think Performance Plus is a good example or it works really well. And like consolidating campaigns works very differently now than than years ago. But for years, they would still underperform relative to stuff that that had was running like very old school style. And there was like no need to rush, you know, switching over because it's supposed to do something or until it, until you, until you get the feedback that something is, is outperforming and then you increase that or scale it up or, or create some kind of action.

William Harris: 1:04:48

Yeah, that makes sense. We've talked about a lot of different myths. Is there something that we haven't talked about that you're like, I want to at least make sure that we're talking about this too. And I'll start with you, Christian.

Christian Limon: 1:05:00

Yeah. I want to touch on something that Eric talked about, about the systematic. Systematic. I have a big thing on that also. But the bar is so low to be systematic in growth.

I like I have this opinion that it's like a almost like a second rate, a separate team, that they'll have a different bar of, of system of like documentation, how they do things. Like think about like the cadence, how much room there is to, to systematize. Like how many campaigns do you launch every week. How do you how do you come up with ideas? Like like how much?

How do you start campaigns? How much are you editing them by? When do you edit them? When you have time. By how much are you rounding?

And all these things that are like making judgments that are like just very like they don't need to make a decision. They and you don't even need to automate it necessarily. It's just systematize it. You can categorize it just like take it off your off the your brain as far as deciding what you're what you're going to do. If it works.

William Harris: 1:06:13

That's true for a lot of things, right to a point that if we could systematize things, we're a lot more effective. The idea of like the 1% a day rule, right? It's just that idea of just.

Christian Limon: 1:06:22

Like a product engineering. They don't they don't work like that. They're not just like, you know, doing things when they have time and by X amount that they want or that they kind of feels right.

William Harris: 1:06:36

Yeah. Eric. Thoughts?

Eric Seufert: 1:06:38

Yeah. I mean, my just just like philosophically, I'm just very allergic to hacks. Right. And like, I mean thankfully we're we're post growth hack era. We're in like the post growth hack era right?

Like I mean that that was just so obnoxious though. Like to now we're in vibe coding, which is probably even worse. And vibe marketing but but like that that that growth hack era I think was just so destructive right from like you know, a good practices standpoint and like, you know, establishing like kind of very, very public like best best practices. And the reality is like what's that?

Christian Limon: 1:07:16

CEO expectations some of them would want you to chase these things.

Eric Seufert: 1:07:20

You know how many marketers got fired because they they got hired and the CEO was like, oh, I heard about this new growth hack. Why don't you just do that? And we don't need to spend any money on marketing. And six weeks later. But, you know, I think, like, you know, systematic thinking, like, the thing is, like anytime you even there are these sort of like apocryphal stories that float around like, oh, this company grew off the off, the off the basis of this growth hack.

And then you go in and look into how the company actually grew from like zero to its first million users. And, no, they got like $1 billion worth of free media, right? Because it was just a novel new product idea, right? It wasn't it wasn't because they did XYZ thing. And then, you know, there are too many cases to count of companies that are trying to, you know, pull off the, the, the sort of like the velvet carpet, like velvet rope growth hack and get celebrities on board.

And or I'm going to limit the number of people that can join and drive FOMO. That just doesn't work. I mean, this is just a very it's kind of boring. And it's supposed to be like growing a product should be boring, systematic, kind of slow, very sort of like analytical. If you're if you're experiencing, you know, if you're experiencing sort of like astronomical, you know, astronomically like high, you know, incredibly fast, uncontrollable organic growth, like you have to plan for when that stops, right?

You can't just project that forward, you know, sort of into the future. And so I've always thought the job should just be like like almost like a librarian. It's like boring, slow, analytical and like repeatable and dependable.

William Harris: 1:08:52

Prime energy drink is a great example of what you just called out to, right? Where it's like absolutely explosive growth, but you're not prepared for when that stops.

Eric Seufert: 1:08:59

Right?

William Harris: 1:09:02

I've absolutely enjoyed talking to you guys. I think that this has been something I've been looking forward to for a few weeks now. If people wanted to follow you or work with you in some way, what is the best way for them to get in touch or follow you? And I'll start with you, Kristian.

Christian Limon: 1:09:19

Well, I'm on X, so at @CPlimon. CPlimon.

William Harris: 1:09:26

Awesome. Eric.

Eric Seufert: 1:09:28

Yeah I would say go to mobiledevmemo.com. It's my my website. I'm on X you know LinkedIn all those places.

William Harris: 1:09:37

Yeah. Cool. I appreciate you guys jumping in here, digging through some interesting myths and how we can hopefully get a little bit more intelligence into the marketing world. It's been fun.

Eric Seufert: 1:09:48

Thank you for having me.

William Harris: 1:09:50

Thanks, everybody for listening. Have a great rest of your day.

Outro: 1:09:53

Thanks for listening to the Up Arrow Podcast with William Harris. We'll see you again next time and be sure to click subscribe to get future episodes.

‍