Today, on episode number 787 of cxo, talk, we're discussing data
and intuition. The role in marketing, we call
that quantitative marketing. Our guests are two folks from
the Columbia business school at Columbia University in New York
City ODed. Netzer is the vice Dean for
research and a professor and author of the new book decisions
over decimals and Amy Jake is the chief marketing officer of
the institution for me marketing.
Is that part of any organization that is on charge of
facilitating and enabling successful transaction with the
customer successful? Whether it could mean profit if
it's for profit could be engagement for example if it's
non profit but the ones who are in charge of making sure that
such transactions relationships do exist and and now you want to
layer on it. The quantitative part of it in
order to create this. Successful interactions.
I really need to understand if under firm who's on the other
side and it's not exactly the other side will have my
partner's, my customers. And for that, I need a lot of
data, a lot of information about my customers and who they are
and good understanding of who I am, what am I capabilities and
that again requires data and analytics.
And specifically the analytics is often done in order to try
and understand this match the met with the match between the
customer. And and they the firm and what
the firm has to offer and even changing what the firm has to
offer to make sure it matches. What what customers want?
Particularly in, in, in, in recent years where a lot of the
effort went over Z, he's getting better and better understanding
of the customers because the better and the more we
understand the customers, the better, we can actually serve
them, what it is that they want, and that requires a lot of data,
a lot of analytics and a lot of quantitative marketing.
Amy, what is the impact of this data driven approach on your
work right now? There was a period of time where
it was somewhat difficult to explain to senior leadership the
value of marketing. The increase in the data that we
have, the ability to show quantitatively, and to connect
an action that marketing is taken to a bottom line impact.
That's what a lot of the data has done.
It has allowed us to be better marketers, we understand a lot
more. So dead was saying about the
audience, we can engage in different tactics and strategies
personalization is so important. You know, we are inundated with
material and it is very difficult for us as humans to
have that, much flying at us and to be able to figure out what to
pay attention to. You know, cognitively, if
there's this influx of information, we have to pick, we
have to sort, we have to think about what makes sense for us to
pay attention to what limited time do we have.
And how We react to that and in Mass marketing sometimes you can
get that benefit with your audience but more often what
we're seeing is that the more personalized, that the marketing
is, the more likely that person who is otherwise inundated will
take a moment will stop and will engage with you.
And the only way to do that is to be able to leverage the data
that you have to say, this is the right course, for this
person, this is the right path. This is the right Next Step for
person a But for person B, it's going to be a little bit
different and every time that our audience or consumer or
customer has that experience, the more likely, they are to
come back because there's a benefit for them.
They're getting what they need. They've seen that it works.
They know it was worth their time and so I think data both
helps us on the strategy side. Develop better strategies.
It helps us on showing results to senior leaders to, to
shareholders to stakeholders and it helps Us to deliver a better
experience to the customer. So they're all wrapped up in
sort of one larger marketing effort.
But I think those three components and thinking about
it, in those ways is really helpful.
I've been teaching the marketing core for quite a few years here
at Columbia business. School is probably the course
that has changed the most actually em out of any course we
teach. And the reason is that the world
of marketing is changing tremendously.
And if you think for example about advertising, The
advertising. Again, many of us have watched
madman, right? And that's that's really the old
days of marketing. But even up to the early 2000's,
the center of advertising was here in New York was in Madison
Avenue. If you think about what are the
three biggest advertising companies in the world then not
anymore in New York and there are not any more advertising
company companies, there are Google Facebook and I'm an
Amazon. These are these companies in
Silicon Valley and they sit in, in, in Seattle, which means
there are technology companies, and with that comes the
quantitative aspect of marketing.
Amy made a very interesting point, she spoke about the
enhanced ability to connect the marketing activities, back to
impact on the organization. Can you talk about that?
Because that's what Business Leaders ultimately care about,
right? We're spending.
A certain amount of money on our marketing and what is the
ultimate impact? And how do we measure that?
The one of the biggest impact of quantitative marketing on their
world of marketing? Was this notion of our eye and
there is return on investment, right?
I mean, will the famous quote and, you know, half of my
marketing expenses are? Well, spent, I just don't know
which half and then came the world of online marketing,
right? Then came the world of digital
marketing and with this promise of we're going to see what
people clicked on. We actually going to see whether
someone clicked on the ad where the summit eventually even
bought some finally, we can get our iron marketing, we know what
is the impact on of a marketing on consumers?
I think it's true. It we have moved a long long way
in the ability to show return on investment on marketing.
And we, it is much easier now to have that conversation.
And I think that marketeers that haven't taken that step have
truly left behind. And if still struggle with are
within their organizations, I will.
So the mention that that promise may have been even too strong.
It, we now started seeing the pendulum shifting on that
statement in the sense that the fact that I see whether customer
clicked on an ad doesn't yet fully tell me whether it's a
full return on investment, right?
It could have been that this customer would have bought
anyway, so we need some sometimes more sophisticated
quantitative techniques in order to truly ask questions, like
attributional in current through a mentality.
And whether we truly have Roi on these.
It seemed as if we solve the problem and now we realize the
problem is, we have way way ahead, but there are still
things to be done, which is where the world of quantitative
marketing works on today. A few years ago, a marketer had
to say I see a correlation but I may not be able to show
causation and now I think you're right there.
Are there still are numerous questions that we have there's
still challenges with attribution models but I think
we are much closer to being able to say It's not just happening
at the same time in, isn't that nice?
We're able to at least connect often some of those dots and
say, well, I know that when my marketing spend goes up 13% 27
percent of customers, you know, purchase more or 13, you know my
marketing spend goes up 13 percent and 50 percent more
individuals enter this online store.
So we're able to do it in a way that you know, if our
attribution models are right and we Have the right technology at
some point in the path we can say there was a direct
connection and we know that it's actually going to make a
difference. And I think directionality is
important, but directionality with, you know, some very
specific models that show that relationship is what ultimately
gets you farther in terms of your budget, in terms of being
able to enter into new markets, and really buying into sort of
the confidence of the c-suite, I think, which is super important.
I think one of the things that truly help with that, and
particularly, in the world of digital marketing is the ability
to run these A/B tests, right? Yes.
I mean it's very difficult to say, you know, how about my
customer going to see one store and a half of my customer going
to see a different version of my store.
You know, my story. My story is very difficult to
play with it in the online World it because it became very easy.
And you know I split my time between Colombia and Amazon, I
spend some of my time at Amazon advertising, the practice at
Amazon advertising is running. These A/B test these, let's try
version a version version B, and when we do that, we truly can
get to RI. We truly can get to.
What is the difference between offering the customer version a
and version B? It's still easier to doing
digital environment where we can take, you know, we have 100,000
visitors, we can split them half or any any way.
We split them randomly and see, throw to to different customers
different different offerings and see how it works.
It's still a little bit more difficult in the offline mode.
But even there, we are making progress toward thinking in the,
in a language of A/B tests in the language of testing, in
order to measure Roi. And in order to measure, truly
what customers want? I'll give you just an example of
that. Several years ago, I was working
at a large company and my role and my team's role was focused
around telling investors the story of the success of the
business. And when we had our hypothesis
about which creative would work, The best to get them to engage
with our financial results. I would have thought one thing
and we a be tested over a period of about a year.
So that's four, four quarters worth of data and we actually
found that in telling the story with our IP, so the show's
themselves when we use that as a creative investors, now, this
isn't an audience that you would naturally think would pay more
attention to, you know, a character on a show than a
financial times headline. But when we Talked about our
efforts. And when we use the creative,
when we use our IP, we found that the interest in the
engagement, in the information, the financial information about
the company increased, you know, exponentially.
And so I think to your point that's an a/b test that 10 years
ago or 20 years ago, one, you wouldn't have been able to run
but to you wouldn't have hypothesized that.
But when we just have the ability to continue to test and
swap in and swap out, Out and we see, very clear results in the
data, you know, it gives us confidence to both try new
things, but then it gives us Security in the decisions that
we make. Because we know now that we can
see that that happening, please subscribe to our YouTube channel
and hit the Subscribe button at the bottom of our website.
So you can get our newsletter and we can keep you up-to-date
on upcoming live shows. What about the data?
What kinds of. Of data, should marketers be
collecting? What's the best way to get that
data and ODed? Can you take us a little bit
behind the scenes of the kinds of models that you're looking
at? And again, my real interest here
is the impact of that data. And those models on the real
lives of marketers, hey, it sounds like a reality TV show,
right? The real lives of ders.
We have a fire hose of data, right?
We have more data than we ever had before.
It's still the situation that we don't always have the data.
We exactly need. We always sometimes, looking for
the right data, we need there is this something we talked about
in the book that you mentioned earlier, then we call the the
certain teammate. There was this belief that
finally when we have all of these data will get to the
certain decisions. Certainties I me that we still
make decisions under uncertainty, but we do have much
more data than we ever had. And Specifically, the data that
marketeers value and cherish tend to be data that tells us
about customer preferences, being able to measure and
understand customer preferences and the reality of Eden.
And the reason why it was difficult, it may be still is
difficult in some ways is because of heterogeneity,
because customers are so different than 0 to customers
that are the same. And, you know, I always tell my
students to know they and always Find within my class.
This type of students, those who treat the Apple Store, like a
like a called. And those who would never be
caught dead in an Apple store, right?
And the marketeers needs to understand this and he's trying
to stand this often with limited data, depending who you are, if
your Google or Amazon, you have a lot of data.
If you just, if you are media Outlet, you may have less data
about about the customer and who they are and we tend to see
unlike. If you go to data for example,
in finance, I can always Go one more year and have longer data
about this about stocks, right? I can also just do an analysis
of the entire Market when it comes to stocks, when it comes
to to Consumers. A I cannot do data just on the
entire Market because of the heterogeneity that I'm talking
about. I do need to understand each
customer in their own preferences and I'm limited with
respect to the history because if I'm talking about a travel,
for example, if I'm Expedia one of these dry only see EU, you
know, five ten times, that's all I have.
So I need to work with that length of history of what I
observed about the customer, but and that's where economics helps
so. Or statistics, there is a
trade-off when it comes to data between how good your data is
and how complicated your model needs to be the better data, you
have the simpler, the model you can use.
If we run the A/B test of a just talked about before.
All I need to do is compare average.
Is a customer in the control but that much customer in treatment
but that much, that's it. I mean, level of math, that that
is, you know, a sixth grader. Could do if I need now to build
a model where I am again a Expedia and I'm trying to
understand customer preferences from this visit and the previous
visits and so on, that's where I need a much more sophisticated
type of modeling and type of analysis.
The other, maybe distinction I want to make about data.
Is it extinct distinction between structure data and
unstructured data until fairly recently, Circa 2010, most of
our analysis was done on what is called structure data structure.
Data are numbers data that comes in the form that we've by the
way, generally think about data in Excel, or in a table with
numbers in it. But if you think about it,
majority of the data we have as business people, as marketeers
is actually unstructured data, it can Ones in text image,
video, Audio customer calls us to the call center.
And we have data in terms of tens of thousands, if not
hundreds of thousands of ads that we can, we can analyze ads
are images and text, we have company reports, different
estimates, depends who you ask, but 80 to 95 percent of data
available. For business is likely in the
form of unstructured data in text audio video and image.
And it is only since 2010 that we actually know how to analyze
this data at scale. I mean I'm structure that
existed probably since the Ten Commandments right.
We're waiting on a stone but but really they build it or to
analyze this data at scale, came with machine learning type
machine, learning methods, type tools and and we seeing more and
more companies are using it. And, of course, recently, the
whole chechi PT and generative AI is an example of Jing,
unstructured data, Amy your marketer.
I'm assuming that you are not and expert statistician and data
scientist. And so given what ODed was just
describing in terms of the quantity of data, the quality of
data, what can marketers do to take advantage of these
important points? But without Without having to be
a deep level statistician. For example, don't be scared of
data. I'm not a statistician as you
very correctly, pointed out, I never have been.
But I think for marketers, when we talk about quantitative
marketing or we talked about data-driven marketing, for some
especially individuals early on in their career, it sounds like
they're going to have to be gathering the data sets.
It sounds like they're going to have to be building the model.
It sounds like they're going to have to be interpreting.
And I think we're very lucky in that, we have a whole generation
of individuals who understand data, who are data scientists,
who are insights and research-driven colleagues.
And the more that we can develop very close relationships with
those individuals as marketers the better.
Because most marketers, I know aren't as you said
statisticians. But what they do know is how to
look at data and I know we're going to talk a little bit about
this and and say, well, based on my understanding of the
audience, this seems odd, let's explore that or they have a
question that they know they want to answer.
And they know that the data is an important part of that and so
they can ask for help in building the model.
So the first is, you know, you don't have to be the expert.
Don't be scared, don't not use it.
Use it, find a partner, find a colleague, you know, find
somebody who is the expert and the second is think about the
data in Meaningful way. So you as a dad said, there's
infinite data available so it really is about what are you
looking for? And then when something and I
know Dad's going to talk a little bit about this, when
something doesn't feel right, ask more questions and a good
example of using kind of numbers and qualitative elements
happened a couple of years ago. I worked in a publishing company
and it's very common for people to unsubscribe from
subscription-based businesses, including magazines.
And so, you build a model and, you know, that churn will be X
based on historical data. And you know, that there might
be some macro or micro conditions that will impact that
Year's churn and you go out and you acquire customers, knowing
this percent is going to leave and you need to have, you know,
X percent of new people coming in and subscribe.
But as a marketer it's not just about making sure that you can,
you know, fix the revenue equation on both sides.
It's also about understanding your customers.
So you have this model we have this model, we know that X
percent, you know, are unsubscribing.
And so then you have to dig deeper to think about why what
is it? That what's the reason that you
know, they've chosen to cancel this subscription and we could
get a little bit of insight in asking structured questions and
our Hypothesis was it was a value situation.
Somebody decided that that product that the magazine was no
longer worth what they were. Paying feels pretty simple,
right? But we actually had done some
brand research and had some qualitative components and what
the qualitative components told us was that actually had nothing
to do with value, it had nothing to do with the product, it was
about unfinished ability. So people felt that this
particular magazine Worth the price worth, you know, worth had
great value, but they couldn't finish it, and they would stack
it up in a pile. And they would say, I'll come
back to it, and they didn't. And every time they looked at
the pile, it crew and their guilt.
So this is an emotional reaction.
Their guilt over not finishing, the magazine was actually, what
was contributing to canceling subscriptions, so that Insight
is not in sight. You were ever going to get from
structured data. It's not even in sight, you were
to get from some of your survey questions.
It came exclusive these through some qualitative conversations
and there were some solutions to that digital.
Combated the problem a little bit because people can't finish
the internet. So if you can't finish a digital
issue of a magazine, you didn't feel as guilty.
You know, you can introduce newsletters and quick quicker, B
of content and summaries and things like that.
But I think the other important thing is I and I want to really
Reiterate. This is that there's the
structured data. There's everything, you know,
there's quantitative and then there's this other qualitative
element. That is equally as important for
marketers to pair together and I think it is important that will
think about data in the most General sense of the word, a
Tamiya conversation with a person is data or just maybe
want to touch on one thing with respect to the, you know, when I
build the end model statistical model and I truly mean it.
I would I would actually Really go to a me, first to tell me
this model is right, before I would go to a statistician to
tell me whether it, whether they think my model is, right,
because the True Value comes from, let's say it's a model
that predicts click click through on an ad and the model
showed, you know, twenty percent click-through rate.
It's the decision wouldn't know to tell me that 20% clicks.
All right, on an ad is unheard of, unless you're Ed truly
offered money to customers. No one.
There's not, there's no one in five.
Customers click and click on a man.
I needed. Cat ears.
I need a me to tell me. I don't know what you've done
wrong, but I can tell you this is wrong because it's it's
impossible. That customers. 20% 1811 in five
customers would click on your ad.
We have a couple of questions that have come up on Twitter,
actually, to folks, Chris Peterson, and our Salon Khan,
who are both regular listeners of cxo Takin.
So thank you guys are asking independently.
Essentially the same question and that is, where do we draw
the line for? Privacy in terms of data-driven
marketing, that's from Chris Peterson.
And arsalan, Khan says how, and who should strike a balance
between collecting data that can affect privacy and who decides,
what's private? What's not?
And also, then you have the issue of biases.
So can we talk about that? And then we'll switch over to
quantitative intuition privacy is an extremely important topic
to think about to discuss. To realize that that companies
have huge responsibility when we are collecting individual data.
I think there are, there are there is the issue of data being
shared and then there is the issue of data being used by, by
the company. I mean despite what actually
most people think most, at least the large companies do not sell
data. Now, I'm not sure it is, by the
way, being done because and because of pure Ethical issues
of because data is just way too expensive to sail, the use of
data by by company. By again, the thing about the
top companies like Facebook, Google Amazon is so important
and useful for their own purposes that actually selling.
It would be, would not be very wise.
So generally, at least from these companies data generally,
do not do not flow around. It generally does tend to stay
Within These these companies but even there they should treat it
with the with the Ability that these data should be treated
with and that's where recent regulation came about first
party data where you're allowed to use your own data.
But you're you're not allowed to actually use someone else's
data, which by the way in, and of itself, there is an
interesting tweak there. It's on the one hand, it sounds
right. It's indeed protects customers.
In terms of their data, being being in floating around but it
also increases, maybe the Gap between the top companies that
have really good first-party data their own data and the
smaller companies that don't have their own data.
And that's what that where they could have closed the Gap by
getting data, Maybe from, from someone else.
I do think that we shouldn't let the cat Garden milk there.
I think we do need regulation there, right?
And Europe is much better than the u.s. in that science.
And we've made that mistake previously, with social media,
where we let the the Guard themselves.
We've made it with the privacy and I think we need to move
there and I'm happy to see if there is generative an RA.
I this conversation is starting already now.
And not once this is fully fully out there and I mean, I don't
know if you have other selves. Yeah, I mean it going to an end
even kind of larger question is, who owns the data?
We're now in a world where our Behavior has been tracked for a
very long period of time, and there are reams of data, people
probably know that I prefer baked macaroni.
And cheese, you know, over brussels sprouts, but who
doesn't, you know, that, that data is gone.
It's with somebody, I hope I get an ad for macaroni and cheese.
I will let everybody know shortly, but I think there's a
larger question as we think about a next-generation, right.
Let's talk about kids for instance, for whom maybe there's
some data, there's not a lot who owns that data, who should own
that data is my child, the holder of that data or is it the
platform? Who's on which they're engaging,
and there have been these really large high-level conversations
around, you know, should I be able to own my data and I choose
where when, and how to monetize that?
Or is there a trade-off? I get to go and watch cat videos
or, you know, whatever it is videos that I like and in
exchange, I'm giving somebody an insight and I know that because
I understand that's, you know, the way that a lot of practices
work because I'm getting a pop-up that says There are
cookies and I'm willingly engaging in a behavior.
So I think there's both the kind of what is happening now.
And then how do we think about data the value of data and
ownership of data for a whole new generation, little bit off
topic. But I think an important
conversation for any of us who are, who are using data and
whose data is being used in traded, in some way and we need
to move too much more of an opt-in environment, right?
We're at least, we make the decision of whether to Charity.
Obviously, this is a crucially important topic.
This issue of data privacy and data ownership.
And we're not going to solve that one now, but we can get a
better understanding of is, oh, Dave.
Aude of your concept of quantitative intuition.
What is that, and why does it matter what even Division?
I think this particular is important in the world of
marketing. Where again, there is a, the
more If you will, they the the unstructured data that Amy
talked about or the the conversations that we have, we
tend to use our intuition regularly in our private lives.
You know, we talked about surprising thing, an interesting
thing. Sometimes we even call them
gossip, but we are very worried of using our intuition, when it
comes to the world of business, to our to our work life.
And the idea here is not to just, you know, trust your gut.
The idea here is You are approaching data, bring in your
your judgment into it. In fact, we have to bring in
your judgment into it. So on on its surface
quantitative in intuition sound like an oxymoron but in I, in
fact, together with my co-authors for the book, Chris,
Frank and permanently, we believe that not only that,
they're not oxymorons, particularly the level of
leadership. It's the only way, the only way
to make decisions is to combine the data with a good sense of of
judgment. And when I say judgment, I mean,
Intuition or judgment. This could come actually, in in
at least three stages of the decision making process in the
questions we asked in being a very clear about what problem
we're trying to solve more often than not, we are just going on a
straightening. Oh, we've been collecting all of
this data, then got to be something interesting in.
Its right, we need to guide the process.
To me, the one of the biggest Nono in making data-driven
decision, making is we expect the data to provide both a
questions and There's we should ask questions then.
Hopefully data. Can provide answers.
In fact, we've learned it better than ever since November of
twenty twenty with with a 2022. Sorry with GPT.
We need to provide good proms. If you want to get good answers,
we need to ask good questions. Second is something you already
talked about how do we interrogate data and, and
exactly what I mentioned that you do need the context as
humans. We are very good in context and
particularly, if you are an expert in your domain, as Amy is
in, Amy is in in, in marketing, it's much easier for her to look
at the model of very sophisticated analysis and
interrogated. Not from the p-value of from you
didn't use this three-letter acronyms or another, but from
this number doesn't make sense. I don't know what you've done
wrong, but I can tell you that that it doesn't make sense.
And finally, you're in a crucial place for judgment is in the
synthesis of the information generally data and Analysis will
tell you the what it would not tell you the server.
We need to do it. What does it mean?
And then now what are we going to do about it?
And these are different components where we want to
combine the quantitative together with intuition.
I use the so and learn a lot all the time.
After you taught me that it's those two questions for
marketers. Truly will change the way that
you approach your, your data and your strategies.
And in fact we said that one of the reasons we went on this
journey of quantitative intuition was we were simply
tired of meetings of horrible meetings or meetings that go on
the water. And so we had a meeting and
someone shows data and now we're going to spend an hour talking
about the water. In fact, there is a Persona, we
talked about in the book, we call them the seymours, the
seymours in the organization are the ones who in every meeting of
only one comment can, I see more data?
And then we can have meetings over meetings and postpone any
decision Forever by asking for more data by by keep digesting.
And, and, and slicing, and And slicing again the data moving
from from the walk, right from what's in the data to?
What does it mean? Right?
What are the implications? And again this is a place where
you know people are even fearing a lot with chat GPT and so on
with generative AI what's left for us as human humans.
If all you do is the what, yeah, you will be replaced by a
machine but if your focus is on the so, what on What does it
mean on the now? But what are we going to do?
Not yet. I mean, I'm not saying that
machines will Not get there at some point but not at least in
the near future are so and calm comes back and he says, as AI
continues to be used more and more, do we really need
marketing people? If a I can understand the
context around that data. Yes.
Definitely. You do you do and I'm sure your
dad has a lot to say as well. I mean, where we are in terms of
chat GPT right now is I would say it is a Full assistant in
many ways, you know, having used it myself, my team uses it, it
gets you, I'm making this number up but let's say a third of the
way but it's not replacing what we're doing, it's not replacing
the knowledge that we have. And it's not doesn't always get
the context and the nuances that are very specific to humans and
then even more, you know, kind of granular or smaller there
more, Subsets of nuances amongst different audiences.
Now, you could argue all of that is data, and overtime, maybe
those things could be put into a formula and maybe it could be
understood. But I think we're still a bit
far away from saying this will resonate because or from
understanding that a word, that means something in this context
can be taken in an entirely different way in that context.
And that context is, Medic. And so yes, is it helpful for
instance, first summarizing some research that you might have.
Absolutely. Does it give you something to
gut? Check your approach?
Can it provide more information? Yeah, it can.
But we're not at the point where it could run this sophisticated
campaign for you and get equal or greater results.
And I think Michael you you really hit it on the head with
it within the day Ward context. I think this is the key.
The key is truly context. Both in terms of, by the way,
how these methods have improved. So the difference between, you
know, GPT to which was the previous version and the GPT
three and a half, which is Chad upto.
Now we already GPT for is context meaning and what I mean
by that is in order to predict the next word on door in order
to interpret a board understand the world with these tools.
Do they take the previous that many words to understand in the
context of the board and the reason why and it truly was.
I mean Church uppity Was a huge leap in Innovation.
Over the previous versions is, literally because of context
because they had they use usable is called 8000 tokens.
Think about 8,000 tokens, something like 6,000 words
because it includes periods and so on but six the previous six
thousand words to understand a particular word, the previous
versions, for example, GPT to use two thousand words to
understand any particular word. Is humans.
We are tremendous actually in doing that in in context, I'll
give you two examples. The first example is when I say
the word model and in fact, it Michael, when you use the word
model, both of us understood that in the in this context, we
are not talking about the fashion type.
We're talking about the nerdy type, right?
Because we have enough context to understand that in this
conversation, unless I'm going to give you a real clue with my
previous words that we are talking about the fashion type
model. We are talking about the nerdy
type. Type right?
And and in fact, we don't even understand how good or how our
brain actually in understand context.
I'll give you an example for that.
Think about a toddler at one and a half year old does not have
the intelligence to speak yet. And we show them illustration in
a book and the book has an illustration of a cat.
And we tell them. This thing does meow.
And then we show them illustration.
The same book of a dog kind of book about animals as we often
read. You are a one and a half year
olds and he tells them. Well this thing does wolf and
then we show them maybe 45 times illustration, different
illustrations from different books and so on the next day
we're going to take them to the street.
They cannot speak here they're going to see a cat or dog a
real. A real cat and a dog will first
time ever they see a real one and they're actually going to be
up meow and wolf without even again they can speak yet but
they do it with five observations a few years back.
The first time that machines went close to human level, in
detecting cats from dogs from images and they've done it
because the researcher made available to million
observations take by humans million of cats and dogs.
How can a human do it with four five observations and a machine
needs to million. We don't understand how we do it
with only four, five observations.
If we did, we would have trained machines to do it because it
will be very useful to have less data.
And I think that's gives you the the The understanding of why
when it comes to The Sword. And the novel words, we don't
have enough data of similar. So what an hour maybe if you are
a physician and we have enough decisions of the kind given
symptoms, maybe it works. But for typical business
decision, which is doesn't come that often.
It's unlikely. It's unlikely that at least not
with the current tools and where we are today or even in the near
future that machines would have enough context is 8000 context.
But repeatedly, Only with enough data to truly go to to that
step. Again, may happen at some point,
not yet. If you want one place, where it
did happen already and happened before generative, AI online
advertising, the allocation of Heirs of which atoms.
And seeing when I go to Google, or when I go to Amazon or
Facebook is almost fully automated apart from the choice
of the creative of The Advertiser.
Yeah. That's has been already
automated because the rules are fairly clear there isn't a Oh so
what does it mean? Well it means that this end is
likely to lead to a higher click-through rate, then another
ad, that's a predictive model machine.
Learning can do that already, how can marketers avoid being
data myopic, which is to say losing sight of the fact that
business decisions involve people circumstances.
We use that term context and not just numbers when marketing has
a seat at the table and is involved.
And in the Strategic decisions things like growth or new
audiences or understand where value is derived for a company
that's when they can put together the pieces and they're
not only seeing their small part.
So I think being part of having a marketer, truly, truly
involved at the senior level, with the seat at the table,
who's, you know, just as involved as anyone else in the
growth of the company is really important because it then
becomes top-down. You ask your team, not to look
at this slice. You ask your team to think about
how it connects to something larger.
Oh, Dad, how can organizations strike, the right balance
between relying on quantitative marketing techniques, versus
trusting, their own intuition and experience bringing the two
together. Ask yourself as you looking at
data. Ask yourself what surprised you
And it's amazing. How this fairly simple.
Deceptively simple question often cut straight to the
straight to the chase. You're either finding a mistake
or you're finding an Insight either way, you benefit.
And so, in other words, don't trust your gut.
Trust your doubts, look for these days surprises.
In the data, Amy, you're going to get the last word here.
What advice do you have for marketers given everything?
We've just been talking about your best friend, is the
personal? One who can build the models,
your you need to be embedded in business decisions to drive True
Value. And I don't think that chat GPT,
we'll take our jobs, but if you don't know how to use it, to
make yourself more efficient, or more effective as a marketer, it
will so it's not a substitute but it's an important complement
that we all have to be using and working with them on a daily
basis. Can I accurately Phrase that as
saying, be part of the business, understand the tools.
Understand the data and bring all of that together to make the
decisions that rely on both the data and your experience with
the business and with the context even better.
And with that a huge thank you to ODed netzer and Amy.
Jake from the Columbia Business. School Columbia University.
Thank you both for taking the time.
Time to be with us and share your expertise with us today.
Well thank you, it's beneficial. Yeah thank you by going to thank
you for the audience and thank you to the audience.
You guys are awesome and your questions are great.
Now before you go, please subscribe to our YouTube channel
and hit the Subscribe button at the bottom of our website.
So you can get our newsletter and we can keep you up-to-date
on upcoming live shows. Just like this one.
Thanks so much. Everybody, check out cxo
talk.com and we'll see you next time.
Have a great day.