Google and Facebook are worth a combined $2 trillion, with the vast majority of their revenue coming from advertising. In our previous episode, we learned that TV advertising is much less effective than the industry says. Is digital any better? Some say yes, some say no — and some say we’re in a full-blown digital-ad bubble.
Listen and subscribe to our podcast at Apple Podcasts, Stitcher, or elsewhere. Below is a transcript of the episode, edited for readability. For more information on the people and ideas in the episode, see the links at the bottom of this post.
* * *
In our previous episode, we learned that more than $250 billion a year is spent in the U.S. on advertising. Globally, the figure is nearly $600 billion. That’s more than half a trillion dollars. On advertising. Because of the digital revolution, television advertising has lost some of its primacy. But TV still accounts for roughly a third of ad spending in the U.S.; the Super Bowl alone brings in more than $300 million.
And how effective is all that TV advertising? I mean, how good is it at actually selling the products it’s telling you to buy? The conventional wisdom says it’s got to be effective; why else would companies spend so much money on it? But the data tell a different story. Here’s what we heard last week from Anna Tuchman. She’s a marketing professor at Northwestern University, and she recently co-authored a massive study on the efficacy of TV advertising:
Anna TUCHMAN: This means that doubling the amount of advertising would lead to about a 1 percent increase in sales.
Stephen DUBNER: So, your research argues that TV advertising is about 15 to 20 times less effective than the conventional wisdom says, yes?
TUCHMAN: That’s right.
There are, not surprisingly, objections to this research. Especially from the marketing industry. For instance, they’ll point to the brand-building aspect of advertising: “It’s not just about short-term sales,” they’ll say. Or the game-theory aspect — that is, if you don’t advertise your product and your rivals do, where does that leave you? Still, any company that spends even thousands of dollars on TV ads, much less millions or billions, would have to be sobered by Anna Tuchman’s findings.
Was TV advertising always so inefficient — or did it lose its luster recently with the arrival of digital giants like Google and Facebook? We don’t know the answer to that question. What we do know is that people are spending more time online than ever before, and that digital advertising holds the promise of matching advertisers precisely to the people who want their products.
Keith WEED: At the end of the day, everything around marketing strategy is around segmentation.
That’s the former chief marketing officer of Unilever, Keith Weed.
WEED: The more relevant you are to the audience, the more interested they’ll be in your message, and the more interested they’ll be in your message, the more likely they are to buy your product.
And thanks to the cookies on your phone or computer — such a cute name for such a powerful tracking device — you are constantly telling the companies who installed those cookies exactly what you are interested in. You also tell them where you live; what you wear and listen to and eat; what kind of people you hang out with; which God, if any, you believe in; which political party you hate less than the other.
WEED: Yes. The thinking behind advertising has changed radically with the arrival of digital and data.
The internet has made it almost too easy to sell to us. And sell to us they do. Last year, advertisers spent $123 billion on internet ads in the U.S., just less than half the total ad spending across all media. That’s how Facebook and Alphabet, the parent company of Google, have become two of the most valuable companies in the world. More than 80 percent of Google’s revenue comes from advertising; more than 98 percent of Facebook’s revenue comes from advertising. With so many advertisers spending so many billions, they must be getting a healthy return on their investment, right? So, digital advertising must be effective, right?
Steve TADELIS: Uh.
Tim HWANG: Um.
TADELIS: Uh.
Today on Freakonomics Radio: a hard look at the hard-to-find evidence around digital advertising.
HWANG: “Oh, ads definitely work. But we can’t tell you how or why or give you any evidence for it.”
* * *
Steve Tadelis is a professor at the Haas School of Business at the University of California, Berkeley.
TADELIS: Yes. And I teach and do research in economics.
A lot of his research is on the economics of the internet. Back in 2011, he started doing some work with eBay.
TADELIS: I was asked to hire and lead a team of economists and work with a variety of the businesses within eBay.
The hiring of prominent academic economists is a long-standing practice at tech firms. Hal Varian, also from Berkeley, has worked for years as chief economist at Google; John List, from the University of Chicago, has done similar work, first at Uber and now at Lyft. So, here was Steve Tadelis at eBay:
TADELIS: And at one point, one of the directors there realized that as economists, we should know something about econometrics, which is the statistics of measuring economic activity. And they wanted to see if we could vet a consultant that they recently hired, a consulting firm, to do quantitative marketing analytics and help eBay figure out how to spend their scarce marketing dollars.
Let me just say that “scarce” here is a relative term.
TADELIS: At the time, eBay was spending about a billion dollars a year in a variety of marketing activities.
Okay, so it makes sense that eBay would want to know how effectively that money was being spent. More than half of it was going toward internet ads. And this outside consulting firm was going to analyze the efficacy of those ads.
TADELIS: And the goal was to speak with this consulting company and see if they were going to use solid and vetted ways of measuring these returns to advertising.
To be fair, measuring the efficacy of advertising can be really hard. Last week, my Freakonomics friend and co-author Steve Levitt told us about trying to help a big-box retailer measure the efficacy of their TV ads. One issue is that they really only advertised three times a year: for Father’s Day, Black Friday, and Christmas.
LEVITT: And so, you, of course, have a correlation between when you’re advertising on TV and when you’re selling things. But it’s not necessarily or even primarily because of the ads. It’s because the company knows when the big selling days are, and they target the ads around it. So, teasing out the causal part, the sales that wouldn’t have happened absent the advertising, it’s just a really hard problem.
But within that firm, Levitt found the executives were convinced it was the ads that caused the sales. Steve Tadelis again:
TADELIS: This is the kind of wrong analysis that many people end up falling into when they are not that conscientious about the difference between causation and correlation.
Tadelis knew this would be a hard problem for eBay as well. And that’s what these outside consultants were supposed to figure out:
TADELIS: So, I got on a call with them and very quickly was able to confirm that what they were doing was quite wrong.
Uh-oh! Just so you know, consulting firms also like to hire academic economists to do this kind of work. So, you can maybe see where this is going. Tadelis is listening intently on the call.
TADELIS: Which was on a landline, which will make sense in a second.
And when Tadelis suggested that the consultants’ proposed methodology wouldn’t be able to untangle correlation and causation:
TADELIS: They responded using a whole bunch of jargon, especially the term “proprietary transformation functions.”
“Proprietary” meaning “that’s our secret.” “Transformation functions” meaning — well, who knows? Tadelis said he still didn’t understand how these consultants were going to measure the thing that needed to be measured: the marginal effect of the ads that eBay was spending millions on.
TADELIS: Then the head of the company replied by saying that to do the marginal measurement, they’re going to use Lagrange multipliers. Well, I paused for a second because I know what Lagrange multipliers are. I used to teach this stuff, and I couldn’t understand what they were trying to do here. And that’s when the dime dropped. They were trying to out-jargon me.
So, I replied by saying, “Well, we all know that the Lagrange multipliers measure the shadow values of constraints in an optimization problem. So, it would really help me if you explain to me, what is your objective function and what are your constraints?” After a short pause, and this is where I have to take my hat off to the founder of that consulting company, he immediately responded with the only and best answer he could give, which was, “Steve, are you driving now? Because I can’t hear you. You’re breaking up.”
Tadelis took his concerns back to his bosses at eBay. He proposed a different way to understand the impact of the online ads eBay was buying: he offered to run some randomized experiments. To a researcher, that’s the gold standard.
TADELIS: And there was not any buy-in.
But coincidentally:
TADELIS: But coincidentally, one of the advertising-and-marketing teams wanted to renegotiate a deal with Yahoo and Microsoft network, namely the Bing search engine.
That is, eBay wanted to renegotiate the terms under which they bought search ads on Bing:
TADELIS: And before doing that, they ceased all payments for brand-keyword advertising.
That’s because eBay was trying to create some leverage heading into their renegotiation. Okay, but now we need a primer on “keyword advertising” and how you actually buy online ads.
TADELIS: Right. So, just like you could imagine the auction for a piece of art at Sotheby’s, where the auctioneer puts up this Picasso and says, “How much are people willing to pay?” and then give it to the highest bidder — something similar is happening every time someone searches for anything on Google or Yahoo! or Bing or any other search engine. And you have companies that are bidding on different kinds of keywords.
Companies that want to advertise online come up with lists of words that they think will give them the best chance of connecting with people who are searching for what the companies are selling. These could be branded keywords, like “eBay” or “Verizon,” or non-branded keywords, like “stiletto heels” or “cable TV.”
TADELIS: If I’m in the insurance industry and someone is searching “car insurance” on Google, I want to be there. But I don’t want to be there when someone is searching for “falafel.”
Unless, maybe, you’ve got a specialty line of falafel insurance.
TADELIS: And once someone types “car insurance,” the second they click “enter,” all these computers are basically running these auctions instantaneously.
This is done with the aid of sophisticated algorithms.
TADELIS: So, they have all the bids. They allocate it to the winner. And that’s how the ad pops up.
When Tadelis was working for eBay, the company was in the practice of buying brand-keyword ads. Which meant that if you did an online search for “eBay,” the top result — before all the organic-search results — was a paid ad for eBay. The Federal Trade Commission requires such ads to be clearly labeled as ads — although some of the labeling is pretty subtle. Go do a Google search for something like “best running shoes,” and you’ll see what I mean. Anyway, these paid eBay ads that Tadelis is talking about:
TADELIS: Now this is what the consulting company said had the highest bang for the buck — which, of course, made no sense from a common-sense perspective, because if I’m already searching for eBay, I know exactly where I want to go. So, the fact that the ad intercepts me, and I click on it, is just coincidental and it cannibalizes what would have been a free click on the organic search.
So, when eBay, in planning to renegotiate their deal with Bing, turned off their brand-keyword advertising:
TADELIS: That gave us what’s called a natural experiment.
This is what economists dream about — a change in a variable that affords a real opportunity to separate correlation from causation.
TADELIS: We could measure visits and we could measure purchases and we could see whether there was any drop in clicks and purchases. And — not surprisingly — all the search that was taken away from the ads just ended up coming for free through the organic search. Because right below the ad was the free link to eBay. Once we had those results, I went to the chief financial officer of eBay North America and showed him the analysis, to which he responded, “Okay, you guys were right. What do we do next?” And that gave us the open door to design more sophisticated experiments.
Tadelis wound up running two experiments, along with fellow economists Thomas Blake and Chris Nosko. The first one essentially mirrored the natural experiment: they turned off all brand-keyword search ads.
TADELIS: Which means that if someone searches for “eBay,” eBay will not surface an ad. The conjecture that we had — which I can’t see any other conjecture — is that if there is no competition for keyword “eBay,” then once you remove that ad, the organic search is going to be the first thing that people see, and that means that they’re going to go to eBay directly without having eBay pay any fees to the search engine for the advertising.
And that is precisely what they found.
TADELIS: Surprise, surprise.
Other researchers have found even more pronounced results. Imagine that an eBay competitor — like Amazon — enters an ad auction for the keyword “eBay” and wins that auction. This means that when you search for “eBay,” the top link you’ll get is an Amazon link. In such cases, researchers found, most people just ignore the Amazon link and move on directly to eBay. Which means brand-keyword advertising — whether it’s your brand or someone else’s — is a waste of money.
TADELIS: Exactly. Imagine you’re a restaurant owner and you want to hand out coupons in order to get people to come to your restaurant. And if you want to measure the returns on that coupon, you really need to know how many people would come without the coupon. Because here’s the thing. For every person who would have come anyway, you’re losing money. The analogy, in my view of brand keyword advertising is handing out the coupons inside the restaurant.
The second experiment Tadelis and his colleagues ran used non-branded keywords — just the names of things that people might be looking to buy online, like “guitar” or “boots” or “picture frame.”
TADELIS: Yes. So, for non-branded search, we actually had no idea what the results are going to be. Because here, if I am searching for, example, a studio microphone I’m sure that on eBay I might find a variety of used ones. But if I’m not thinking about eBay, and I just search for “studio microphone,” if eBay doesn’t pay an ad, they might not even show up on the first page. And by the way, the automated machines at eBay that were doing the online bidding, they had a basic library of close to 100 million different combinations of keywords, because eBay has practically everything you could imagine for sale on the site. So, we really had no idea what the returns for the non-branded searches would be.
The experiment they designed used a “border strategy,” the same method we heard about in our earlier episode on TV advertising. This takes advantage of what are called D.M.A.’s, or designated market areas — what most of us think of as “media markets.” It turns out that most people living on one side of a market border are socioeconomically and demographically similar to the people living on the other side. When two similar populations are served different ads — which happens all the time — you can isolate the impact of the advertising on their actual purchases. There are 210 D.M.A.’s in the U.S.
TADELIS: And we took a third of these D.M.A.’s, and we turned off all paid-search advertising. This was an extremely blunt experiment where we’re saying, “What would happen if we didn’t advertise at all?” And to our surprise the impact on average was pretty much zero.
Did you catch that? They turned off all their keyword-search ads, then measured actual sales:
TADELIS: And the impact on average was pretty much zero.
What was eBay’s existing belief about paid-search advertising?
TADELIS: The company believed that roughly 5 percent of sales were driven by paid-search advertising, meaning that they believed that if we would pull the plug on advertising, sales would drop by 5 percent. What we found was that sales dropped by about half a percent. So, that’s an order of magnitude less. And it was not statistically different from zero.
But maybe it’s still worth it to gain even that half a percent? Now we have to know what the advertising costs, and measure the return on investment.
TADELIS: When you did the return on investment for every dollar that eBay spends — eBay believed that for every dollar they’re spending, they’re getting roughly a dollar-and-a-half back, meaning 50 cents of net profits. And what we showed is that on average, they’re losing more than 60 cents on every dollar.
So, how did these results go over?
TADELIS: Well, the president of eBay, who later became the C.E.O., he cut the paid-search marketing budget immediately by $100 million a year.
So, what happened next? You might think — what with capitalism being the hyper-competitive, market-optimizing, perfect-information ecosystem it’s supposed to be — you might think that other companies, once they learned about this eBay research, would cut their online ad spending. Or at least commission their own research to test the theories. So, did they?
TADELIS: Excellent question. There was a lot of chatter online after our experiments became public, suggesting that folks at eBay don’t know what they’re doing. And paid-search advertising works wonderfully if you know how to do it. But of course, that was backed with no data and no analysis.
In other words, the digital-ad community did not rush to replicate the results. Now, given the opportunity to save millions of dollars that the eBay research showed was being wasted, why wouldn’t other companies at least poke their own data a little harder?
TADELIS: Well, I think there are many reasons. Let’s start with the way in which this industry is structured. You could think of four different actors here. There’s the customer, which is the company or the person who wants to advertise in order to get business. And then you have three players sitting on the other side of this market. One is the publishers. That would be Google. That would be The New York Times, or any other place where the ad appears in front of people. The other are the people who create ads. And then finally a smaller part of the industry are these analytics companies that, like that company eBay hired, are trying to help companies spend this money. And if you think of all these three players on the other side of the fence, no one there has an incentive to basically open this Pandora’s box.
Even within the company that’s buying the ads, the incentives can be complicated. Steve Levitt again:
LEVITT: If you think about it, no chief marketing officer is ever going to say, “Hey, I don’t know, maybe ads don’t work. Let’s just not do them and see what happens.” So, don’t get me wrong. I’m not implying that advertising doesn’t work. I’m implying that we don’t have a very good idea about how well it works.
Steve Tadelis agrees. The potential for digital advertising would seem especially large, given its ability to micro-target consumers.
TADELIS: And targeting really is key because one of the lessons we learned from the experiments at eBay was that people who never shopped on eBay, they were very much influenced by having eBay ads for non-brand keywords. You know, “guitar, “chair,” “studio microphone.” And if eBay would be able to better target ads to customers that are not frequent customers, that’s where you would get the real bang for the buck. So, as companies become more sophisticated, they could try to engage in these kind of experiments to focus attention on different customer segments in order to see where they get the highest returns on advertising. By and large, I don’t see that happening. A big part of it is the naivete on the side of these customers.
“Customers” meaning the companies who are buying these keyword ads — the one actor sitting alone on their side of the fence, across from the agencies, the publishers, the ad-tech firms.
TADELIS: And one of the things that I try hard to do is to give people enough information so that they wouldn’t be able to do the job themselves, but if someone is trying to sell them snake oil, they’ll smell something is not working here.
* * *
We reached out to Facebook and Google with some questions about their ad business, and to get a response to the research we’ve been discussing today, which argues that paid-search advertising is substantially less effective than the conventional wisdom holds. We got no reply from Facebook. The Google representatives wrote back to say, “Advertisers invest money in search ads because they work.” They also sent some internal Google research to back up their claim. Separately, we received an unsolicited e-mail from Hal Varian, the chief economist at Google. He attached a long list of research papers that assert the efficacy of search advertising as well as advertising on YouTube, which is owned by Google. Much of the research Varian sent was done by Google analysts; it offers a robust defense of the status quo.
The online ad ecosystem Google has built off of their search capability is quite literally a license to print money. Alphabet, Google’s parent company, has a market capitalization of nearly $1.2 trillion; last year, 83 percent of their revenue came from advertising. So, it would probably behoove all of us to know a little bit more about how this ecosystem functions, whether it’s as effective as Google says it is or as ineffective as researchers like Steve Tadelis told us earlier, about his research at eBay.
TADELIS: eBay believed that for every dollar they’re spending, they’re getting 50 cents of net profits. And what we showed is that on average, they’re losing more than 60 cents on every dollar.
So, okay, let’s try to understand this ecosystem better. First, we’ll need a guide:
HWANG: Sure. My name is Tim Hwang. H-W-A-N-G. My day job is I’m a research fellow at the Center for Security and Emerging Technology at Georgetown.
And, before that:
HWANG: I was previously global head of public policy for A.I. and machine learning at Google.
Hwang recently published a book called Subprime Attention Crisis: Advertising and the Time Bomb at the Heart of the Internet. It’s about how big tech monetizes our attention.
HWANG: When I started to do research, I very naturally started to talk to a couple friends who work at these big tech companies. And it was a little bit like talking to someone who works in national security or the intelligence community or something like that. Because they would be like, “Oh, ads definitely work. But we can’t tell you how or why or give you any evidence for it.”
Google would plainly dispute that there is no evidence for whether online ads work. Tim Hwang recognizes he is tilting at windmills here, trillion-dollar windmills. But in fact he first grew skeptical about online advertising while still working at Google. He began reading trade journals and going to conferences.
HWANG: And I had this fascinating experience where one of these keynotes at this conference was given by Nico Neumann, who basically is a big ad critic.
Nico Neumann is a marketing professor at the Melbourne Business School in Australia.
HWANG: And he presented two really fascinating studies that his lab had done. The first one was looking into the quality of data used in the ad-tech industry — basically demonstrating in many cases it was incredibly inaccurate. And the second one was, he took dead aim at the hype cycle around A.I. that exists in ad tech right now, where people are saying, “If you have this latest machine learning, you have this A.I., you’ll be able to do targeting in a way that you never, ever were able to do before.” And Nico’s lab did some experiments that demonstrated that, in many cases, machine learning was finding people who would have bought the product anyways.
Bringing this message to an ad-tech conference is a bit like bringing a safety pin to a balloon conference.
HWANG: I looked around being like, “Wait, so, would people be angry?” And it was just total dead air. No one responded. No one engaged with it. And it got me really interested in thinking about, is there a bubble here?
A bubble like the dot-com bubble or the subprime bubble. Or the tulip bubble.
HWANG: Because this is exactly the kinds of behavior that occur in other financial bubbles, where the red lights are flashing but everybody in the industry just refuses to take a look at the real data.
Hwang began thinking about how bubbles happen.
HWANG: So, the origins of every bubble come in this gap that occurs in a marketplace. On one hand, you have people who believe that an asset, whether it’s collateralized debt obligations or advertising inventory, is extremely valuable. And then, on the other hand, what you have is declining asset value. So, in the subprime-mortgage crisis, we believed that mortgages were always going to just pay out regularly forever, when it actually turned out that the package of mortgages were actually a terrible asset. They were toxic and about to go belly-up.
So, how can Hwang justify a parallel with digital advertising?
HWANG: I think the first piece is really the big question of do people ever see ads at all. So, Google actually did a fascinating study not too long ago, which concluded that close to 60 percent of ads on the internet are never, ever even seen. The ad is delivered, but it just ends up in some dumb part of the page, right? It’s below the fold or it’s along a sideline.
But what about the precise targeting that digital ads are supposed to offer? A 2019 study, this one done by three academic researchers, addressed this question by measuring the impact of a user’s cookies. Those, remember, are the tracking codes that most of us allow to roam our computers and phones in exchange for all the free information we get from companies like Google and Facebook. This study found that when a user’s cookies were unavailable, ad revenues only dropped by around 4 percent. Why would cookies be so ineffective? Tim Hwang argues that people pay a lot less attention to online ads than they used to.
HWANG: People often forget that when banner ads were first launched on the internet, their click-through rate was like 50 percent, completely mind-bending, right? And it’s just continued to fall and fall and fall. And now, it’s like 0.01 to 0.03 percent.
Some estimates of click-through rates are higher than what Hwang cites here; that said, precise measurement is hard because there are so many bots clicking on ads — a whole other problem with the digital-ad universe. But no matter how you measure it, click-through rates have fallen a lot. As the novelty wears off, habituation sets in, and an ad that might have once grabbed your attention becomes invisible — or, worse, annoying.
HWANG: People increasingly don’t want ads. So, ad blocking, for example, is really, really increasing over time. And I think these factors — not being able to see ads, the questions about the effectiveness of ads, and the rise of things like ad blocking, bring into question whether this thing that we think is so valuable is actually worth as much as we think it is.
But if there’s such a big gap between the perceived and real value of digital advertising, why are Google and Facebook worth so much money? Look at it this way: there are a couple trillion reasons why Tim Hwang might be wrong. But he doesn’t think so. His theory is that digital advertising is grotesquely overvalued because it’s still so hard to measure, and one reason it’s hard to measure is that the marketplace is exceedingly opaque.
HWANG: So, there’s a fascinating incident that I always think about, which is one of the last times that Mark Zuckerberg was called up to Congress. And one of the questions that he got from one of the senators was — “Well, how do you guys make money?”
Orrin HATCH: How do you sustain a business model in which users don’t pay for your service?
HWANG: And Mark Zuckerberg was like —.
Mark ZUCKERBERG: Senator, we run ads.
HWANG: And at the time, a lot of the chatter on Twitter was like, “Ha ha, look at this super-old senator. He doesn’t know anything about the internet.” But it’s true that even if you talk to people in the tech industry and you’re like, “Okay, level with me, Joe Engineer, how do ads work on the internet?” It’s kind of a rumor. Like, we know this is how the business model works. But no one can really explain how it works in detail. So, when I say advertising, a lot of people normally think of like “Mad Men,” right? But it really looks like what the Nasdaq looks like, which is a largely automated system that moves millions and millions and billions of pieces of ad inventory on a daily basis.
As Steve Tadelis explained earlier, most ad inventory is sold by auctions, which are run by algorithms operating at phenomenal speed. This is one contributor to the opacity of the industry. For instance, it can be hard to figure out why certain ads end up on certain pages. If you are a family-friendly brand like Disney, you don’t want your ad popping up on a YouTube video showing a terrorist beheading.
HWANG: This has been a source of chronic embarrassment to the ad industry.
The prevention of this kind of ad mismatch is known as “brand safety.”
HWANG: And despite the greatest efforts at trying to eliminate the risk of brand safety from the ad market, people by and large just haven’t been able to.
Again, the industry itself would disagree. We asked Google how they ensure an ad doesn’t show up on a page promoting misinformation or a conspiracy theory. Here’s their reply: “We have strict policies that govern what kind of content we place ads on, and if we find a page or website that violates our policies, we take immediate action.” In 2019, Keith Weed — the former marketing boss at Unilever — helped create a consortium called the Global Alliance for Responsible Media, which pushes for better ad controls to protect brand safety.
WEED: At the end of the day, everyone wants a well-functioning internet, and everyone wants it to have a positive impact on the world and not to have some of the issues we’re wrestling with right now. I think the path has not been easy so far.
This September, after months of advertiser boycotts, Facebook, Twitter, and YouTube agreed to adopt a common set of definitions for hate speech and develop tools to let advertisers have more control over where their ads show up. But Tim Hwang thinks the long-standing opacity of the online ad marketplace is just one reason we might be in a digital-advertising bubble.
HWANG: I think a second thing is, a little bit like in the subprime-mortgage crisis, you do have people who have very perverse incentives, I think, to push the effectiveness of online ads. That’s the ad agencies, the ad platforms themselves, the people who run ad technology. All these people, I think, have a very strong incentive to say, “No, this stuff is way better than earlier generations of advertising, and this is why you should use it.”
If you’ve been listening closely, you’ll notice this is the exact same problem Steve Levitt talked about regarding the TV-ad ecosystem: human beings generally make decisions based on self-interest.
LEVITT: No chief marketing officer is ever going to say, “Hey, I don’t know, maybe ads don’t work. Let’s just not do them and see what happens.”
Or, as the author Upton Sinclair once wrote: “It is difficult to get a man to understand something when his salary depends on his not understanding it.”
HWANG: So, there is a common practice, which is not very well-disclosed in the ad industry, whereby an ad-tech company will basically offer ad inventory at a cheap price to the advertising agency.
The agency, remember, is paid by you, the client, who hired them to help you sell what you’re selling.
HWANG: And then the agency will turn around and say, “You should really use this ad-tech product,” and sell it at a higher price. And one of the worries about this is that it changes the incentives, right, which is typically the ad agency should be working on behalf of the client. But in these cases, they have very perverse incentives to push a distribution of a message that may not otherwise be rational or even useful to them.
All of these issues, and all the new empirical evidence we’ve been discussing about the ineffectiveness of advertising, has persuaded Tim Hwang that yes, the online ad marketplace is a bubble and it might soon pop. In fact, the deflation may have already begun.
HWANG: A few years back, Procter & Gamble, which is one of the largest advertisers in the world, decided that they would run a little experiment. They were going to take about $200 million of their digital-ad spending and just cut it out of their budget to see what happened.
Procter & Gamble said they were doing this because of concerns over brand safety and the proliferation of bots, which can pollute the data on ad impressions.
HWANG: And the end result was fascinating. Basically, they said that there was no noticeable impact on their bottom line.
Again, the ad industry will have a lot of explanations for why this might be. Or for why there’s a lot of value in advertising beyond short-term sales figures. But Procter & Gamble is a big player. Even if they’re wrong, even to a small degree — they’re the ones whose money drives the advertising ecosystem. What would happen if this turned into a mass movement among advertisers? One shouldn’t underestimate the size and reach of the advertising ecosystem. The sports you watch on TV: supported by ads. The journalism you consume: supported by ads, at least much of it. Google Maps and Google Drive and — well, Google: supported by ads. As well as Facebook and Instagram and Twitter and nearly everything else you consume online and don’t pay for. Including this podcast and just about every other podcast you listen to. Advertising is also important for Tim Hwang, whose day job is researching artificial intelligence and machine learning,
HWANG: Some of the most cutting-edge research in the world is being funded by ads, right? If you think machine learning and A.I. are going to have a huge impact—.
A huge impact in the world, that is — from autonomous travel to medical research and diagnosis:
HWANG: You may want to think about what it means that most of it is subsidized through this ad infrastructure. So, there’s lots and lots of links through the economy that are not always obvious, but I think are worth thinking through because it points out how widespread a downturn could be.
So, if there is an advertising bubble, can it be deflated in a slow, controlled way to avoid a massive economic unraveling? Tim Hwang thinks so, but change won’t come from the industry players. They have too much incentive to keep selling. It’s got to be driven by the buyers.
HWANG: One of the groups that has the most to lose are people who theoretically might be wasting a lot of money on advertising.
But Hwang doesn’t think buy-side pressure will be enough.
HWANG: Because I think one of the biggest problems in the space is that there’s no objective, third-party evaluator of some of these claims.
Claims, that is, about ad efficacy.
HWANG: And so, what I advise is kind of a punk-rock N.B.E.R.
The N.B.E.R. is the National Bureau of Economic Research. What does Hwang’s punk-rock version look like?
HWANG: It’s basically a research group that is willing to be a little bit of a troll to the advertising industry. And so, again, how do we throw off the veil, reduce the opacity in the marketplace? You really need a dedicated group of people who are doing good research on this front. What you want is a handbrake, where you can slowly bring down the momentum in the market so that it can deflate without exploding.
There are, of course, many people and institutions already doing research on advertising spending. But let’s be honest — most of them have someone’s thumb on the scale. And conventional wisdom isn’t the only thing that someone like Tim Hwang is challenging; it’s also the billions of advertising dollars that drive trillions of dollars of market value. So, it takes some courage to suggest that those billions and trillions may not be kosher.
HWANG: One of my favorite arguments that people are using right now is, “Companies wouldn’t put money into this if it didn’t work. So, isn’t that proof that the ads actually work?”
That, in fact, is exactly what we heard earlier from the Unilever veteran Keith Weed.
WEED: The fact that Coke and Dove and Ford have been around for decades and the fact that companies like Unilever spend billions suggests that maybe advertising does work.
HWANG: Which is kind of this crazy circular mind maze, if you think about it. But I do think that, again, this is very parallel to the kinds of psychology that have driven market bubbles in the past.
One reason to suspect that ads do work well is the underlying assumption that firms like Unilever who buy so much advertising are — as Econ 101 textbooks tell us — profit-maximizers. So, why would they waste so much money?
LEVITT: Any economist who tells you that firms are profit-maximizing has not ever worked with firms.
That, again, is Steve Levitt.
LEVITT: The realistic picture is that firms are composed of people and all of the foibles and shortcomings that people exhibit in their everyday life, they bring those to work with them.
We asked Steve Tadelis, the Berkeley economist who worked for a time at eBay, what he thought of Levitt’s take on the non-profit-maximizing behavior of allegedly profit-maximizing firms.
TADELIS: As an economist, hearing you say that causes my stomach to hurt. But at the same time, I know that you are absolutely correct.
Tadelis, by the end of his time at eBay, had come to think that his cynical view of advertising didn’t go far enough. He recalls the time eBay asked him to measure the efficacy of affiliate advertising.
TADELIS: Think of bloggers who put in links to company websites. Well, we worked closely with the senior director in charge of spending the money on that. And after a two-hour intensive meeting, we figured out a way to do that.
“To do that” meaning to measure whether these affiliate links were really worth buying.
TADELIS: He turned to me and said, “You know, Steve, if your results look as bad as they did for paid search, I’m not going to believe your numbers.” Now, I was obviously shocked because it made me realize that religion, and not science, is what’s winning this battle. But then I realized that it’s something a lot more profound, and for which I actually have a lot of compassion.
If you’re working on something for 10, 20, 25 years, this is part of your identity and this is part of what you believe in. And if I’m going to prove that what you thought worked so well, in ways that you don’t quite understand because you’re not a statistician or an econometrician, and you have to take it at face value, what are you going to believe? Your gut, that tells you that what you’ve been doing for the last 20 years is really influential? Or some egghead academic that’s showing you a bunch of equations that you don’t understand and is claiming that you’re wrong?
This digital-advertising issue is just part of a bigger conversation about the power of modern technology companies. For their first few decades, they were pretty much given free rein. But now they’re facing scrutiny over the breadth and depth of their power — power both seen and unseen. The U.S. government has brought a major antitrust case against Google. Facebook, YouTube, Twitter — and, frankly, thousands of digital platforms and repositories — stand accused of promoting misinformation and/or mishandling user information. Given all that, society probably deserves a better answer than “a lot of companies pay a lot of money for advertising, so it must work.”
HWANG: I think the question isn’t necessarily do you want an internet with ads or without ads?
That, again, is Tim Hwang.
HWANG: The question is, do you want an internet that’s just based on a huge monoculture that’s largely funded through ads, where the most powerful companies use ads, and where V.C’.s don’t choose to invest if you don’t use ads, right? And I humbly suggest no. The kind of internet that I want to see is an internet that has a bigger diversity of business models and where ads don’t suck all the air out of alternative business models. I just think that’s a much more robust market. I think it’s a more stable market over time. And I think it’s one that leads to much better outcomes socially. And so, I think that’s my vote, which is an internet that accepts many different ways of making money.
We should also say that, just as there are different ways of making money, there are many, many, many different forms and styles and purposes of advertising. Over these past two episodes, we’ve talked about two of the biggest ad marketplaces: TV and digital. But there are of course many other advertising channels and within all these channels are countless variations. Local ads versus regional or national ads. Calls to action versus brand-building. Ads about price versus ads about quality. Ads for new products or services versus established ones. Ads meant to inspire or entertain versus ads that just deliver information. Personally, those are the ads I respond to most — learning some information I didn’t know, about some product or service that sounds useful or fun for me or my family.
I can think of at least eight or ten things I bought after first learning about them by voicing the advertisements that go on this podcast! Our business partners have told us that one reason the podcast industry has grown so much lately is because advertising is particularly effective on podcasts, given the intimacy of the medium. I have no idea if this is really true. But I want to believe it is. Why wouldn’t I? It’s my livelihood. So, you could say this puts me on both sides of the fence: true believer and skeptic, at the same time. One might say this is a sign of intellectual laziness, a move right out of Upton Sinclair. But I’m going to go with F. Scott Fitzgerald instead. Here’s how he once put it: “The test of a first-rate intelligence is the ability to hold two opposed ideas in mind at the same time and still retain the ability to function.”
* * *
Freakonomics Radio is produced by Stitcher and Dubner Productions. This episode was produced by Daphne Chen. Our staff also includes Alison Craiglow, Greg Rippin, Mary Diduch, Mark McClusky, Zack Lapinski, and Matt Hickey. Our intern is Emma Tyrrell, we had help this week from James Foster. Our theme song is “Mr. Fortune,” by the Hitchhikers; the rest of the music was composed by Luis Guerra. You can subscribe to Freakonomics Radio on Apple Podcasts, Stitcher, or wherever you get your podcasts.
Here’s where you can learn more about the people and ideas in this episode:
SOURCES
Steve Tadelis, professor at the Haas School of Business at the University of California, Berkeley.
Steven Levitt, co-author of Freakonomics, economist at the University of Chicago, and host of People I (Mostly) Admire.
Anna Tuchman, associate professor of marketing at Northwestern University’s Kellogg School of Management
Keith Weed, president of the Advertising Association and former chief marketing officer at Unilever.
Tim Hwang, research fellow at the Center for Security and Emerging Technology at Georgetown University.
RESOURCES
“Generalizable and Robust TV Advertising Effects,” by Bradley Shapiro, Günter J. Hitsch, and Anna Tuchman (SSRN, 2020).
“Online Tracking and Publishers’ Revenues: An Empirical Analysis,” by Veronica Marotta, Vibhanshu Abhishek, and Alessandro Acquisti (2019).
“Frontiers: How Effective Is Third-Party Consumer Profiling? Evidence from Field Studies,” by Nico Neumann, Catherine E. Tucker, and Timothy Whitfield (Marketing Science, 2019).
“The Effects of Search Advertising on Competitors: An Experiment Before a Merger,” by Joseph M. Golden and John J. Horton (Management Science, 2018).
“Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment,” by Tom Blake, Chris Nosko, and Steven Tadelis (National Bureau of Economic Research, 2014).
“FTC staff to search engines: Differentiate ads from natural results,” by Lesley Fair (Federal Trade Commission, 2013).
“Incremental Clicks Impact Of Search Advertising,” by David X. Chan, Yuan Yuan, Jim Koehler, and Deepak Kumar (Google Inc., 2011).
“How Wrong Audience Targeting And AI-driven Campaigns Undermine Brand Growth,” by Nico Neumann (Programmatic I/O, 2019).
“Five Factors of Viewability,” by Think wih Google.
EXTRA
Subprime Attention Crisis: Advertising and the Time Bomb at the Heart of the Internet, by Tim Hwang.
“Does Advertising Actually Work? (Part 1: TV) (Ep. 440),” by Freakonomics Radio (2020).
The post Does Advertising Actually Work? (Part 2: Digital) (Ep. 441) appeared first on Freakonomics.
Read more: freakonomics.com