iPhone 14 Pro vs. Galaxy S22 Ultra HP Pavilion Plus Planet Crossword Pixel Watch Apple Watch Ultra AirPods Pro 2 iPhone 14 Pro Camera Best Android Phones
Want CNET to notify you of price drops and the latest stories?
No, thank you

Pop-ups annoy, but do they work?

Wharton experts try to figure out just how annoying pop-up ads really are and whether intrusive necessarily means ineffective.

Pop-ups annoy, but do they work?
From Knowledge@Wharton
Special to CNET News.com
August 23, 2003, 6:00 AM PT

They create as much clutter as those slippery advertising inserts that fatten a Sunday newspaper and are as inescapable as humidity in August.

But just how annoying are those pop-up ads that appear unwanted on your computer screen as you cruise the Internet? How effective are they at selling stuff? And do they raise privacy issues in the same way that unsolicited e-mail does?

E-commerce experts at Wharton and elsewhere assert that pop-ups are not universally loathed and irrevocably worthless. But collectively, they can indeed be a nuisance. Pop-ups are a lot like other forms of advertising: If they are presented to a consumer at the wrong time and in the wrong way, they can be a big turn-off. But if a consumer sees them at the right time, they can provide useful information or at least be entertaining and nonoffensive.

Wharton marketing professor Patricia Williams, who teaches a course on electronic commerce, says there is not enough research to measure the effectiveness of pop-ups. She adds that some research she has seen asserts that consumers think pop-ups are actually worse than telemarketing calls. This would be something of an achievement, since telemarketing calls have attained the distinction of being perhaps the most unwanted and intrusive sales attempts in marketing history.

But Williams says she does not necessarily believe those studies. "My intuition tells me that ads that have more a traditional advertising format are viewed differently than telemarketing or spam," she says. "Consumers feel that when they watch television, ads will appear. It's part of the consumer process. My intuition is that pop-up ads are more like television ads that we're used to."

David Croson disagrees.

"Pop-up ads are probably the only form of advertising that has spawned a whole industry designed to help you get them off your screen," says Croson, a visiting professor of digital strategy at the Massachusetts Institute of Technology and a researcher at MIT's Center for eBusiness. "I think pop-up ads have created a really strong negative response."

An experiment in effectiveness
Wendy Moe, who earned a Ph.D. at Wharton and now teaches at the McCombs School of Business at the University of Texas at Austin, has conducted a field experiment that tests the effectiveness of pop-up ads in certain circumstances. The results are outlined in a paper titled, "Should We Wait to Promote?: The Effect of Timing on Response to Pop-Up Promotions."

Moe found that pop-up ads do not necessarily annoy Web users; the level of annoyance depends on the situation. For the study, Moe offered pop-up ads to people of a fairly well-trafficked content site. For some, the ads were made to pop up very early in the session. For others, there was a delay before the pop-up was offered.

"The effects varied depending on what the user's purpose at the site was," Moe says. "For users who sought out fairly in-depth information from the site, the added pop-up basically overloaded them with information. And as a result, they exited the Web site earlier than they probably would have otherwise. For those who were browsing at the site and were not seriously seeking out lots of information, the pop-up was a welcomed interruption to their browsing activities. Some pop-ups lengthened these users' stay at the site."

During the first seven months of 2002, advertisers launched some 11.3 billion pop-up ad impressions on the Web.
Pop-ups, which appear in the foreground of a computer screen and block parts of the main page that a person wants to view, are just one of a number of promotional formats found on the Web.

Other types mentioned in Moe's paper include "banners," which are similar to standard print ads; "pop-unders," which appear in windows that open in the background of the screen and only become visible when the person closes the main window; "bridge pages," to which a user is redirected when navigating from one page to another; and "in-page animations," which use pictures and sound like TV ads and, like a pop-up, block the page a person is trying to view.

Dan Hunter, a professor of legal studies at Wharton who conducts research on e-commerce, agrees that the degree of annoyance at pop-ups will differ depending on the temperament of the user and the circumstances. "Apart from the degree of tolerance of any individual user, I think that there are two aspects that affect degree of annoyance: speed of connection and whether the pop is up or under," he says.

"As to the first, if you're on a slow dial-up, the pop-ups are a major hassle, because they use valuable bandwidth that slows the download of valuable stuff, especially since the pop-ups tend to be flash-enabled or use fairly large graphics files. As for the second aspect, pop-ups are more intrusive than pop-unders," Hunter adds.

"Though at one point it seemed that pop-unders were going to be used more than pop-ups (because of the perceived difference in intrusion), I've noticed that this has now reversed. I suspect because the advertisers were worried that you just shut the browser down without ever seeing the pop-under. Internet advertising has been such a roller-coaster ride that any small disincentive is enough to tip the balance, and so I've seen more pop-ups recently."

In the final analysis, all advertising is an imprecise science. No one knows with any degree of consistency how well any ads increase the sales of products and services. Sometimes they help; sometimes they don't. The better ones can create a nice positive buzz; the poorer ones can be embarrassing.

"The pop-ups that are more effective are the ones related to the site you're looking at," Wharton marketing professor David Reibstein says. "Some advertisers use pop-ups everywhere, but they may be totally irrelevant to you at the time. If I want to buy a new car and there's a pop-up of a Volvo, it's much more likely to be effective."

Marketers must find something useful in pop-ups because they continue to use them, as they do telemarketing calls. "The reason we end up getting such a large number of phone calls at home is because it's economically worthwhile to do," Reibstein says. "They wouldn't be calling us at home if it didn't pay economically. The best solution to get rid of those annoying phone calls is not to respond. But there are enough people that we continue to be harassed. Pop-up ads are so cheap to do. What's it cost for me to add another million of them? It's inconsequential, so we're being bombarded. Given the cost to the provider, they're going to continue offering them. There's not an easy way around that."

Billions and billions
Pop-ups, which cost just pennies each to put online, are ubiquitous. During the first seven months of 2002, advertisers launched some 11.3 billion pop-up-ad impressions on the Web, according to "Spotlight on Online Advertising," a report published in August 2002 by Nielsen/NetRatings. Most pop-ups originated from a handful of advertisers: Sixty advertisers accounted for 80 percent of all pop-ups.

According to another report by Nielsen/NetRatings--"The State of Online Advertising," published in February 2003--pop-ups still represent a relatively small fraction of online ads, even though they continued to grow in use from 2001 to 2002. As of the fourth quarter of 2002, pop-ups comprised 3.5 percent of all online ad impressions, up from 1.9 percent in the year-earlier period.

Pop-ups may be relatively new, but they have characteristics not unlike those of other forms of traditional advertising.
"Mitigating the rapid growth of pop-up advertising is the fact that advertisers and Web sites have become increasingly aware of negativity surrounding the delivery mechanism," according to the report. "Some notable Web sites have ceased offering pop-ups as an ad solution, and a few online services have gone as far as offering pop-up-blocking software with their service."

Nonetheless, figures compiled by DoubleClick, a provider of marketing tools for advertisers, direct marketers and Web publishers, show that the use of "rich media" in general continues to grow. Rich-media ads represented 31.7 percent of all online ads in the second quarter of 2003, up from 17.3 percent in the first quarter of 2002. DoubleClick defines rich media as pop-ups, dynamic ads that move across Web pages and any ads that include Macromedia Flash technology.

As things stand, consumers can take steps to keep pop-ups off their screen by installing one of many anti-pop-up software programs readily available on the Web. Some, which can be downloaded for free, eliminate virtually all pop-ups. Others, which cost $20 or $30, give the user greater control in being selective about the kind of pop-ups to be killed off.

"Basically, what a pop-up ad does is give instructions to your browser to do something special," Croson says. "What the pop-up-ad-killing software does is tell your browser to ignore those instructions. The downside is that pop-ups sometimes do good things. When you're at a brokerage Web site, you want that pop-up window to tell you your trade has been executed. So this kind of software does reduce the functionality of your browser."

Reibstein believes it is possible that the federal government could step in to regulate pop-ups in the way it has taken steps to curb telemarketing calls and spam. Earlier this year, the government announced the creation of a Web site that consumers could register with in order to have their names removed from telemarketing call lists. The Federal Trade Commission has also taken steps to halt fraudulent spammers.

"As we hear more concern about spam, pop-ups will get lumped together with spam and there will be negative reactions and calls for restrictions to be imposed; it's around the corner," Reibstein says. "All marketers really want some self-regulation to happen. They'd prefer that, rather than have government step in. I would be surprised if we do not see some strong efforts on self-regulation. If we don't do it, it's going to be done to us." One possible self-policing mechanism, Reibstein suggests, would be for producers of pop-ups to carry a kind of seal of approval.

But Williams says pop-ups, as annoying as they may be, do not typically lend themselves to fraud the way spam does, and she does not foresee any movement to curtail the use of pop-ups. "My sense is, when it comes to advertising on the Net, the government has its hands full controlling unwanted e-mails and spam, and I don't see them moving to set formats for other types of ads. Spam is considered much more problematic, and the rate of fraud with those is much higher. My sense is that spam is a much bigger or noticeable problem for the government than pop-ups ads."

Hunter says pop-ups are not an invasion of privacy in any meaningful sense, and he doubts that regulators are interested in curbing their use. "I haven't seen either regulatory movements or consumer group concern," he says. "Though it's certainly a possibility, the response on the part of regulators to really serious Net problems--such as spam and privacy issues surrounding the collection of information about consumers--has been fragmented, slow and mostly ineffective. I am doubtful that, if we can't fix spam and privacy intrusions, that we're going to see regulatory activity on this type of advertising."

Instead, it is more likely that advertisers themselves will come to see pop-ups as counterproductive. "Unlike spammers, most advertisers who use pop-ups need a better than a one-in-a-million response rate," Hunter notes. "So they'll need to find advertising mechanisms that are more about permission and less about destroying the user's experience of the Web site." Williams and Reibstein agree the marketplace is the mechanism that tells marketers they had better use pop-ups carefully. According to Williams, "There's a movement in the marketplace to say: 'Maybe pop-up ads aren't right for us. If we continue to overload consumers with pop-up ads, we may drive them away.'"

Says Reibstein: "One real danger is if people get annoyed, they may not want to come to your Web site anymore. If you're running a Web site, you have to be careful as to how much you abuse your customers."

Spam, telemarketing calls and pop-ups are all examples of communications tools that irk people. But Hunter makes the point that pop-ups are far less egregious and far less harmful to the technology that allows them to be deployed. Both spam and telemarketing involve the long-term "destruction" of a communications mechanism--e-mail and telephones--as a result of reduced transaction costs, he explains. As the financial costs of communication approach zero, the social cost becomes immense.

"You can't use your e-mail account to get important mail; you don't want answer the phone at all anymore," Hunter says. "Pop-ups are intrusive, but they stem from a site that you're interested in. If The New York Times' site continues to besiege me with pop-ups when I'm reading their content, then I can switch to the Philadelphia Inquirer site. Competition means that some sites will gain a competitive advantage from not using pop-ups, and so the inevitable destruction of the resource is mitigated."

Croson notes that although pop-ups will not win many popularity contests, they may not be destined to disappear altogether. "Being annoying does not necessarily translate into bad advertising. Door-to-door salespeople are pretty intrusive. Just because it's intrusive doesn't mean it can't be effective."

To read more articles like this one, visit Knowledge@Wharton.

All materials copyright © 2003 of the Wharton School of the University of Pennsylvania.

Interested in more research studies like these? If so, sign up for the Knowledge@Wharton Newsletter, a free service of the Wharton School of the University of Pennsylvania.