Where does social media go from here? The leaked documents known as the Facebook Papers hammered home the fact—if there was any doubt remaining—that even the world’s most sophisticated content moderation systems can’t keep pace with human misbehavior on the billions-of-users scale, or the damage generated by algorithms designed to maximize engagement.
Jeff Allen spent four years working as a data scientist at Facebook, including two years on its integrity team, before leaving in late 2019. In October, along with former colleague Sahar Massachi, he launched the Integrity Institute, an organization devoted to bringing together current and former integrity researchers to develop ideas and best practices for an industry that sorely needs them. We spoke earlier this week about how companies like Facebook should rethink their approach to platform design—including taking a page from the media industry at the turn of the 20th century.
WIRED: We’ve all heard a lot about how Facebook optimizes for engagement. But you could argue that people just like to click on the scandalous, the naughty, the provocative, the whatever. Are we blaming algorithms for user behavior?
Jeff Allen: This is a really good question, and sort of a hot debate topic internally. To what extent are you just giving people what they want?
A historical anecdote I keep in my head is The New York Times in the 1890s. You had yellow journalism, you had young boys standing on street corners yelling out crazy things just to get attention so people would notice them as they walked by. If you have to say something that'll get people to buy a newspaper in that moment, the headlines are going to be really sensational.
My understanding is that the new business model led by The New York Times was, “What if you made a brand that people trust and you yelled out, ‘Hey, this is our brand; this is where you buy our newspaper.’” That’s where “All the news that's fit to print” comes from. Instead of yelling about whatever crazy story they're inventing, you're yelling about, “Hey, this is where you get The New York Times.”
I actually did look it up, and in the early 1900s The New York Times was experiencing tech levels of growth, like 10X-ing over a decade kind of stuff. So the New York Times was like the tech industry of the early 1900s.
It was a unicorn.
Like the OG analog unicorn.
I'm glad you brought this up because one thing people will say is, “Look, all media optimizes for engagement because you want people to click on your headlines.” And there's some truth to that for sure, but at the same time, there's a pretty hard limit to it. If you look at the front page of The New York Times today, one article is “Biden Signs Bill for Bolstering Infrastructure” and another one is “Europe Targets Unvaccinated in Virus Spike.” That's not clickbait. There's some judgment going on there about what's important, what would be good for readers to know. It's not purely about what will optimize the click rate. There are other values at play.
Most PopularThe End of Airbnb in New YorkBusiness
I’ll push this to its limit. There are things that happen in the media industry that are very problematic, and that would be “If it bleeds, it leads” evening news kind of stuff. I got really into this because how crime is discussed online is very problematic.
Interestingly, when the “If it bleeds, it leads” type of TV evening news emerged in the late ’80s and early ’90s, it was very metrics-driven. There were new measurement systems that allowed the TV industry to realize that people who watch the program before the evening news—whether or not they stick around to watch the actual evening news broadcast after the entertaining show has a whole lot to do with the first five minutes.
What you’re saying is the whole TV news thing of leading with salacious crimes, which gives viewers a warped perception of the risks of violent crime, came from a new ability to focus on engagement as a metric.
So yes, I do believe that if you’re just giving people what they want in that moment, it is going to skew toward sensational stuff. That was true in the newspapers on the sidewalks in the 1800s, it was true in evening news in the ’90s, and it’s true when you’re optimizing click-through rate and stuff like that.
If you’re a social media company, your goal actually isn’t to maximize engagement tomorrow. Your goal should be to maximize engagement a year from now, or five years from now, or 10 years from now.
You can see from Facebook’s own “widely viewed content” reports that the posts that get the most views are not necessarily pernicious, but are probably not what most people would consider a really meaningful, high-quality experience. They might be just regurgitated engagement bait, like posting a question that people just can’t resist commenting on.
That seems like an example of the short-term incentives being very meaningful, because if you were thinking about long-term incentives, you probably wouldn’t want to see these engagement-bait posts doing such fantastic traffic on the platform.
I will quote my cofounder, Sahar Massachi. If you have a platform and you designed it in a certain way, and it operates in a certain way, and it rewards users in a certain way, if you let it run that way for a long time you’re going to build up winners and losers. And you can end up in what Sahar calls the “stakeholder trap.” Which is, if your platform rewards the wrong people for long enough, suddenly those people can become very powerful.
If we had a hypothetical platform—and maybe we don’t have to be too hypothetical—that was like, “Hey, we’re the most toxic content platform. If you’re getting banned everywhere else for your toxic content, come here and post it.” And they build up a million users branding themselves as the toxic content platform. Now, say they have a new CEO who wants to grow beyond that. They’re going to be trapped because all of their users are going to be into this toxic content. In order to pivot out of that strategy it’s going to be a very, very big Band-Aid to rip off.
Most PopularThe End of Airbnb in New YorkBusiness
So what’s a better way to think about addressing these problems from a design perspective?
This question is one of the reasons why we founded the Institute, because there’s actually been a handful of things that have been tried in the space, to varying degrees of success, but a lot of that knowledge exists within small teams within companies and it hasn’t been broadly disseminated.
One of my favorite examples that I always point to is the Google search quality team and the work that they were doing at least until 2015 or so. Google built out search quality guidelines. Everything is very objective; they’re not qualitatively assessing the content, they’re just looking for objective criteria. A lot of it is actually just basic media literacy checks, like: All things being equal, it is better if the publisher or the creator of the content is transparent about who they are. Another one is various ways to assess how much effort went into the content, because all things being equal, it’s better if more effort is going in. The lowest-quality signals here would be, is the content copied from somewhere else?
On this point, though, about determining measures of quality, it seems, on the one hand, like: Duh, of course platforms should try to show users the good and not show them the bad. But it seems they shy away from this, at least in Facebook’s case, because they’re afraid to be seen as playing favorites, especially among user-generated content.
A lot of the social media companies coming out of the 2000s era of the internet, a lot of their mission statements and their values are all toward giving everyone a voice. YouTube’s mission statement is “Give everyone a voice and show them the world.” Twitter’s mission statement is, I forget exactly—
“To give everyone the power to create and share ideas and information instantly without barriers.”
“Instantly without barriers,” yeah. Facebook’s early mission statement was like, “connect everyone in the world.”
All these mission statements are very much just like, “Let everyone speak, show everyone everything, put everyone together,” and these aren’t amenable to any objective definition of quality, to saying what is the kind of content that we want to succeed on the platform.
And they’re all very growth-amenable. We should not at all be surprised that the big platforms that survived the first generation or two of social media companies were the ones that prioritized growth, the ones that saw that the bigger you are, the more useful you are, and so you have to get as big as possible as fast as possible.
There’s a pessimistic implication of that, which is, Facebook and other dominant platforms make a ton of money doing things the way they do them now. And yet one thing that the Facebook Papers revealed is that as dominant as Facebook—or Meta—is in the market, they’re still really scared about potential rivals like TikTok. So if you’re proposing making changes that could sacrifice some of that short-term immediate engagement, you imagine the leaders of these companies thinking they can’t take the risk of some bored kid opening up TikTok because Facebook is trying to make them read a New Yorker article. So are we being naive even talking about platforms changing course in this way?
Most PopularThe End of Airbnb in New YorkBusiness
That’s a great point. I’m not entirely pessimistic, and I do think there’s platforms that are doing things right here and there. I don’t think any platform solves all the problems, but I do think, if you look across all the platforms, you will find that various platforms are doing better or worse than others. We haven’t found one that’s been great on everything, but bits and pieces here, you can see success.
Is there a particular example of a design change that sticks out to you as one that should be celebrated or that makes you optimistic?
A lot of the work that YouTube has done over the past few years to figure out what do authoritative videos look like, and rewarding more authoritative videos, has made it much harder to exploit or game YouTube. And when you look at the most-viewed content on YouTube, it is all generally content that people are putting effort into. So I do think that there are signs that a big platform can think about the ecosystem that they’re creating and the long-term incentives of it.
One thing I think about a lot is, we’re still not done here. TikTok is not the last platform that will reach a billion users. In the long-term, we’re going to have a lot of platforms that figure out that the easiest, fastest business model is to show sensationalist stuff. The integrity worker part of my brain is like, we’re not done seeing all the different types of content that can be bad on the internet. We’re not done seen all the different ways social media platforms can be poorly designed.
So I think there’s a big role for regulation to play in constraining how these businesses can run themselves, and hopefully for the Integrity Institute or another organization like it. There’s a few aspirational models that we have for the Integrity Institute. One of them is the Open Web Application Security Project, a group that came together around cybersecurity. They all worked at separate companies, but they were talking to each other and saying, “Hey, this is the best practice,” and these best practices were shared among the companies and the community.
Everyone benefits when the web is more secure.
Another aspirational model would be the Associated Press. Where journalists sort of came together to say, “Let’s make journalism a profession, and if journalism is going to be a profession, here are going to be the rules of the road.”
I’m super sympathetic to that argument. I mean, I’ve written that exact article—
Yeah, actually your articles were some of the most popular ones in our community.
I'm keeping that in the transcript, for sure. But to challenge my own idea, there are partisan dynamics that really complicate the effort to talk about quality or standards. In the case of Facebook, there have been episodes over the company’s history where somebody comes up with an idea to penalize lower-quality publications, and then somebody in the policy department pipes up and says, “Well, that would disproportionately hurt conservative publishers, so we shouldn’t do it.”
Most PopularThe End of Airbnb in New YorkBusiness
As much as I agree with your analogy to journalistic standards, I wonder how much of that can get done, given the way media has polarized asymmetrically. You have a right-wing press that, whatever the weaknesses of the left-wing press, is on a qualitatively different level of unscrupulousness. Because you have that dynamic, you’re always going to run up against this partisan problem.
I’ll just say it’s a genuine challenge. Attention online is valuable, and so a lot of different people are going to be fighting over it, on a lot of different fronts. And if we think it’s bad in the US, it gets much worse internationally.
On that point, how should platforms think about the tradeoff between introducing a new communication platform in parts of the developing world where it could be really useful to people against the fact that the risks of platforms being used badly are so much higher in some of those same places?
This is something that we're really concerned about at the Institute. There’s this idea floating around that in order to launch a social media platform in a particular country, you need to have representatives of that country guiding you and telling you what a responsible platform looks like.
Do you have any advice on how to have the most positive experience on social platforms?
Yeah: Put a lot of effort into it.
Whenever you find yourself lazily scrolling through Twitter or Instagram or Facebook or YouTube, stop yourself, think about content on that platform that you would like to see, and then figure out what are the right accounts that are producing the right content for that.
When you start doing this, social media becomes a wonderful place. My Instagram is nothing but astronauts and really great amateur astronomers and physics experiments. The algorithm didn’t figure out, “Oh, he’s a former physics PhD and an astronomer and we should recommend astronauts and physics experiments to him.” No, I had to seek it out. I love the CERN account on Instagram; they have throwback Thursday where they post, like, images of the Large Hadron Collider in the ’60s, and it’s phenomenal. I love it.
Readers won’t be able to tell this because they’re not seeing the video, just the transcript, but you’re, like, jumping out of your chair with enthusiasm.
I freaking love physics Instagram. But the recommendations are not going to give it to you; you have to go find it. You have to figure out what you’re passionate about and you want to go deep on and where’s the best experience for that in social media.
So if I’m just scrolling mindlessly—
Stop and go searching for something cool.
More Great WIRED Stories📩 The latest on tech, science, and more: Get our newsletters!The 10,000 faces that launched an NFT revolutionA cosmic ray event pinpoints the Viking landing in CanadaHow to delete your Facebook account foreverA look inside Apple's silicon playbookWant a better PC? Try building your own👁️ Explore AI like never before with our new database🏃🏽♀️ Want the best tools to get healthy? Check out our Gear team’s picks for the best fitness trackers, running gear (including shoes and socks), and best headphones