Summary
The Chaos Machine is an all-encompassing narrative of the history of social media and weaves information about aspects of psychology and newer discoveries about psychological vulnerabilities with the cultural and structural elements of the big social media tech companies to give insights into the problems that social media has created in our world.
You've probably heard bits and pieces of this story before, but this was the first time I have seen the whole narrative laid out so well. The book is unique in that it traces back some of the problems with social media back to the very beginning of Silicon Valley. The cultural legacy that continues to leave its mark within the leadership of social media companies are prioritizing very young (and perhaps not wise) talent, predominantly male, who often have a contrarian and libertarian outlook on the world, and will work hard to grow extremely fast. Because of economic factors at play, there is a lot of money from angel investors being spread across a lot of different companies, hoping to invest in one that makes it very big, which encourages extremely fast moving and sometimes non-specific growth models with poorly constructed business plans. Together this environment leads to a culture that strongly promotes individual responsibility, with less institutional incentive to internally address negative uses of the technology in the world, and also a culture obsessed with metrics of engagement and usage above everything else.
My favorite aspect of this book was the description of the ways in which social media can target certain psychological vulnerabilities. There are certain aspects of psychology that could be "hacked" which were known already at the time that social media was being developed, and social media developers explicitly tried to use those to drive engagement. Those were mostly around motivation. They tap into motivational hacks like by providing external rewards in ‘likes’ and social approbation, which are extremely motivating. They also apply rewards on an intermittent variable schedule (for instance by intermittently raising your post to the top of feeds where it will receive more likes), which has been shown to maximize engagement within a reward schedule.
But there were other psychological phenomena that really weren't well understood until after social media. One was increased stress and the tendency to enforce hierarchies if social communities become too large, which is also seen in macaques. Another was how certain kinds of posts are particularly attention grabbing- mainly posts with moral emotional words, such as ‘abuse’, ‘honor’, ‘spite’, ‘shame’, ‘disgust’, etc.. - and this interacts with algorithms that attempt to incentive behavior that captures attention. Algorithms try to provide you with attention grabbing material, while also shaping you to become someone who attends more to certain kinds of content. Because these moral-emotional words are used most in divisive posts, algorithms show more divisive content and also prioritize divisive content that you agree with (which is more engaging). The result is that you are presented with an echo chamber where extreme expressions of divisive opinions are both normalized and rewarded, which ends up shaping your own opinions as well.
Algorithms also recommend relatively fringe ideas more frequently because they are more divisive, which interacts with our cognitive bias to believe that information frequency is a proxy for something being true, called the ‘illusory truth effect’.
One of the most disturbing aspects of the book was the ways in which social media algorithms attempt to change you, rather than just catering to you. It would be bad enough if social media was just presenting you with false and divisive information all the time, but it also tries to transform you into someone who is more engaged with its material. One example that the book discussed was around irony poisoning, where researchers found that starting in communities that make fun of something "ironically" can lead to desensitization and a greater likelihood of adopting a real viewpoint - so like if you're in a community that makes a lot of sexist jokes, to joke about negative stereotypes of men or women, even if it starts out as just funny or like emotionally releasing, it can make you actually more sexist over time. Another example was about the process of radicalization, where YouTube will lead you down a rabbit hole of progressively more problematic content - so you could start off being interested in something more borderline - like say, if cultural appropriation is taken too far - and it quickly leads you into white nationalist content. Another disturbing example was the discussion around sexual desire, where algorithms can learn to bring you to progressively more "deviant" content. The very creepy example here was YouTube learning that it could increase engagement by progressively promoting videos of younger and younger girls until it was promoting home videos of very young children, which were getting millions of views.
There were also many examples of the results of how social media has caused large amounts of suffering and problems in the world, and also how unmotivated they seem to be to do anything about it. I personally felt that the most egregious examples were of Facebook working with a dictatorship in Eastern Asia to support governmental speech and suppress oppositional content in order to keep Facebook operating there. The second yucky example was when Facebook removed its network from Australia in response to a battle with the Murdoch family to bully Australian legislators into supporting Facebook, even though Facebook wouldn't shut down its services when its use was driving a genocide in Malaysia. These examples really demonstrate how hypocritical and evil social media companies can be.
Discussion Questions
I had heard a lot of this story before, but there were lots of new parts of it for me, too. Did you learn anything new that really stood out to you?
Did this book make you change your behavior on or use of social media?
What do you think the answer is to ameliorate some of the problems that social media is causing?
Do you think that there could be a positive social media in the world, or do you think that these problems are intrinsic to the technology?
Is social media posing a new problem or more of the same? (If misinformation and divisiveness drives engagement, didn't news sources of the past have the same incentives and problems?).
In a world where we see such powerful negative effects of social media, have your feelings about the constitutional right to freedom of speech changed? What is the best way to balance freedom of speech with the profound negative consequences of misinformation and promoting hateful content?