Chapter 33: The risk of a techno-dystopian future

Today, many aspects of our lives remain offline. If you walk into a store with nothing but a handful of coins, you know you’ll usually be treated just as fairly as the next customer. If you leave your technology at home and go for a hike on a nature trail, you know you’re anonymous, untracked and free to commune with nature, uninterrupted. But we’re already getting glimpses of a future where that isn’t true – where you’re perpetually monitored, sold to, manipulated and given a different deal to the next person, owing to some opaque business decision that occurred on a server farm on a different continent. In the 2014 research paper ‘Dark Patterns in Proxemic Interactions’, Greenberg et al. describe the possibility of science fiction like Minority Report or Black Mirror happening in real life, where invasive tracking and targeted advertisements can never be avoided.¹ They mention the example of experimental vending machines in Japan that have dynamic pricing based on face recognition, emotion detection, time of day and temperature – in other words, the price is only presented to you once the system has worked out the most you’re likely to be willing to pay.² In the book You’ve Been Played, Adrian Hon describes the slide of tech industry philosophy into everyday life: how Amazon warehouse workers are tracked and pressured to maximise their physical performance; how Uber drivers are sent personalised ‘quests’ to keep them driving; and how China scores its citizens to ensure compliant behaviour.³

There’s something very unsettling about the online world becoming the only world, and having no ability to escape it. Perhaps the problem with deceptive pattern research today is ...