The past decade has seen the rise of tech-evangelism, predicated on the belief that every weakness of the human condition has a data-driven solution. However, while we are enthralled by the ‘ease’ offered by technology, we must remain wary of the destination we are driving or being driven to. In their quest for ‘smart’ solutions, technologists may create a future where choices are made for us, human activity is reduced to ‘manageable’ data points and spaces for dissent are eroded.
‘Wearable technologies’ have fast evolved from tracking our step count and helping us navigate roads, to monitoring the performance of our heart and lungs, and even “augmenting” our perception of reality.
“Efficiency and conformity” is at the core of this age of wearables: getting the optimum number of hours of sleep, clocking those 10,000 steps a day, counting calories or reaching your destination as fast as possible, all with the ultimate aim of being able to easily quantify our existence and activities and enable us to function like well-oiled machines.
Amazon is perhaps most notorious for the use of wearables to monitor worker productivity – a future that may be in store for the millions around the world now “working from home”. A link less well-documented is that between the “beautiful madness” of Amazon’s warehouses and its role in enabling employee surveillance. Amazon fulfilment centres follow an organising system called “chaotic storage”: products are stored, seemingly at random, with each shelf marked by a barcode. When products have to be retrieved, a computer system generates a “picking list” and directs workers to the relevant shelves. Chaotic storage is meant to reduce the time workers spend in retrieving products, but in the process, make workers completely reliant on trackers, which in turn opens them to perpetual surveillance. Does the hunger for “efficiency” cost workers their dignity and rights?
The coming decades will witness both the growing ubiquity of smart technologies as well as the increasing invisibility of the people behind them. Whether it be contact-less delivery, the reduction of workers to a moving dots on your phone screen, or the millions of data labellers feeding the AI boom, earning a few cents for hours of work. Is the dystopian future where humans serve the needs of machines already upon us?
There is a definite tinge of paternalism behind many of the most egregious uses of smart technologies. As the emotion recognition company Affectiva proudly declares, “We help businesses understand how their customers and consumers feel when they can’t or won’t say so themselves.” Do the machines understand mere mortals already? The unshakeable belief that innovation and its products will bring great benefit to the future of humanity leads to a blindness to the very real, very immediate consequences it may have.
The ongoing pandemic alerts us to this tendency. The widespread and necessary use of masks interfered with the functioning of facial recognition software. Enter the solutionists: masked face recognition datasets were made free to download, makers of facial recognition tech raced to adapt, with some working to recognise individuals just from their forehead.
The rationale – making facial recognition more accurate for COVID times – adds an aura of legitimacy to the task of making masked faces easily recognisable by machines. Masks were once an accessible way for protestors and activists, notably in Hong Kong, to thwart law enforcement use of facial recognition systems to suppress citizens’ freedom to dissent and express themselves. It is no surprise then that companies in mainland China were early providers of a workaround against the masked faces. By March, one company – Hanwang – claimed their tech had achieved 95% accuracy in unmasking faces. Courtesy the pandemic, law enforcement agencies the world over can legitimise the deployment of “mask-proof” facial recognition.
The history of technology is a history of expansion of human functionality, and emerging technologies in the field of artificial intelligence will take us to the next frontier of this human story. However, as companies race to find solutions, with complete disregard for humans and humanity, it is important for technologists, users and policymakers to consider whether more technology truly is the answer to all our problems. In the words of Dr Ian Malcolm, “…
COVID-19 has exaggerated the trend toward tech ‘solutionism’. Nations and societies have turned to a host of data-driven technologies as the central element of their response to the pandemic. In the process, governments have discovered once again the utility of technology as a tool to manage citizens, a trend that began well before the pandemic, but has gotten a fresh lease on life under the garb of crisis response. The list of victims of the pandemic may well include human choice, human expression and being human.
The views expressed above belong to the author(s). ORF research and analyses now available on Telegram! Click here to access our curated content — blogs, longforms and interviews.
Trisha Ray is an associate director and resident fellow at the Atlantic Council’s GeoTech Center. Her research interests lie in geopolitical and security trends in ...
Read More +
PREV
