Amazon’s AI managers targeted in California legislation meant to empower warehouse workers
- AB 701 prohibits the use of monitoring systems that thwart worker rights such as rest periods and safety
- Amazon argues that automation, which it uses to monitor worker productivity, is needed to manage its sprawling operations and mostly works as intended
Authored by Assemblywoman Lorena Gonzalez, AB 701 prohibits the use of monitoring systems that thwart basic worker rights such as rest periods, bathroom breaks and safety. The legislation will help determine whether governments can regulate human-resources software that’s expected to play an increasing role in deciding who gets hired and fired, how much workers are paid and how hard they work.
“This is just the beginning of our work to regulate Amazon & its algorithms that put profits over workers’ safety,” Gonzalez, a San Diego Democrat, tweeted earlier this year. The legislation, signed by California Governor Gavin Newsom in September, goes into force on January 1.
Regulators are constantly playing catch-up, especially with the tech industry. Computer science experts are doubtful laws can effectively regulate machines that are ultimately expected to be smarter than people and even capable of tricking them. Designing artificial intelligence systems that meet their intended purpose remains a feat in itself. Ensuring they don’t cause unintended consequences is even harder.
Amazon has outsourced many of the roles traditionally played by human managers to machines. At giant fulfilment centers, software determines how many items a facility can handle, where each product is supposed to go, how many people are required for a given shift and which truck is best positioned to speed an order to a customer on time. Algorithms and cameras constantly monitor delivery drivers, ensuring they drop off a certain number of packages per shift, place them correctly and obey traffic laws.
The company argues that automation is required to manage its sprawling operations and says the technology mostly works as intended. But no algorithm is perfect, and even a small margin of error at a company of Amazon’s size can inflict lasting collateral damage.
Over the past year, a Bloomberg investigation into algorithmic management chronicled the experience of a gig delivery driver mistakenly fired by a machine, an aspiring doctor paralysed after a harried Amazon delivery driver ploughed into his car and warehouse workers who said they felt like disposable cogs in a machine. Time and again, workers characterised the algorithms as merciless taskmasters. They described an unforgiving workplace where people often don’t stay long, injuries are higher than the industry average, and employees are expected to meet unreasonable productivity quotas.
In the coming years, companies are widely expected to adopt aspects of the management automation pioneered by Amazon. Machines already routinely sift through job applications, determine work schedules and even figure out which employees are planning to quit.