A new California law designed to prevent the warehouse industry from overburdening employees does not name a specific company. But the purpose of the legislation is clear: Amazon, which has given machines unprecedented control over workers and is accused of using technology to place unreasonable demands on them.
Written by MK Lorena Gonzalez, the bill bans the use of surveillance systems that undermine basic workers’ rights such as rest periods, toilet breaks and safety. The legislation will help determine whether governments can regulate human resource software which should play an increasingly important role in deciding who gets hired and fired, how much workers are paid, and how hard they work.
“This is just the start of our work to regulate Amazon and its algorithms that put profits before worker safety,” Gonzalez, a Democrat from San Diego, tweeted earlier this year. The legislation, signed by California Governor Gavin Newsom in September, will come into effect on January 1.
Regulators are constantly catching up, especially with the tech industry. Computer experts doubt that laws can effectively regulate machines that should ultimately be smarter than humans. Designing artificial intelligence systems that meet their objective remains a feat in itself. It is even more difficult to ensure that they do not cause unintended consequences.
Amazon has outsourced many of the roles traditionally played by human managers to machines. In giant distribution centers, the software determines how many items a facility can handle, where each product is supposed to go, how many people are needed for a given shift, and which truck is best positioned to speed up a shift. order to a customer on time. Algorithms and cameras continuously monitor delivery drivers, making sure they drop off a certain number of packages per shift, place them correctly, and obey traffic laws.
The company argues that automation is necessary to run its sprawling operations, and says the technology mostly works as intended. But no algorithm is perfect, and even a small margin of error in a company the size of Amazon can inflict lasting collateral damage.
In the past year, a Bloomberg Algorithmic Management survey recounted the experience of a concert delivery driver mistakenly fired by a machine, an aspiring doctor paralyzed after an Amazon delivery driver harassed crashed his car and warehouse workers who said they felt like throwaway cogs in a machine.
Time and time again, workers have called algorithms masters of ruthless tasks. They described a ruthless workplace where people often don’t stay long, injuries are higher than the industry average, and employees are expected to meet unreasonable productivity quotas.
In the coming years, companies are generally expected to embrace aspects of the management automation pioneered by Amazon. The machines already regularly review job applications, determine work schedules, and even determine which employees plan to quit.
So far, Amazon hasn’t sold its worker monitoring software to other companies – and some industry watchers believe it never will. But Amazon’s cloud division has started in recent years selling tools designed to automate real-world tasks, including some developed for its e-commerce and logistics operations.
Last year, Amazon Web Services announced a suite of products to monitor plant equipment and lines, supplementing or even replacing human workers. Companies as diverse as Capital One, Labcorp and GE Appliances use Amazon Connect call center software, which deploys artificial intelligence tools to make human agents more productive and fully automate certain interactions with customers.
The growing ubiquity of algorithms has sparked calls for legislation that would force companies to be more open about how this software affects people. Last December, Senator Chris Coons, a Democrat from Delaware, introduced the Algorithmic Fairness Act. This would force the Federal Trade Commission to create rules to ensure that algorithms are used fairly and that those affected by their decisions are informed and have the opportunity to correct their mistakes.
“Artificial intelligence brings real benefits to society and opens up exciting possibilities. However, it also comes with risks, ”Coons said, after unveiling the bill. “Companies are increasingly using algorithms to decide who gets a job or a promotion, who goes to a certain school or who gets a loan. If these decisions are made by artificial intelligence that uses unfair, biased or incorrect data, it has a disproportionate impact on people’s lives.
So far, the proposal has stalled in a divided Washington struggling with more immediate concerns, from the pandemic to voting rights.
California law is narrower in scope. It requires warehouses with 100 or more employees to disclose performance quotas to workers and ban workloads that prevent them from taking legally mandatory meals and breaks. The law seeks to give workers redress if quotas violate safety regulations and gives the state labor commissioner the power to review workers’ compensation cases at facilities with high injury rates and issue citations if injuries are attributable to excessive workloads.
Industry groups opposed the bill, arguing it would encourage workers to file lawsuits that would impose unnecessary costs on employers.
An Amazon spokesperson said the company would brief officials at its 32 California fulfillment centers on the productivity tracking process, but declined to say what changes would be made to comply with the law. He also said Amazon doesn’t have workload quotas but uses productivity metrics to identify employees who need help meeting expectations.
IT scientists have long debated whether algorithms and artificial intelligence can be regulated effectively. Roman Yampolskiy, a professor at the University of Louisville who studies AI security, says the technology is such a black box that it’s hard to know if it creates potentially dangerous situations. He is skeptical of the intended effect of the California law.
“We don’t always know why machines make certain decisions, because it’s a complex network of neural networks,” Yampolskiy said. “Every time you write down what you are trying to accomplish, smart people will find a loophole to get around this problem. The legislation will be played by algorithms and their owners. “