What is the emotion expressed in the following sentence?
Mark dropped his ice cream cone and began to cry.
- Happiness
- Sadness
- Anger
- Indifference
- Regret
- Frustration
- Joyful
- Excitement
Clearly, the answer is 2. Of course he is sad! Unless he is crying because he is frustrated. Maybe he is crying because he regrets dropping the ice cream? Well, he could also be crying out of anger.
I’m one of the people sitting behind a computer, completing tasks on Amazon’s Mechanical Turk (AMT) Crowdsource Platform and I see tasks that look like the one above often. By the time I finished contemplating the “simple question” I had spent 1-2 minutes reading and rereading. After completing 100 tasks, my efforts netted me $3.00 and cost me nearly two hours of my time.
Would it surprise you to learn that I help program the smart devices you use in your daily life? I work on a crowdsourcing platform Amazon created to power AI. Amazon dabbles in many things. While the general public is familiar with Alexa, Prime Video, Audible, and so on, many people have not heard of Amazon Mechanical Turk (AMT). In its simplest form, it is a crowdsourcing platform in which requesters approved by Amazon, often data scientists and AI researchers, place microwork on AMT to be completed. Workers known as Turkers choose tasks and complete them for money.
As of 2019, there were 250,810 of us who had completed at least one task on AMT. I am certainly not saying each one of those turkers completed tasks related to machine learning (ML) and large language models (LLMs). Amazon did begin to market Turkers in 2017 with a tutorial on crowdsourcing ML training data, however.
With big tech companies throwing everything into the “Great AI Race,” it wasn’t long before Amazon introduced Titan to compete with ChatGPT and Bard. Was anyone surprised?
Being the curious person I am, I clicked around on Amazon’s website to read more about Titan. I took notice of what the bottom of the page said in large print:
“AI and ML have been a focus for Amazon for over 20 years, and many of the capabilities customers use with Amazon are driven by ML. We have thousands of engineers at Amazon committed to ML, and it’s a big part of our heritage, current ethos, and future. Amazon Titan models are built by leveraging Amazon’s decades of experience to make ML accessible to anyone who wants to use it.”
Coincidentally, AMT, founded in 2005, was ripe with batches of work asking users to analyze sentiments, flag hate speech, annotate data, and more. While “thousands of engineers at Amazon [are] committed to ML,” thousands of underpaid and unrepresented workers have performed tasks over the last 18 years on the AMT platform to build the products you are now being introduced to.
Jassy is likely right. I am certain it does take billions of dollars to train ‘really good AI models.” That seems like a great reason for a company to build a crowd-work platform. Amazon could use its product to build its two LLMs that are marketed as Titan and make a small profit off of other companies needing to harness the “power of the crowd,” as crowdwork is so often referenced. The crowdwork model was, of course, designed to save companies money and deliver quick results. I imagine that Jassy and Amazon saved billions more by having turkers around the world complete the tasks necessary to build Titan at a MUCH lower rate of pay than they would be required to pay their employees. Yet, it should be noted that even though AMT workers are the backbone of these products, Amazon does little to recognize their efforts or fix even the simplest of problems.
On August 19, 2022, one of the requesters approved by Amazon to list work posted a very large batch of work. Many turkers completed at least a couple of the tasks. Some turkers completed a couple hundred (or more.) As soon as the worker submitted the tasks, they came back rejected. The impact was huge! The workers lost pay and lost time. Worse than that, worker approval ratings took a negative hit with each rejection. Some workers lost their ability to qualify for work on the platform. What did these workers do to deserve this?
Nothing.
The workers did nothing to deserve this. The requester made an error in their coding, took the data they did generate, rejected all of the work, and disappeared. What the requester did was against the AMT Terms of Service. It would seem that Amazon would surely make this right for the workers…right? It is worth mentioning again that Amazon’s smart products wouldn’t function without the turkers. It actually would seem that this means nothing because Amazon won’t even go into their system and manually override the rejections or restore the approval ratings to their previous number. They do not care that workers lost time, money, or even the ability to do work on the platform in some cases. Turkopticon and supporters have asked for this situation to be corrected by Amazon. Turkers have spoken up, asking Amazon to fix this mess that happened on their platform. We are not a product. We are humans working on AI whether companies like Amazon want to recognize that or not. The above article about the ‘really good expensive AI models’ has an important piece that I would wager is being overlooked by many, although I’m sure not intentionally. This isn’t the flashy part but it does, in fact, speak to Amazon’s ethos as they state on their website:
“Sivasubramanian and Saha declined to talk about the size of the Titan models or identify the data Amazon used to train them, and Saha would not describe the process Amazon followed to remove problematic parts of the model training data.”
We know those answers.
There are many responses from PR regarding these things and I expect we’ll be seeing them again very soon. I have just one request: please see the workers.
Amazon, you can do better. |
