Welcome to your Turkopticon Newsletter
We have had a busy year so far and with your help, we can carry on trying to make turking better for us all.
To leave reviews please visit our homepage turkopticon.net.
Have you been affected by a mass rejection? We’d love to hear your story. Please contact us.
See the bottom of the newsletter for how you can submit short articles.
Our Next Open Forum
If you have thoughts on Turkopticon’s mass rejection campaign or any other issues facing turkers, we would like to invite you to our open community forum on Thursday, September 8th. Turkers can stop by between the hours of 1-3 pm EST. As always, we will try to answer questions about the platform itself, turker tools and forums, how to get started on AMT, and more. Please fill out the registration form at the link below and we will provide the Zoom link. Camera use is optional. Feel free to come for 2 minutes or the full 2 hours. We look forward to chatting with you all next week!
Signup at https://turkopticon.net/open-
If you want to learn more or have questions, please email us at firstname.lastname@example.org.
It is challenging to come up with a new, original idea on what to say about an issue that’s been going on for years: unfair mass rejections.
It has been more than 14 days since requester AI Insights immediately rejected all of the HITs submitted to them without any feedback or communication. Assignments (submitted tasks) can be reversed within 30 days of submission. Currently, the requester has less than 16 days to overturn the rejections. We, the workers, can only hope these will be overturned while our approval ratings and ability to work on the platform hang in the balance.
This requester had an approval rating of 99% before posting these HITs. Using this information, workers felt safe completing the HITs, and the batch, made up of over 75,000 tasks, disappeared quickly. The HITs were simple choice tasks and seemed like a great batch until the rejections came in. The requester’s approval rating promptly fell to 5% the following day. [If you want to read some of the nightmare-ish reviews that workers have left, you can go to Turkopticon, Turkerview, or Reddit (1 / 2 / 3 / 4 / 5).]
Even worse, this is not an isolated case. There have been reports of workers’ submissions being rejected because, among other reasons, the requester had an error on their side (like Team Seminar & Fan Zeng), cancelled the study at the last minute (like SeunghyunShin), or gave vague feedback like “Wrong Answer” (Janine Chen, which has an approval rating of 2%) or “Doesn’t meet my requirements” (Pedro Amaro) only for the approval rating to disappear from the platform without any further explanation, or even took the responsibility to Amazon Mechanical Turk saying it is with them that we, the workers, have a contractual relationship and thus “they are the ones who should face the consequences” (Victor Ramos).
According to Amazon’s Developer Guide, it’s recommended that requesters “reject work when workers are clearly putting in no effort to submit an accurate response to your task. It’s inappropriate to penalize a worker for submitting data incorrectly because you provided unclear instructions or they simply made a mistake in interpreting what you wanted them to do. Most workers zealously guard their approval rating and avoid doing work for requesters that they believe are unfair in how they reject work”.
If you ever wondered what a big subsidiary of Amazon comes up with as a way of protecting workers, there you have it, it is an article. This is the same “solution” offered to the Turkopticon team during their last meeting with the Mturk leadership.
Amazon needs to address this issue seriously. Washing its hands of the matter while saying, “we are unable to mediate in disputes between Workers and Requesters” is not a solution, especially if workers and requesters are to believe that Mechanical Turk is an ethical and fair platform for everyone.
Other marketplaces have developed better mechanisms for handling complaints, appeals, and bad actors, so why is Amazon unable to do the same thing? Don’t they have enough resources to work on this?
Even if you were not affected by this issue with any of these requesters, it shows that something like this can happen to you and your account at any time and that you will not have protection or recourse.
Per Amazon’s suggestion, workers should report any questionable behavior on their platform, so we invite you to send a polite and respectful email to mturk-worker-support@amazon.
We are not “ghost workers”, our livelihoods and needs are real and affected by your inaction.
Turkopticon is looking for suggestions!
Do you have suggestions for how to make Turkopticon better and would like to share them with us?
Please email email@example.com and someone will be in contact to further hear about your suggestions and see how we all can work together to make Turk better for all of us.
Even if you think your ideas are ones we should know please don’t be afraid to send them our way because we listen to all suggestions and want everyone to be heard!