Turk Workshop

Workshop @ Association for Psychological Science 2016

hi

Helpful Turk Links

Experiment platforms

Blogs, tutorials

Journal Articles

  • Barnhoorn, J. S., Haasnoot, E., Bocanegra, B. R., & van Steenbergen, H. (2014). QRTEngine: An easy solution for running online reaction time experiments using Qualtrics. Behavior Research Methods, 1–12.
  • Brown, H. R., Zeidman, P., Smittenaar, P., Adams, R. A., McNab, F., Rutledge, R. B., & Dolan, R. J. (2014). Crowdsourcing for Cognitive Science–The Utility of Smartphones. PloS One, 9, e100662.
  • Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3–5.
  • Coenen, A., Markant, D., Martin, J., & McDonnell, J. (2013). Using mechanical turk and psiturk for dynamic web experiments. In Proceedings of the 35th annual conference of the cognitive science society (pp. 22–23).
  • Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS ONE, 8, e57410.
  • De Leeuw, J. R. (2014). jsPsych: A JavaScript library for creating behavioral experiments in a Web browser. Behavior Research Methods, 1–12.
  • De Leeuw, J. R., Coenen, A., Markant, D., Martin, J. B., McDonnell, J. V., & Gureckis, T. M. (n.d.). Online Experiments using jsPsych, psiTurk, and Amazon Mechanical Turk. Retrieved from http://www.nyuccl.org/papers/jspsych-psiturk-tutorial-2014.pdf
  • De Leeuw, J. R., & Motz, B. A. (2015). Psychophysics in a Web browser? Comparing response times collected with JavaScript and Psychophysics Toolbox in a visual search task. Behavior Research Methods, 1–12.
  • Enochson, K., & Culbertson, J. (2015). Collecting psycholinguistic response time data using Amazon mechanical turk. PloS One, 10, e0116946.
  • Finnerty, A., Kucherbaev, P., Tranquillini, S., & Convertino, G. (2013). Keep it simple: Reward and task design in crowdsourcing. In Proceedings of the Biannual Conference of the Italian Chapter of SIGCHI (p. 14). ACM.
  • Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26, 213–224.
  • Gosling, S. D., & Mason, W. (2015). Internet Research in Psychology. Psychology, 66. Retrieved from http://www.annualreviews.org/eprint/9dfeCg4rImCvYPKdjYc7/full/10.1146/annurev-psych-010814-015321
  • Griffiths, T. L. (2015). Manifesto for a new (computational) cognitive revolution. Cognition, 135, 21–23.
  • Hauser, D. J., & Schwarz, N. (2015). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 1–8.
  • Lakkaraju, K. (2015a). A Preliminary Study of Daily Sample Composition on Amazon Mechanical Turk. Available at SSRN 2560840. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2560840
  • Lakkaraju, K. (2015b). A Study of Daily Sample Composition on Amazon Mechanical Turk. In Social Computing, Behavioral-Cultural Modeling, and Prediction (pp. 333–338). Springer.
  • Lakkaraju, K., Medina, B., Rogers, A. N., Trumbo, D. M., Speed, A., & McClain, J. T. (2015). The Controlled, Large Online Social Experimentation Platform (CLOSE). In Social Computing, Behavioral-Cultural Modeling, and Prediction (pp. 339–344). Springer.
  • Mason, W. A., & Suri, S. (2011). How to use mechanical turk for cognitive science research. In Proceedings of the 33rd annual conference of the cognitive science society (pp. 66–67).
  • Mason, W., & Suri, S. (2011). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44, 1–23.
  • Mitra, T., Hutto, C. J., & Gilbert, E. (2015). Comparing Person-and Process-centric Strategies for Obtaining Quality Data on Amazon Mechanical Turk. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1345–1354). ACM.
  • Paolacci, G., & Chandler, J. (2014). Inside the Turk Understanding Mechanical Turk as a Participant Pool. Current Directions in Psychological Science, 23, 184–188.
  • Paolacci, G., Chandler, J., & Ipeirotis, P. (2010). Running experiments on amazon mechanical turk. Judgment and Decision Making, 5, 411–419.
  • Paxton, A., Rodriguez, K., & Dale, R. (2015). PsyGlass: Capitalizing on Google Glass for naturalistic data collection. Behavior Research Methods, 1–12.
  • Ranard, B. L., Ha, Y. P., Meisel, Z. F., Asch, D. A., Hill, S. S., Becker, L. B., … Merchant, R. M. (2014). Crowdsourcing—harnessing the masses to advance health and medicine, a systematic review. Journal of General Internal Medicine, 29, 187–203.
  • Reimers, S., & Stewart, N. (2014). Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments. Behavior Research Methods, 1–19.
  • Schubert, T. W., Murteira, C., Collins, E. C., & Lopes, D. (2013). ScriptingRT: A software library for collecting response latencies in online studies of cognition. PloS One, 8, e67769.
  • Simcox, T., & Fiez, J. A. (2013). Collecting response times using Amazon Mechanical Turk and Adobe Flash. Behavior Research Methods, 1–17.
  • Van Bavel, J. J., Rand, D. G., & Phelps, E. A. (n.d.). Restocking Our Subject Pools. Retrieved from https://www.psychologicalscience.org/index.php/publications/observer/2013/november-13/restocking-our-subject-pools.html
  • Yung, A., Cardoso-Leite, P., Dale, G., Bavelier, D., & Green, C. S. (2015). Methods to Test Visual Attention Online. JoVE (Journal of Visualized Experiments), e52470–e52470.