Juho Leinonen
Nea Pirttinen
ABSTRACT
Crowdsourcing refers to the act of using the crowd to create content or to collect feedback on some particular tasks or ideas. Within computer science education, crowdsourcing has been used -- for example -- to create rehearsal questions and programming assignments. As a part of their computer science education, students often learn relational databases as well as working with the databases using SQL statements. In this article, we describe a system for practicing SQL statements. The system uses teacher-provided topics and assignments, augmented with crowdsourced assignments and reviews. We study how students use the system, what sort of feedback students provide to the teacher-generated and crowdsourced assignments, and how practice affects the feedback. Our results suggest that students rate assignments highly, and there are only minor differences between assignments generated by students and assignments generated by the instructor.
- Alberto Abelló, M Elena Rodriguez, Toni Urpi, Xavier Burgués, M José Casany, Carme Martin, and Carme Quer. 2008. LEARN-SQL: Automatic assessment of SQL based on IMS QTI specification. In 2008 Eighth IEEE International Conference on Advanced Learning Technologies. IEEE, 592--593.Google Scholar
Digital Library
- Alireza Ahadi, Vahid Behbood, Arto Vihavainen, Julia Prior, and Raymond Lister. 2016a. Students' syntactic mistakes in writing seven different types of SQL queries and its application to predicting students' success. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education. ACM, 401--406.Google ScholarDigital Library
- Alireza Ahadi, Julia Prior, Vahid Behbood, and Raymond Lister. 2015. A quantitative study of the relative difficulty for novices of writing seven different types of SQL queries. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education. ACM, 201--206.Google ScholarDigital Library
- Alireza Ahadi, Julia Prior, Vahid Behbood, and Raymond Lister. 2016b. Students' semantic mistakes in writing seven different types of SQL queries. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education. ACM, 272--277.Google ScholarDigital Library
- Huda Al-Shuaily and Karen Renaud. 2010. SQL Patterns-A New Approach for Teaching SQL. In 8th HEA Workshop on Teaching, Learning and Assessment of Databases, Abertay-Dundee. 29--40.Google Scholar
- M. Allahbakhsh, B. Benatallah, A. Ignjatovic, H. R. Motahari-Nezhad, E. Bertino, and S. Dustdar. 2013. Quality Control in Crowdsourcing Systems: Issues and Directions. IEEE Internet Computing, Vol. 17, 2 (March 2013), 76--81. https://doi.org/10.1109/MIC.2013.20Google ScholarDigital Library
- Micheal Axelsen, A Faye Borthick, and Paul Bowen. 2001. A model for and the effects of information request ambiguity on end-user query performance. ICIS 2001 Proceedings (2001), 68.Google Scholar
- A Faye Borthick, Paul L Bowen, Donald R Jones, and Michael Hung Kam Tse. 2001. The effects of information request ambiguity and construct incongruence on query development. Decision Support Systems, Vol. 32, 1 (2001), 3--25.Google ScholarDigital Library
- Stefan Brass and Christian Goldberg. 2006. Semantic errors in SQL queries: A quite complete list. Journal of Systems and Software, Vol. 79, 5 (2006), 630--644.Google ScholarDigital Library
- Gretchen I Casterella and Leo Vijayasarathy. 2013. An experimental investigation of complexity in database query formulation tasks. Journal of Information Systems Education, Vol. 24, 3 (2013), 6.Google Scholar
- Seth Chaiklin. 2003. The zone of proximal development in Vygotsky's analysis of learning and instruction. Vygotsky's educational theory in cultural context, Vol. 1 (2003), 39--64.Google Scholar
- Paul Denny, Andrew Luxton-Reilly, and John Hamer. 2008. The PeerWise System of Student Contributed Assessment Questions. In Proceedings of the Tenth Conference on Australasian Computing Education - Volume 78 (ACE '08). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 69--74. http://dl.acm.org/citation.cfm?id=1379249.1379255Google ScholarDigital Library
- Paul Denny, Andrew Luxton-Reilly, John Hamer, and Helen Purchase. 2009b. Coverage of Course Topics in a Student Generated MCQ Repository. In Proceedings of the 14th Annual ACM SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE '09). ACM, New York, NY, USA, 11--15. https://doi.org/10.1145/1562877.1562888Google ScholarDigital Library
- Paul Denny, Andrew Luxton-Reilly, and Beth Simon. 2009a. Quality of student contributed questions using PeerWise. In Proceedings of the Eleventh Australasian Conference on Computing Education-Volume 95. Australian Computer Society, Inc., 55--63.Google ScholarDigital Library
- Paul Denny, Andrew Luxton-Reilly, Ewan Tempero, and Jacob Hendrickx. 2011. CodeWrite: supporting student-driven practice of java. In Proceedings of the 42nd ACM technical symposium on Computer science education. ACM, 471--476.Google ScholarDigital Library
- Olive Jean Dunn. 1961. Multiple comparisons among means. Journal of the American statistical association, Vol. 56, 293 (1961), 52--64.Google ScholarCross Ref
- Masoud I El Agha, Abdallah M Jarghon, and Samy S Abu-Naser. 2018. SQL Tutor for Novice Students. (2018).Google Scholar
- Alan Fekete. 2005. Teaching transaction management with SQL examples. In ACM SIGCSE Bulletin, Vol. 37. ACM, 163--167.Google ScholarDigital Library
- John Hamer, Helen C. Purchase, Paul Denny, and Andrew Luxton-Reilly. 2009. Quality of Peer Assessment in CS1. In Proc. of the 5th International Workshop on Computing Education Research Workshop (ICER '09). ACM, New York, NY, USA, 27--36.Google ScholarDigital Library
- Joseph E. Hollingsworth. 2008. Teaching Query Writing: An Informed Instruction Approach. SIGCSE Bull., Vol. 40, 3 (June 2008), 351. https://doi.org/10.1145/1597849.1384393Google Scholar
- Jeff Howe. 2006. The Rise of Crowdsourcing. Wired magazine, Vol. 14, 6 (2006), 1--4.Google Scholar
- Alison Hull and Benedict du Boulay. 2015. Motivational and metacognitive feedback in SQL-Tutor. Computer Science Education, Vol. 25, 2 (2015), 238--256.Google ScholarCross Ref
- Claire Kenny and Claus Pahl. 2005. Automated tutoring for a database skills training environment. Vol. 37. ACM.Google Scholar
- H Laine. 2001. SQL-trainer. In Proceedings of Kolin Kolistelut/Koli Calling-First Annual Baltic Conference on Computer Science Education, Report A-2002, Vol. 1. 13--17.Google Scholar
- Robert L Leitheiser and Salvatore T March. 1996. The influence of database structure representation on database system learning and use. Journal of Management Information Systems, Vol. 12, 4 (1996), 187--213.Google ScholarDigital Library
- Roman Lukyanenko, Jeffrey Parsons, and Yolanda F. Wiersma. 2014. The IQ of the Crowd: Understanding and Improving Information Quality in Structured User-Generated Content. Information Systems Research, Vol. 25, 4 (2014), 669--689. https://doi.org/10.1287/isre.2014.0537Google ScholarDigital Library
- Raina Mason, Carolyn Seton, and Graham Cooper. 2016. Applying cognitive load theory to the redesign of a conventional database systems course. Computer Science Education, Vol. 26, 1 (2016), 68--87.Google ScholarCross Ref
- Antonija Mitrovic. 1998. A knowledge-based teaching system for SQL. In Proceedings of ED-MEDIA, Vol. 98. 1027--1032.Google Scholar
- Antonija Mitrovic. 2003. An intelligent SQL tutor on the web. International Journal of Artificial Intelligence in Education, Vol. 13, 2--4 (2003), 173--197.Google ScholarDigital Library
- Claus Pahl, Ronan Barrett, and Claire Kenny. 2004. Supporting active database learning and training through interactive multimedia. ACM SIGCSE Bulletin, Vol. 36, 3 (2004), 27--31.Google ScholarDigital Library
- Nea Pirttinen, Vilma Kangas, Irene Nikkarinen, Henrik Nygren, Juho Leinonen, and Arto Hellas. 2018a. Crowdsourcing Programming Assignments with CrowdSorcerer. In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE 2018). ACM, New York, NY, USA, 326--331. https://doi.org/10.1145/3197091.3197117Google ScholarDigital Library
- Nea Pirttinen, Vilma Kangas, Henrik Nygren, Juho Leinonen, and Arto Hellas. 2018b. Analysis of Students' Peer Reviews to Crowdsourced Programming Assignments. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (Koli Calling 2018). ACM, New York, NY, USA, 5.Google ScholarDigital Library
- Gang Qian. 2018. Teaching SQL: a divide-and-conquer method for writing queries. Journal of Computing Sciences in Colleges, Vol. 33, 4 (2018), 37--44.Google ScholarDigital Library
- Catherine Roster, Gerald Albaum, and Robert Rogers. 2006. Can cross-national/cultural studies presume etic equivalency in respondents' use of extreme categories of Likert rating scales? International Journal of Market Research, Vol. 48, 6 (2006), 741--759.Google ScholarCross Ref
- Kate Sanders, Marzieh Ahmadzadeh, Tony Clear, Stephen H. Edwards, Mikey Goldweber, Chris Johnson, Raymond Lister, Robert McCartney, Elizabeth Patitsas, and Jaime Spacco. 2013. The Canterbury QuestionBank: Building a Repository of Multiple-choice CS1 and CS2 Questions. In Proceedings of the ITiCSE Working Group Reports Conference on Innovation and Technology in Computer Science Education-working Group Reports (ITiCSE -WGR '13). ACM, New York, NY, USA, 33--52. https://doi.org/10.1145/2543882.2543885Google ScholarDigital Library
- Clifford A. Shaffer, Ville Karavirta, Ari Korhonen, and Thomas L. Naps. 2011. OpenDSA: Beginning a Community active-eBook Project. In Proceedings of the 11th Koli Calling International Conference on Computing Education Research (Koli Calling '11). ACM, New York, NY, USA, 112--117. https://doi.org/10.1145/2094131.2094154Google Scholar
- John B Smelcer. 1995. User errors in database query composition. International Journal of Human-Computer Studies, Vol. 42, 4 (1995), 353--381.Google ScholarDigital Library
- Josep Soler, Ferran Prados, Imma Boada, and Jordi Poch. 2006. A Web-based tool for teaching and learning SQL. In International Conference on Information Technology Based Higher Education and Training, ITHET .Google Scholar
- Lovisa Sundin and Quintin Cutts. 2019. Is it feasible to teach query programming in three different languages in a single session? A study on a pattern-oriented tutorial and cheat sheets. In Proceedings of the 1st UK & Ireland Computing Education Research Conference. 1--7.Google ScholarDigital Library
- Toni Taipalus. 2019. A Notation for Planning SQL Queries. Journal of Information Systems Education, Vol. 30, 3 (2019), 2.Google Scholar
- Toni Taipalus and Piia Perala. 2019. What to Expect and What to Focus on in SQL Query Teaching. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. ACM, 198--203.Google ScholarDigital Library
- Toni Taipalus, Mikko Siponen, and Tero Vartiainen. 2018. Errors and complications in SQL query formulation. ACM Transactions on Computing Education (TOCE), Vol. 18, 3 (2018), 15.Google Scholar
Index Terms
- Crowdsourcing Content Creation for SQL Practice
Recommendations
Crowdsourcing programming assignments with CrowdSorcerer
ITiCSE 2018: Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science EducationSmall automatically assessed programming assignments are an often used resource for learning programming. Creating sufficiently large amounts of such assignments is, however, time consuming. As a consequence, offering large quantities of practice ...
Comparing NoSQL MongoDB to an SQL DB
ACMSE '13: Proceedings of the 51st ACM Southeast ConferenceNoSQL database solutions are becoming more and more prevalent in a world currently dominated by SQL relational databases. NoSQL databases were designed to provide database solutions for large volumes of data that is not structured. However, the ...
SQL: From Traditional Databases to Big Data
SIGCSE '16: Proceedings of the 47th ACM Technical Symposium on Computing Science EducationThe Structured Query Language (SQL) is the main programing language designed to manage data stored in database systems. While SQL was initially used only with relational database management systems (RDBMS), its use has been significantly extended with ...
Comments