research-article

Paper vs. Computer-based Exams: A Study of Errors in Recursive Binary Tree Algorithms

Authors:

Scott Grissom,

Laurie Murphy,

Renée McCauley,

Sue FitzgeraldAuthors Info & Claims

SIGCSE '16: Proceedings of the 47th ACM Technical Symposium on Computing Science Education

Pages 6 - 11

https://doi.org/10.1145/2839509.2844587

Published: 17 February 2016 Publication History

Get Access

Abstract

This paper reports on a study of goal-plans and errors produced by students who wrote recursive solutions for a binary tree operation. This work extends a previous study of difficulties CS2 students experienced while writing solutions on paper-based exams. In this study, participants solved the same recursive binary tree problem as part of a hands-on computer-based exam where students had access to an IDE and Java API documentation. Not surprisingly, students who took the computer-based exams were more successful than those who took the paper-based exams (58% vs. 17% correct solutions). However, even with the advantage of access to an IDE, documentation, and test cases, 42% of students taking the computer-based exam still made errors, indicating that students exhibit persistent errors even with support. The most common errors observed included incorrect calculations, missing method calls and missing and incorrect base cases.

References

[1]

Benander, A. C., Benander, B. A., and Pu, H. 1996. Recursion vs. iteration: An empirical study of comprehension. J. of Systems and Software 32, 1, 73--82.

Digital Library

Google Scholar

[2]

Benander, A. C., Benander, B. A., and Sang, J. 2000. An empirical analysis of debugging performance - differences between iterative and recursive constructs. J. of Systems and Software 54, 1, 17--28.

Digital Library

Google Scholar

[3]

Biggs, J. and Collis, K. 1982. Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome). Academic Press, New York, NY.

Google Scholar

[4]

Corney, M., Fitzgerald, S., Hanks, B., Lister, R., McCauley, R., and Murphy, L. 2014. 'Explain in plain English' questions revisited: Data structures problems. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (Atlanta, GA, USA, March 5-8, 2014). SIGCSE '14. ACM, New York, NY, USA, 591--596.

Digital Library

Google Scholar

[5]

Danielsiek, H., Paul, W., and Vahrenhold, J. 2012. Detecting and understanding students' misconceptions related to algorithms and data structures. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (Raleigh, NC, USA, Feb. 29-March 3, 2102). SIGCSE '12. ACM, New York, NY, USA, 21--26.

Digital Library

Google Scholar

[6]

Karpierz, K. and Wolfman, S. A. 2014. Misconceptions and concept inventory questions for binary search trees and hash tables. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (Atlanta, GA, USA, March 5-8, 2014). SIGCSE '14. ACM, New York, NY, USA, 109--114.

Digital Library

Google Scholar

[7]

Kopec, D., Yarmish, G., and Cheung, P. 2007. A description and study of intermediate student programmer errors. SIGCSE Bull. 39, 2, 146--156.

Digital Library

Google Scholar

[8]

Lister, R., Clear, T., Simon, Bouvier, J., Carter, P., Eckerdal, A., Jacková, J., Lopez, M., McCartney, R., Robbins, P., Seppälä, O. and Thompson, E. 2010. Naturally occurring data as research instrument: analyzing examination responses to study the novice programmer. SIGCSE Bull. 41, 4 (January 2010), 156--173.

Digital Library

Google Scholar

[9]

McCauley, R., Hanks, B., Fitzgerald, S., and Murphy,L. Recursion vs. Iteration: An Empirical Study of Comprehension Revisited, in Proceedings of the 49th Technical Symposium on Computer Science Education, (Kansas City, MO, USA, March 4-7, 2015). SIGCSE '15. ACM, New York, NY, USA 350--355.

Digital Library

Google Scholar

[10]

Murphy, L., Fitzgerald, S., Grissom, S. and McCauley, R. 2015. Recursion vs. iteration: An empirical study of comprehension. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (Kansas City, MO, USA, March 4-7, 2015). SIGCSE '15. ACM, New York, NY, USA, 482--487.

Google Scholar

[11]

Rinderknecht, C. 2014. A survey on teaching and learning recursive programming. Informatics in Educ 13, 1, 87--119.

Crossref

Google Scholar

[12]

Spohrer, J. C., Soloway, E., and Pope, E. 1985. A goal/plan analysis of buggy Pascal programs. Human-Computer Interaction 1, 2, 163--207.

Digital Library

Google Scholar

[13]

Tenenberg, J. and Murphy, L. 2005. Knowing what I know: An investigation of undergraduate knowledge and self-knowledge of data structures. Computer Science Education 15, 4, 297--315.

Crossref

Google Scholar

[14]

Yarmish, G. and Kopec, D. 2007. Revisiting novice programmer errors. SIGCSE Bull. 39, 2, 131--137.

Digital Library

Google Scholar

Cited By

View all

Bourke CErez YHazzan ODoyle MStephenson BDorn BSoh LBattestilli L(2023)Executable ExamsProceedings of the 54th ACM Technical Symposium on Computer Science Education V. 110.1145/3545945.3569724(381-387)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3545945.3569724
McMahon CYao BYokota JGarcia DMerkle LDoyle MSheard JSoh LDorn B(2022)Lessons Learned from Asynchronous Online Assessment Formats in CS0 and CS3Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499386(640-646)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499386
VanDeGrift TMerkle LDoyle MSheard JSoh LDorn B(2022)Post-Exam Videos for Assessment in Computing CoursesProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499273(230-236)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499273
Show More Cited By

Index Terms

Paper vs. Computer-based Exams: A Study of Errors in Recursive Binary Tree Algorithms
1. Applied computing
  1. Education

Recommendations

Bug Infestation!: A Goal-Plan Analysis of CS2 Students' Recursive Binary Tree Solutions
SIGCSE '15: Proceedings of the 46th ACM Technical Symposium on Computer Science Education

A goal-plan analysis was conducted to examine the variety of plans students use in writing a recursive method for an operation on a binary search tree. Students were asked to write a recursive method to count the nodes in a binary search tree with ...
Oracle8i Certified Professional DBA Practice Exams
Oracle8i Certified Professional DBA Practice Exams

Comments

Information & Contributors

Information

Published In

SIGCSE '16: Proceedings of the 47th ACM Technical Symposium on Computing Science Education

February 2016

768 pages

ISBN:9781450336857

DOI:10.1145/2839509

General Chairs:
Carl Alphonce
University at Buffalo
,
Jodi Tims
Baldwin Wallace University
,
Program Chairs:
Michael Caspersen
Aarhus University
,
Stephen Edwards
Virginia Tech University

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 February 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCSE '16

Sponsor:

SIGCSE

SIGCSE '16: The 47th ACM Technical Symposium on Computing Science Education

March 2 - 5, 2016

Tennessee, Memphis, USA

Acceptance Rates

SIGCSE '16 Paper Acceptance Rate 105 of 297 submissions, 35%;

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE Virtual 2024

Sponsor:
sigcse

1st ACM Virtual Global Computing Education Conference

December 5 - 8, 2024

Virtual Event , NC , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
405
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)1

Reflects downloads up to 24 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Bourke CErez YHazzan ODoyle MStephenson BDorn BSoh LBattestilli L(2023)Executable ExamsProceedings of the 54th ACM Technical Symposium on Computer Science Education V. 110.1145/3545945.3569724(381-387)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3545945.3569724
McMahon CYao BYokota JGarcia DMerkle LDoyle MSheard JSoh LDorn B(2022)Lessons Learned from Asynchronous Online Assessment Formats in CS0 and CS3Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499386(640-646)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499386
VanDeGrift TMerkle LDoyle MSheard JSoh LDorn B(2022)Post-Exam Videos for Assessment in Computing CoursesProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499273(230-236)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499273
Valentine AVrbik PThomas R(2022)A systematic review of paper-based versus computer-based testing in engineering and computing education2022 IEEE Global Engineering Education Conference (EDUCON)10.1109/EDUCON52537.2022.9766469(364-372)Online publication date: 28-Mar-2022
https://doi.org/10.1109/EDUCON52537.2022.9766469
Fowler MSmith IV DHassan MPoulsen SWest MZilles C(2022)Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication studyComputer Science Education10.1080/08993408.2022.207986632:3(355-383)Online publication date: 10-Jun-2022
https://doi.org/10.1080/08993408.2022.2079866
Hassan MZilles C(2021)Exploring ‘reverse-tracing’ Questions as a Means of Assessing the Tracing Skill on Computer-based CS 1 ExamsProceedings of the 17th ACM Conference on International Computing Education Research10.1145/3446871.3469765(115-126)Online publication date: 16-Aug-2021
https://dl.acm.org/doi/10.1145/3446871.3469765
Zhang LPetersen ALiut MSimion BAlaca FSherriff MMerkle LCutter PMonge ASheard J(2021)A Multi-Course Report on the Experience of Unplanned Online ExamsProceedings of the 52nd ACM Technical Symposium on Computer Science Education10.1145/3408877.3432515(17-23)Online publication date: 3-Mar-2021
https://dl.acm.org/doi/10.1145/3408877.3432515
Corley JStanescu ABaumstark LOrsega MZhang JSherriff MHeckman SCutter PMonge A(2020)Paper Or IDE?Proceedings of the 51st ACM Technical Symposium on Computer Science Education10.1145/3328778.3366857(706-712)Online publication date: 26-Feb-2020
https://dl.acm.org/doi/10.1145/3328778.3366857
Hamouda SEdwards SElmongui HErnst JShaffer C(2020)BTRecurTutor: a tutorial for practicing recursion in binary treesComputer Science Education10.1080/08993408.2020.171453330:2(216-248)Online publication date: 20-Jan-2020
https://doi.org/10.1080/08993408.2020.1714533
Stephenson B(2018)An Experience Using On-Computer Programming Questions During ExamsProceedings of the 23rd Western Canadian Conference on Computing Education10.1145/3209635.3209639(1-6)Online publication date: 4-May-2018
https://dl.acm.org/doi/10.1145/3209635.3209639
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Bug Infestation!: A Goal-Plan Analysis of CS2 Students' Recursive Binary Tree Solutions

Oracle8i Certified Professional DBA Practice Exams

Oracle8i Certified Professional DBA Practice Exams