research-article

What are we thinking when we grade programs?

Authors:

Laurie MurphyAuthors Info & Claims

SIGCSE '13: Proceeding of the 44th ACM technical symposium on Computer science education

Pages 471 - 476

https://doi.org/10.1145/2445196.2445339

Published: 06 March 2013 Publication History

Get Access

Abstract

This paper reports on a mixed methods study which examines how four experienced instructors approached the grading of a programming problem. Two instructors used a detailed, analytic approach and two instructors employed a holistic approach. One instructor exhibited elements of a primary trait approach. Even though the four instructors used different grading scales and philosophies, their raw scores were highly correlated (Spearman's rho of .81) supporting the conclusion that experienced instructors usually agree on whether a program is 'very good' or 'very bad'. Clearly there is no single right way to grade programs. Further discourse should be encouraged for the benefit of both educators and students.

References

[1]

Ala-Mutka, K. 2005. A survey of automated assessment approaches for programming assignments. Computer Science Education 15, 2 (June 2005), 83--102.

Crossref

Google Scholar

[2]

Becker, K. 2003. Grading programming assignments using rubrics. In Proceedings of the 8th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE '03), David Finkel (Ed.). ACM, New York, NY, USA, 253--253. DOI=10.1145/961511.961613 http://doi.acm.org/10.1145/961511.961613.

Digital Library

Google Scholar

[3]

Carter, J., Ala-Mutka, K., Fuller, U., Dick, M., English, J., Fone, W. and Sheard, J. 2003. How shall we assess this?. SIGCSE Bull. 35, 4 (June 2003), 107--123. DOI=10.1145/960492.960539 http://doi.acm.org/10.1145/960492.960539.

Digital Library

Google Scholar

[4]

Douce, C., Livingstone, D., and Orwell, J. 2006. Automatic test-based assessment of programming: A review. ACM Journal of Educational Resources in Computing 5, 3 (September 2005).

Digital Library

Google Scholar

[5]

Hamm, R. W., Henderson, K. D., Repsher, M. L. and Timmer, K. L. 1983. A tool for program grading: The Jacksonville University scale. In Proceedings of the 14th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '83). ACM, New York, NY, USA, 248--252. DOI=10.1145/800038.801059 http://doi.acm.org/10.1145/800038.801059.

Digital Library

Google Scholar

[6]

Howatt, J. 1994. On criteria for grading student programs. SIGCSE Bull. 26, 3 (September 1994), 3--7. DOI=10.1145/187387.187389 http://doi.acm.org/10.1145/187387.187389.

Digital Library

Google Scholar

[7]

Loacker, G., Cromwell, C. and O'Brien, K. 1986. Assessment in higher education: To serve the learner. In Adelman, C. (ed.), Assessment in American Higher Education. U.S. Department of Education, Office of Educational Research and Improvement, Washington, DC, 47--62.

Google Scholar

[8]

Miller, N. E. and Peterson, C. G. 1980. A method for evaluating student written computer programs in an undergraduate computer science programming language course. SIGCSE Bull. 12, 4 (December 1980), 9--17. DOI=10.1145/989274.989276 http://doi.acm.org/10.1145/989274.989276.

Digital Library

Google Scholar

[9]

Murphy, L. McCauley, R. and Fitzgerald, S. 2012. 'Explain in plain English' questions: Implications for teaching. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (SIGCSE '12). ACM, New York, NY, USA, 385--390.

Digital Library

Google Scholar

[10]

Olson, D. 1988. The reliability of analytic and holistic methods in rating students' computer programs. SIGCSE Bull. 20, 1 (February 1988), 293--298. DOI=10.1145/52965.53037 http://doi.acm.org/10.1145/52965.53037.

Digital Library

Google Scholar

[11]

Smith, L. and Cordova, J. 2005. Weighted primary trait analysis for computer program evaluation. J. Comput. Small Coll. 20, 6 (June 2005), 14--19. http://dl.acm.org/citation.cfm?id= 1060405.1060409&coll=DL&dl=ACM&CFID=106820723&CFTOKEN=10091255.

Digital Library

Google Scholar

[12]

Taggart, G., Phifer, S. and Nixon, J. 2001. In Wood, M. (ed.), Rubrics: A Handbook for Construction and Use. Scarecrow Press, Latham, MD, USA.

Google Scholar

Cited By

View all

Hickman HBell T(2024)Automated Assessment: Does It Align With Teachers' Views?Proceedings of the 19th WiPSCE Conference on Primary and Secondary Computing Education Research10.1145/3677619.3678113(1-10)Online publication date: 16-Sep-2024
https://dl.acm.org/doi/10.1145/3677619.3678113
Hickman HBell T(2023)Towards Automated Assessment of High School Programming2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343243(1-9)Online publication date: 18-Oct-2023
https://doi.org/10.1109/FIE58773.2023.10343243
Alves NGresse von Wangenheim CHauck JBorgatto A(2022)Automating the Assessment of Algorithms and Programming Concepts in App Inventor Projects in Middle SchoolResearch Anthology on Computational Thinking, Programming, and Robotics in the Classroom10.4018/978-1-6684-2411-7.ch025(524-549)Online publication date: 2022
https://doi.org/10.4018/978-1-6684-2411-7.ch025
Show More Cited By

Index Terms

What are we thinking when we grade programs?
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education

Recommendations

Adaptive Rubrics
SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education

Grading is a notoriously difficult and time-consuming part of teaching. For open-ended programming, mathematical, or design problems, assigning consistent scores and giving useful feedback can be very challenging. Large classes compound this difficulty. ...
Improving pedagogical feedback and objective grading
SIGCSE 08

It is important for learning that students receive enough of educational feedback of their work. To get the students to be seriously disposed to the feedback it has to be personal, objective and consistent. In large classes ensuring such feedback can be ...
Utilization of Rubrics for Self-assessment of Computer Science Students Enrolled in a Research Writing Course
WCCCE '17: Proceedings of the 22nd Western Canadian Conference on Computing Education

This study 1 determined the utilization of rubrics in self-assessment of Computer Science students who underwent research writing course. Two sections with a total of 69 students served as participants of the study. The grades of the teacher in the ...

Comments

Information & Contributors

Information

Published In

SIGCSE '13: Proceeding of the 44th ACM technical symposium on Computer science education

March 2013

818 pages

ISBN:9781450318686

DOI:10.1145/2445196

Conference Chairs:
Tracy Camp
Colorado School of Mines
,
Paul Tymann
Rochester Institute of Technology
,
Program Chairs:
J. D. Dougherty
Haverford College
,
Kris Nagel
Georgia Gwinnett College

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 March 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCSE '13

Sponsor:

SIGCSE

SIGCSE '13: The 44th ACM Technical Symposium on Computer Science Education

March 6 - 9, 2013

Colorado, Denver, USA

Acceptance Rates

SIGCSE '13 Paper Acceptance Rate 111 of 293 submissions, 38%;

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE Virtual 2024

Sponsor:
sigcse

1st ACM Virtual Global Computing Education Conference

December 5 - 8, 2024

Virtual Event , NC , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
471
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Hickman HBell T(2024)Automated Assessment: Does It Align With Teachers' Views?Proceedings of the 19th WiPSCE Conference on Primary and Secondary Computing Education Research10.1145/3677619.3678113(1-10)Online publication date: 16-Sep-2024
https://dl.acm.org/doi/10.1145/3677619.3678113
Hickman HBell T(2023)Towards Automated Assessment of High School Programming2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343243(1-9)Online publication date: 18-Oct-2023
https://doi.org/10.1109/FIE58773.2023.10343243
Alves NGresse von Wangenheim CHauck JBorgatto A(2022)Automating the Assessment of Algorithms and Programming Concepts in App Inventor Projects in Middle SchoolResearch Anthology on Computational Thinking, Programming, and Robotics in the Classroom10.4018/978-1-6684-2411-7.ch025(524-549)Online publication date: 2022
https://doi.org/10.4018/978-1-6684-2411-7.ch025
Marin VHosseini HRivero C(2021)Customizing Feedback for Introductory Programming Courses Using Semantic ClustersIntelligent Tutoring Systems10.1007/978-3-030-80421-3_30(279-285)Online publication date: 9-Jul-2021
https://doi.org/10.1007/978-3-030-80421-3_30
Alves NGresse von Wangenheim CHauck JBorgatto A(2020)Automating the Assessment of Algorithms and Programming Concepts in App Inventor Projects in Middle SchoolHandbook of Research on Tools for Teaching Computational Thinking in P-12 Education10.4018/978-1-7998-4576-8.ch004(76-102)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-4576-8.ch004
Bian WAlam OKienzle JSyriani ESahraoui H(2020)Is automated grading of models effective?Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems10.1145/3365438.3410944(365-376)Online publication date: 16-Oct-2020
https://dl.acm.org/doi/10.1145/3365438.3410944
Carmosino MMinnes MZhang JSherriff MHeckman SCutter PMonge A(2020)Adaptive RubricsProceedings of the 51st ACM Technical Symposium on Computer Science Education10.1145/3328778.3366946(549-555)Online publication date: 26-Feb-2020
https://dl.acm.org/doi/10.1145/3328778.3366946
Costantini ULonati VMorpurgo AZhang JSherriff MHeckman SCutter PMonge A(2020)How Plans Occur in Novices' ProgramsProceedings of the 51st ACM Technical Symposium on Computer Science Education10.1145/3328778.3366870(852-858)Online publication date: 26-Feb-2020
https://dl.acm.org/doi/10.1145/3328778.3366870
Marin VRivero CZhu WTao DCheng XCui PRundensteiner ECarmel DHe QXu Yu J(2019)Clustering Recurrent and Semantically Cohesive Program Statements in Introductory Programming AssignmentsProceedings of the 28th ACM International Conference on Information and Knowledge Management10.1145/3357384.3357960(911-920)Online publication date: 3-Nov-2019
https://dl.acm.org/doi/10.1145/3357384.3357960
Lancaster TRobins AFincher S(2019)Assessment and PlagiarismThe Cambridge Handbook of Computing Education Research10.1017/9781108654555.015(414-444)Online publication date: 15-Feb-2019
https://doi.org/10.1017/9781108654555.015
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Adaptive Rubrics

Improving pedagogical feedback and objective grading

Utilization of Rubrics for Self-assessment of Computer Science Students Enrolled in a Research Writing Course