Lambda: An Autograder for Snap!

Michael Ball

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2018-2

January 16, 2018

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-2.pdf

While visual programming languages are hardly a new development, recent pushes for increased equity and access to computer science have led to a renewed interest and use of visual programming languages. Autograders are a critical component of both in-person and online computer science courses. However, until this point, there haven't been any autograders documented for general-purpose, visual programming development environments. This is a particular shortcoming as introductory courses have scaled to larger numbers of students and online environments. While there are challenges to using autograders, we believe that the instant feedback capabilities, as well as potential time savings for course staff will help us teach a greater number of students.

Over the past year, we have built an autograder, named λ (Lambda), for Snap<em>!</em>, a visual blocks-based programming language inspired by MIT's Scratch. The primary motivation for developing the autograder was to run a series of Massive Open Online Courses (MOOCs) on edx.org throughout the 2015-2016 academic year. However, we also wish to use the autograder to better support in-person computer science courses. In Spring 2016, the Snap<em>!</em> autograder was used as a part of UC Berkeley's CS10, <i>_The Beauty and Joy of Computing_</i>, a "CS0" non-majors course.

This report describes λ, which consists of a "backend" Ruby on Rails webserver that allows us to use the autograder in a classroom setting, through a protocol called LTI. The backend web application contains a database of questions and test files, while the Snap<em>!</em> interface contains new features and a view to present the results of the autograder. Our initial results show the autograder successfully being used in CS10, where the autograder was used to supplement oral lab checkoffs, and on edx.org where the autograder was the primary method for students to receive credit for code, graded both for effort and correctness.

Advisors: Dan Garcia

BibTeX citation:

@mastersthesis{Ball:EECS-2018-2,
    Author= {Ball, Michael},
    Editor= {Garcia, Dan},
    Title= {Lambda: An Autograder for Snap!},
    School= {EECS Department, University of California, Berkeley},
    Year= {2018},
    Month= {Jan},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-2.html},
    Number= {UCB/EECS-2018-2},
    Abstract= {While visual programming languages are hardly a new development, recent pushes for increased equity and access to computer science have led to a renewed interest and use of visual programming languages. Autograders are a critical component of both in-person and online computer science courses. However, until this point, there haven't been any autograders documented for general-purpose, visual programming development environments. This is a particular shortcoming as introductory courses have scaled to larger numbers of students and online environments. While there are challenges to using autograders, we believe that the instant feedback capabilities, as well as potential time savings for course staff will help us  teach a greater number of students.

Over the past year, we have built an autograder, named λ (Lambda), for Snap<em>!</em>, a visual blocks-based programming language inspired by MIT's Scratch. The primary motivation for developing the autograder was to run a series of Massive Open Online Courses (MOOCs) on edx.org throughout the 2015-2016 academic year. However, we also wish to use the autograder to better support in-person computer science courses. In Spring 2016, the Snap<em>!</em> autograder was used as a part of UC Berkeley's CS10, <i>_The Beauty and Joy of Computing_</i>, a "CS0" non-majors course.

This report describes λ, which consists of a "backend" Ruby on Rails webserver that allows us to use the autograder in a classroom setting, through a protocol called LTI. The backend web application contains a database of questions and test files, while the Snap<em>!</em> interface contains new features and a view to present the results of the autograder. Our initial results show the autograder successfully being used in CS10, where the autograder was used to supplement oral lab checkoffs, and on edx.org where the autograder was the primary method for students to receive credit for code, graded both for effort and correctness.},
}

EndNote citation:

%0 Thesis
%A Ball, Michael 
%E Garcia, Dan 
%T Lambda: An Autograder for Snap!
%I EECS Department, University of California, Berkeley
%D 2018
%8 January 16
%@ UCB/EECS-2018-2
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-2.html
%F Ball:EECS-2018-2