Table of Contents

Codio Structure and Grading for Facilitators

Ted Blanchard Updated by Ted Blanchard

Overview

In courses that use Codio to provide a coding interface for student practice and assessment, there are several different methods used for grading the student’s responses, and for passing those grades back to the Canvas course. These also interact in various ways with the configuration of Codio being used: specifically, whether Codio itself is used as a coding IDE, or whether it is used to contain and manage another coding environment such as RStudio (for R), OmniDB (for SQL), or Jupyter Notebooks.

General Codio Exercise Concept: Completed vs. Not Completed

In each Codio unit -- whether it is autograded or manually graded (or contains a combination of graded and ungraded exercises) -- the students will need to manually mark themselves Completed when they have finished the unit, either via the “Mark as Completed” option in the Education drop-down menu in the Codio interface, or with the “Mark as Completed” button we have implemented in the unit pages of some courses.

Once the student is marked Completed for a unit, the green checkmark will show up to the left of their name in the Codio Teacher view:

Note that in the second image above, user OD Tester has not been marked Completed, but still has grades and points accruing to that unit; this means that there are autograded exercises in this unit.

Ungraded exercises

This is the simplest configuration in terms of grading, because there is no grading information to pass back to Canvas: Codio is used only to provide a space for students to practice or work out problems that will be uploaded separately. This also applied to “sandbox” Codio windows -- often simply instances of the Terminal -- provided as a space for students to work on coding exercises in order to answer questions in another modality, like a Canvas Quiz or an embedded tool like ExerPy.

Example of a Codio Terminal embedded in conjunction with an ExerPy exercise:

  • FACILITATOR INTERVENTION:  None

Embedded quiz questions (Codio-native)

Can be used either on their own in a Codio unit or combined with other exercises. These use Codio’s built-in question engine, and store and pass student answers automatically. Here is an example with both a “short answer” question type, which requires a student to copy in output from a code response in the terminal, and a multiple-choice question:

(NOTE: the short-answer questions are a bit finicky, and can be thrown off by unexpected line-breaks in the copied text. If a student is having trouble, make certain the text has no extraneous line-breaks).

  • FACILITATOR INTERVENTION:  None, but can check student’s progress/grades in Codio:
    1. In the Instructor view, the student’s results can be seen by clicking the student’s entry on the Assignments page:
    2. On the Student page, you can see the answers, points, question-type, and correct/incorrect information for the student in that unit:

Embedded quiz questions (H5P)

In some courses, quiz questions are embedded on Codio unit pages via the rich-HTML utility H5P. These provide a greater range of display and response capabilities, but do not allow either grading or persistence: that is, a student's response will not be saved if they leave the page and then return, and the scores on the questions are not passed back to Canvas upon unit completion.

H5P question:

  • FACILITATOR INTERVENTION:  None

Autograded exercises

For some courses, the student’s code exercises are automatically checked and graded by autograders implemented in Codio, which provide feedback and give the student a grade based on points/percentage that is passed to Canvas. Note that students still need to mark their units Completed before the grade is registered in the Canvas course. 

EXAMPLE 1:

The student completes a coding exercise in the python file and then clicks “Check It!” to test the code; feedback will appear in the left screen. In this course, the exercise is part of a long sequence in a single unit: we are on page #18. On the final page of the unit, there is a “Mark as Completed” button.

  • FACILITATOR INTERVENTION:  None* directly, but if a student is having trouble understanding the exercise or with the incorrect feedback/response from the autograder, they may need assistance completing the exercise. You can check a student’s completion, grades and points, and whether it has been marked Complete:

(*NOTE: in some units, autograded assignments can be mixed with manually graded assignments, as in the exercises titled “Reflect on Exercise…” in the screenshot above. See below or this article for more information on how to manually grade students on those assignments).

EXAMPLE 2:

The student completes a coding exercise in either a python file (upper right window) or the ipython interpreter (lower right window), and then clicks “Test Code” to check the code; feedback appears in the left window. Wince we are on the second and final page of the unit, the “Mark as Complete” button appears.

  • FACILITATOR INTERVENTION:   None directly, but if a student is having trouble understanding the exercise or with the incorrect feedback/response from the autograder, they may need assistance completing the exercise. You can check a student’s completion, grades and points, and whether it has been marked Complete:

EXAMPLE 3:

In this example, the student writes SQL code in the OmniDB application, which is embedded in a Codio unit; the Codio autograder checks the student’s results, and the response (and correct/incorrect indication) appears below the question. When finished, the student can use the button or the Education menu to mark the unit Completed.

  • FACILITATOR INTERVENTION:   None directly, but if a student is having trouble understanding the exercise or with the incorrect feedback/response from the autograder, they may need assistance completing the exercise. You can check a student’s completion, grades and points, and whether it has been marked Complete:

EXAMPLE 4:

In this example, the student follows instructions to write/modify code in the cells of a Jupyter Notebook embedded in Codio; the student’s response is autograded in Codio:

  • FACILITATOR INTERVENTION:   None directly, but if a student is having trouble understanding the exercise or with the incorrect feedback/response from the autograder, they may need assistance completing the exercise. You can check a student’s completion, grades and points, and whether it has been marked Complete:

Manually graded exercises

  • Native Codio

For exercises in which the students write code directly in Codio (in exercise files), the results are generally autograded; see above for examples. One exception is “Reflect” exercises, which are meant to provide students a way to respond in writing to concepts explored in the exercises. The format is generally like this:

 

See this article for more information on reviewing and grading these exercises.

  • Embedded Tools

For courses which use additional environments embedded within Codio, such as RStudio, OmniDB or Jupyter Notebooks, the units are sometimes set up so that the students complete the exercises and mark the unit Completed, and then the instructor reviews the student’s work and gives them the appropriate grade. The examples for Jupyter Notebooks or OmniDB would look very similar to the ones above in the Autograded Exercises section, with the exception that the students will show no points/grades in Codio until the instructor manually sets them. Here is an example using embedded RStudio:

With manually graded exercises, once the student has marked the unit complete, you need to enter Codio in teacher view, review their work, and assign them the appropriate grade as shown: 

In the above example, there is a Grading Scheme in Codio, as shown in the two options available after the “0” grade field is clicked on. In some cases, there is not a Codio grading scheme; in those cases, the facilitator will need to fill in a (percentage: 0-100) grade manually, based on the student’s work

  • Uploaded Documents

There are some projects which consist of a student working through a code exercise in Codio and/or an embedded tool, submitting a response to an external online evaluation site (e.g., Kaggle), and then uploading the resulting document in Canvas for the facilitator to evaluate. The grading for these exercises takes place entirely within Canvas, and the facilitator should not have to engage Codio at all.

Example of a Kaggle submission exercise:

Appendix: Manual Grading in PAM2070: Big Data for Big Policy Problems

Manual Grading for Labs/Problem Sets in Codio

In order to make sure the Problem Set questions are graded correctly, you need to take the right path to the grading page in order to access the grading rubric that was created for these questions. 

  1. In the Teacher view of the Lab, you will see the students listed. If the student has marked his/her Lab Completed, the green checkmark will appear:

  1. Do not click the student’s name: this will bring you to the Students screen where the standard Codio question-by-question grading functionality is exposed. Since this only allows for Correct/Incorrect response per question, we are not using this approach. 

Instead, follow this process:

    1. First, open the student’s project in Codio by accessing the three-dots menu:

  1. Since the student’s project is Completed, this will actually open two additional browser tabs. The second one, containing the per-question screen, you can ignore and close:

  1. The first one contains the student’s answers to the Lab questions; you’ll need to refer to this during the grading process:

  1. Back on the main Canvas tab, viewing the Lab as a Teacher, click the icon in the “Grade” column of the appropriate students’ row:

  1. This will pop up the “Assign A Grade” box:

  1. Click on “Add Grade”, then click into the grade number box (it will have a “0” in it):

  1. This will pop up the Grading Rubric screen:

  1. Select the correct grading response for each question for this student, referring to his/her answers to the Lab questions in the other browser tab. As you select, the running percentage grade will update on the bottom:

    1. Note that the lowest possible selection, 1, will still give the student points; grading every question “1” will give the student 20% total:

  1. When you have selected the appropriate grade for each question, click “Apply”:

  1. Then, in the “Assign A Grade” window, click “Done:”

If you want to add comments, you can add them in the comments box before clicking Done. These will display to the student in his/her Gradebook.
  1. Click “Close” to close the “Assign A Grade” dialog and return to the Teacher view:

The student’s grade will be updated in the Codio Teacher view for that Lab:

And also in the Canvas Gradebook:

How did we do?

Managing Manually Graded “Reflect and Submit” Codio Exercises

Premade Codio Docs for Ops & Facilitators

Contact