Table of Contents

Creating NbGrader Assignments

Brock Schmutzler Updated by Brock Schmutzler

You can perform auto-grading in Jupyter Notebooks using a JupyterLab extension called NbGrader (i.e., nbgrader). This article is a complement to Using the JupyterLab Starter Pack, which goes into more detail about creating Codio assignments that host JupyterLab.

If you are not using NbGrader, you can remove the .codio-jupyter and nbgrader_config.py files. This is not necessary, but the advantage is that publishing your Codio assignment will take less time because the absence of a .codio-jupyter file tells Codio to bypass the NbGrader processing that happens in the background when you publish an assignment with the .codio-jupyter file in /home/codio/workspace. The process for creating an ungraded activity in Canvas in the similar to the one described in the Ungraded Activities in Canvas section of Using the RStudio Starter Pack.

Codio Assignment

To create an NbGrader assignment in Codio:

  1. Add a Jupyter Notebook (see Adding Jupyter Notebooks) that contains all the "solution code" needed to successfully run all the cells in the entire notebook
  2. Put ### BEGIN SOLUTION and ### END SOLUTION delimiters around the "solution code" you want students to provide when they work through the notebook in Canvas
    Include ### BEGIN SOLUTION and ### END SOLUTION delimiters around solution code
  3. In Preview mode, select Nbgrader > Create assignment from the JupyterLab menu
    Select "NbGrader > Create assignment" from JupyterLab menu
    Create assignment right sidebar
  4. In the right sidebar that appears, specify each cell as either "Autograded answer" type or "Read-only" type:
    1. "Autograded answer" type for each cell with ### BEGIN SOLUTION and ### END SOLUTION
    2. "Read-only" type for all other cells (to prevent students from modifying or deleting the cell)
      Specify cells as "Autograded answer" or "Read-only
    3. Below each "Autograded answer" cell you want to test, add a Code cell of "Autograded tests" type — make each "Autograded tests" cell worth one point even if there are multiple tests in the cell. NOTE: If you want some (or all) tests to be hidden from students in Canvas, put the code for those tests between ### BEGIN HIDDEN TESTS and ### END HIDDEN TESTS delimiters (similar to putting solution code between ### BEGIN SOLUTION and ### END SOLUTION delimiters).
      Add Code cell of type "Autograded tests"
  5. Unselect View > Appearance > Show Right Sidebar to close "Create assignment" sidebar (or press command+J)
    Unselect "View > Appearance > Show Right Sidebar"
  6. Save and checkpoint your changes by clicking the disk icon in the upper-left corner of your notebook tab (or press command+S)
    Save and checkpoint your changes
  7. Check that all your tests successfully pass by clicking the Validate button in the top menu bar of your notebook tab
    Click Validate button to check that all NbGrader tests pass
    Successful result of the Validate button
  8. Publish your assignment by selecting Education > Publish Assignment from the top Codio menu
    Publish your Codio assignment after creating NbGrader assignment
You can also check your NbGrader assignment by selecting Education > Validate Jupyter from the Codio menu — this runs a Codio script that returns more detailed information in the case when validation fails.
Sometimes the Validate button in Jupyter or the Education > Validate Jupyter script in the Codio menu will indicate that there is something wrong with your NbGrader assignment even though everything is fine. If you believe there is nothing wrong with your NbGrader assignment, try publishing the Codio unit. If the Codio unit publishes successfully, then all should be good.
For more details, see the NbGrader documentation and Codio's documentation on NbGrader.
If a cell has an NbGrader type of "Autograded answer" and none of the code in that cell is wrapped with ### BEGIN SOLUTION ... ### END SOLUTION delimiters, then the entire contents of the cell will be replaced with ### YOUR CODE HERE ... ### END OF YOUR CODE.
If ### BEGIN SOLUTION ... ### END SOLUTION appears in a cell that is not marked as an "Autograded answer" cell, you will receive a publishing error.

Canvas Assignment

To set up your Codio assignment in Canvas:

  1. Create a new Canvas assignment page
    Create Canvas assignment
  2. Configure assignment settings
    1. Use External Tool submission
    2. Create Codio Projects group
    3. Display grade as Percentage
      Canvas assignment external tool submissioin
  3. Configure Codio assignment settings (make sure to Save Changes for each setting)
    1. Read Only with Resubmit
      Read-only with resubmit
    2. ASSESSMENTS GRADING
      Assessments grading
    3. ASSESSMENTS GRADING and TEACHERS GRADING (does not have to be 50/50)
      Assessments grading and Teachers grading
  4. Test in Student View
    1. Enter solution code between # YOUR CODE HERE and # END OF YOUR CODE (delete the stop("Your code is missing.") command)
      Enter solution code in Jupyter Notebook
    2. Save your changes by pressing command+S
    3. Go to the Education menu and select Mark as Completed
    4. Check that Codio's student feedback report has the expected output
      Codio's student feedback form
      Student's answer in Codio's feedback report
      Test cell in Codio's student feedback report

Ungraded Assignments

You can also use NbGrader to create ungraded assignments. One advantage of doing this is that NbGrader provides an easy way to control whether a cell can be edited or deleted by simply changing the cell type. The steps are similar to creating a graded assignment with the following tweaks.

  1. Modify all the "Autograded tests" cells in the notebook in one of two ways:
    1. Make all "Autograded tests" cells worth zero points (with no hidden tests) in the notebook
    2. Change all "Autograded tests" cells to "Read-only" cells in the notebook
  2. Toggle off all the grading settings (i.e., Teachers, Assessments, Script) for the Codio unit

In Canvas, embed the Codio unit in a Canvas page using the Rich Content Editor (as opposed to creating a Canvas assignment), as described in the Ungraded Canvas Assignments section of Using the RStudio Starter Pack.

How did we do?

Jupyter Notebooks Style Guide

Adding Extensions to Jupyter Notebooks

Contact