Table of Contents

Behavioral Insights: ITG Reference & Setup Guide

Jason Carroll Updated by Jason Carroll

Overview & Purpose

Behavioral Insights (BI) is a Codio feature that surfaces student behavior patterns during coding assignments. It's useful for two distinct things:

  1. Technical investigation - spotting patterns that point to assignment, code, or platform issues.
  2. Instructional support - flagging potential plagiarism or student struggle so facilitators know where to look closer.
Behavioral Insightst surfaces patterns; a human still has to interpret them (see Using BI for plagiarism review below).

Resources

Codio documentation
Canvas course: Codio Behavioral Insights Test

Where Behavioral Insights Is (and Isn't) Available

BI tracks keystroke-level activity inside the native Codio IDE. It does not reliably track activity in:

  • JupyterLab
  • CodioStudio / RStudio

When a course or unit uses Jupyter or CodioStudio instead of the base Codio IDE, BI either shows no data or throws a false-positive flag (red/orange indicator) - because the absence of tracked keystrokes looks, to the system, like the absence of work. See False positives below.

Only turn BI on for courses/units that use the base Codio ID. Turning it on for Jupyter- or CodioStudio-based units will generate noise, not signal.

Course-level vs. unit-level

BI is enabled at the course level, not per unit. Some courses are "split" - part of the course uses the base Codio IDE, part uses Jupyter, within the same course. In those cases, turning BI on still applies it course-wide, so be aware it'll surface false positives for the Jupyter/CodioStudio portions even if the IDE portions are clean.

Coverage matrix

There's a shared spreadsheet showing, by district, which courses currently have BI turned on (highlighted green).
View the BI coverage matrix

Check this matrix before enabling BI for a new district/course - and update it when you toggle BI for a course.

How to Turn On Behavioral Insights

In Codio, for a given course:

  1. In the left-side nav, under Grading, click Basic Settings.
  2. Under Behavioral Insights Options, toggle on Enable Behavioral Insights.
  3. (Optional) Adjust threshold options in the same section if defaults don't fit the course.

Remember: this is a course-level toggle. Confirm the course (or the relevant portion of it) uses the base Codio IDE before turning it on - see Where BI Is Available above.

How to Access Behavioral Insights (Facilitator View)

This is what facilitators will see once BI is on - useful for ITG to know so we can troubleshoot or walk someone through it.

  1. Navigate to the course in Canvas.
  2. Open the Overview tab in the left-side nav.
  3. Go to the Progress tab.
  4. Locate the Behavior column.
  5. Click the indicator next to a student's entry to open the BI panel for that student.

Figure 1 - Accessing Behavioral Insights from the Canvas Progress tab

Indicators and thresholds

The Behavior column uses a four-color indicator system based on configurable thresholds (configured at the course level):

Figure - Behavior column indicators: green (no thresholds broken), yellow/orange (one or more thresholds approaching or broken), red (thresholds clearly broken)

An empty/green indicator means no configured thresholds were broken. When thresholds are broken, the affected tiles inside the BI dashboard get a red outline, dashed threshold lines, and explanatory text under each chart.

Per Codio's documentation: low time spent combined with a high edit rate can indicate potential plagiarism concerns, while high time spent combined with a very low edit rate can indicate possible student struggle.

Example 1 - Honest Work (No Plagiarism Detected)

Assignment: BI - Javascript/Non-Plagerism Honest Example (For Canvas)

This example shows a normal, honest attempt - no plagiarism indicators, in the native Codio IDE.

BI metrics

Figure 2 - Time Spent and Rate of Edits, honest-work example

Time spent: 4 minutes 7 seconds - short, but plausible for a simple exercise. Rate of edits: 1.55 characters per second - a normal typing range, not suggestive of a large block being pasted in at once.

Together, these don't raise flags. They suggest active writing and editing, not a one-time drop-in of a finished solution.

Coding behavior breakdown

Figure 3 - Coding vs. debugging time, external paste history, insertions vs. deletions

The coding-vs-debugging split looks realistic. The History of External Pastes chart is empty - an important signal, since there's no recorded external paste event at all. Insertions vs. deletions also looks like normal iterative work (add code, then revise).

Figure 4 - External paste tile text and link to the behavioral player

From this tile, click "Click here" to open the behavioral player - a time-based replay of the student's work. In this honest example there's no suspicious paste pattern, but the player is still useful to verify the work was built progressively.

Behavioral player / code playback

Figure 5 - Behavioral player: code playback and activity timeline

Key rows/controls in the player:

  • File name - which file was being changed over time.
  • Pastes - where paste events occurred over time.
  • User events - actions like clicking "Check It" (test-code button).
  • Timeline - date/time of activity.
  • Playback controls (lower-left) - step backward, play step-by-step, step forward.

The top code pane shows the contents of the selected file; the timeline beneath shows how the work unfolded. For an honest attempt, this helps confirm the code appeared gradually and testing happened at sensible points.

BI correctly shows this submission as normal work - no suspicious paste activity, no flags, believable code-and-test progression.

Example 2 - Plagiarism Pattern

Assignment: BI - Javascript/Plagerism Example (For Canvas)

Strong indicators of copied work.

Figure 6 - Red behavior indicator; threshold exceeded, flagged for review

BI metrics

Figure 7 - Time Spent: 32 seconds. Rate of Edits: 78.91 cps

Both metrics fall in the red (flagged) zone. Time spent of 32 seconds is, per Codio's own explanatory text, "uncharacteristic of thoughtful programming work" (though this might be normal if the student uploaded prior work). A rate of 78.91 characters per second is, per Codio, "uncharacteristically fast for thoughtful work" and could indicate transcribing a solution or a large paste.

External paste evidence

Figure 8 - Coding vs. debugging (100% coding, no debug time), 22 pasted lines, heavy insertions

The History of External Pastes chart shows 22 pasted lines in a single event - unlike the honest example, where this chart was empty. Combined with near-zero debug time and the gauge readings above, this is a strong combined signal.

Using BI for Plagiarism Review

What it tells you: BI flags students whose time-spent and edit-rate metrics fall outside expected ranges for the assignment. It is based only on data collected within the Codio environment for that specific unit.

What it doesn't tell you: It can't distinguish "pasted from another source dishonestly" from legitimate reasons a student's activity might look unusual, including:

  • The student transferred cohorts or had their enrollment moved (e.g., a New York-to-another-state transfer), and pasted in work they'd already completed locally before the transfer.
  • The student is retaking a unit and reused prior, legitimately-completed work.
  • The student wrote code in an external editor (VS Code, etc.) and pasted it in - which may be common or even expected practice in some courses.

Recommended approach before flagging a student:

  1. Treat a BI flag as a prompt to look closer, not a conclusion.
  2. Check the behavioral player to see the actual paste/edit pattern.
  3. Consider whether the student could plausibly have transferred cohorts, retaken the unit, or worked in an external IDE before concluding anything.
  4. If raising this with a facilitator or in documentation/examples, anonymize identifying details (e.g., blur/redact student names in screenshots).
Remember this opens a broader policy question - whether and how to proactively look for plagiarism, and what process follows a flag - that's a separate discussion from BI mechanics, and not something this doc resolves.

False Positives: JupyterLab & RStudio

BI's metrics rely on tracking keystrokes inside the native Codio IDE. When a unit uses JupyterLab or CodioStudio (RStudio) instead, that tracking doesn't capture activity the same way - so BI can show a red or orange flag on work that was actually done legitimately, simply because the data wasn't captured.

Example 3 - JupyterLab (False Positive)

Figure 9 - Red indicator triggered on a JupyterLab assignment

Figure 10 - Missing Edit Data

Figure 11 — There is no data to display" for coding/debugging time and insertions/deletions; explanatory text notes "Codio IDE buttons were not used to run code

Metrics are incomplete; behavior was legitimate.

Example 4 - RStudio / CodioStudio (False Positive)

Figure 12 - Orange indicator on the Progress tab for an RStudio-based assignment

Figure 13 - Time Spent: 4m 23s (normal range). Rate of Edits: 0.00 cps

Time spent here looks normal, but rate of edits reads 0.00 cps - not because the student did nothing, but because CodioStudio/RStudio activity isn't captured by BI's edit-tracking the same way the base IDE is.

Figure 13 - "There is no data to display" panels for coding/debugging time and insertions/deletions on an RStudio assignment

Any red or orange flag on a Jupyter- or CodioStudio-based unit should be treated as a likely false positive first, not investigated as potential plagiarism, unless there's separate evidence. This is exactly why BI should generally stay off for these unit types until Codio's roadmap work changes how tracking works there.

Behavior Comparison Table

Scenario Time Edit Behavior Interpretation
Honest (native IDE) Normal Gradual edits Independent work
Plagiarism (native IDE) Very low Extremely high cps Likely pasted
JupyterLab Low No data captured False positive
RStudio / CodioStudio Normal No data captured False positive

How did we do?

Contact