Table of Contents

Technical Student Experience (Tech SE) Review Process

Jason Carroll Updated by Jason Carroll

Overview

What is the Tech SE Review?

The Technical Student Experience (Tech SE) Review replaces the former Alpha testing process as the first limited-public review stage in eCornell’s product development pipeline.

Its goal is to simulate a real student experience—especially for STEM and technically complex courses—before launch, validating both instructional flow and technical integrity.

While the original Alpha process involved multiple external testers and extended timeframes, the new Tech SE Review is a streamlined, single-tester model designed to provide faster turnaround, higher-quality feedback, and reduced coordination overhead. 

NOTE: Optional: Add a 2nd for a supplementary run if justified.

This review ensures:

  • The course runs smoothly from both a student and facilitator perspective.
  • Grading structures and submission workflows function as intended.
  • Content is instructionally and technically sound before final QA and faculty sign-off.

When or why do we run a Tech SE Review?

The Tech SE Review is run for:

  • STEM and technical courses, especially those using ITG-maintained integrations (Codio, Jupyter Notebooks, etc.) and focusing on coding.
  • Any course with custom Canvas components or unique grading logic.
  • Pilot courses or significant course revisions where a live student test is valuable.

It is not mandatory for every course, but recommended for any technical or coding-intensive project.

Who is involved in the Tech SE Review?

Role

Responsibilities

Tech SE Review Coordinator (ITG)

Coordinates review logistics, prepares Canvas dev shell, communicates with tester, ensures BugHerd and course access setup.

Instructional Designer (ID)

Provides context, tracks review status, and implements instructional fixes.

Technical Lead / ITG

Addresses bugs, grading issues, or system errors discovered during testing.

Program Facilitation Manager, STEM (Justin)

Identifies and assigns one qualified technical facilitator to serve as the reviewer.

Tech SE Tester (Facilitator)

Acts as both student and instructor to test the course experience from both sides. (See Canvas Role Setup section below for technical access details, including dual enrollment and “Act As” use.)

Product Manager

Reviews timing and ensures process documentation and Wrike plan updates.

Technical Facilitator Expectations

Facilitators serving as Tech SE testers play a vital role in validating both the student and facilitator experience before the course launches. Their feedback ensures that future facilitators have clear, complete, and accurate guidance when the course goes live.

Facilitators should:

  • Complete the course as a student, testing all assignments, submissions, and grading rubrics.
  • Switch to their instructor role to confirm grades, rubrics, and facilitator tools work as expected.
  • Log all issues in BugHerd, including any content, tool, or navigation problems.
  • Email the Tech SE Review Coordinator directly for any access issues, technical blockers, or broken tool integrations.

Providing Feedback for Facilitator Notes

One of the most valuable outcomes of the Tech SE Review is refining the Facilitator Notes used by future instructors. The tester’s unique perspective as both a student and facilitator helps identify areas where guidance can be clarified or expanded.

During the review, testers should specifically note:

  • Potential student pain points (e.g., confusing instructions, unclear deliverables, or tricky grading elements).
  • Missing or ambiguous facilitator guidance (e.g., steps for grading Codio submissions, timing for discussions, solution code, or expected rubric interpretation).
  • Helpful “teaching tips” or insights that would aid future facilitators in running the course smoothly.
  • Any logistical details—file links, tool quirks, or grading workflow nuances—that should be captured permanently.

Facilitator Notes updates will be compiled after the review. The Tech SE Review Coordinator and the assigned Instructional Designer will review tester comments in BugHerd or email and incorporate the most relevant suggestions into the official Facilitator Notes document for the course.

TIP: This ensures each Tech SE Review improves not just the student experience but also the facilitator’s ability to teach effectively and confidently.

Instructional Designer (ID) Expectations

Before the Review

  • Confirm the course is fully built and all IDD and Tech QA edits are complete.
  • Work with the Tech SE Review Coordinator (ITG) and STEM Program Facilitation Manager to identify the facilitator-tester and verify the Dev shell and BugHerd setup.
  • Ensure all assignments, rubrics, and Canvas modules reflect final approved content.
  • Review the Tech SE Kickoff Email for accuracy and completeness.
  • No external documents are needed—coordination happens in Wrike.

During the Review

  • Stay available during the 1-week review for content or grading questions.
  • Monitor BugHerd for content-related issues and triage as needed.
  • Collaborate with ITG to confirm whether issues are instructional or technical.
  • The Tech SE Review Coordinator manages tester communication; ID monitors progress via Wrike and BugHerd.

After the Review

  • Resolve all content-related BugHerd tickets and confirm fixes in Canvas.
  • Collaborate with ITG on technical validation.
  • Update Facilitator Notes with tester insights that improve guidance or grading clarity.
  • Confirm completion with the Tech SE Review Coordinator so the course can move to Faculty Review.

Canvas Role Setup

Each Technical Facilitator (tester) must be enrolled twice in the Canvas Dev course—once as a Student and once as an Instructor. When testing, the facilitator accesses the course as the Instructor and uses Canvas’s “Act As” feature to complete and submit work as the Student account. This dual-role setup allows the reviewer to experience the full life cycle of an activity: submitting work as a learner and then grading or reviewing that same submission as a facilitator.

ITG will:

  • Enroll the tester using both roles in the Canvas Dev environment. Confirm with the STEM Program Facilitation Manager  that the assigned facilitator has both Instructor and Student logins and Act As permissions enabled.
  • Provide confirmation that both invitations have been sent and accepted prior to the review start date.
  • Ensure all linked integrations (Codio, Jupyter, custom tools) are active for both accounts.

Tester Responsibilities:

Student Role – Work through the course exactly as a learner would, completing assignments, discussions, and submissions.

NOTE: If an assignment is autograded, avoid submitting the provided solution verbatim. Try tweaking your code or writing a slightly different version to make sure the grader catches all valid approaches. This helps reveal any gaps or overly rigid checks in the autograder’s setup.
  • Instructor Role – After submitting work, switch to the instructor view to confirm:
    • Grades appear correctly in SpeedGrader or integrated grading tools.
    • Rubrics display and calculate properly.
    • Submitted files or code cells open and render without issue.
  • Confirm Access – Before beginning the review, verify that both accounts are visible in the course navigation bar and that switching roles does not cause permission or loading errors.
If either role is missing or access appears limited, contact the Tech SE Review Coordinator immediately before starting the review. NOTE:  If “Act As” permissions appear disabled, notify the Tech SE Review Coordinator immediately so ITG can enable access before testing begins.

This dual enrollment ensures testers can validate the end-to-end workflow—from submission through grading and facilitator feedback—mirroring real course conditions.

Process

Timing Overview

The Tech SE Review occurs during the course development Review Phase, after initial IDD edits and technical QA but before faculty sign-off.

Phase

Purpose / Focus

Example Key Tasks

Owner(s)

Phase 1 – Design & Development

Core course buildout and early reviews

• Build course in Canvas Dev • Conduct Faculty & IDD content review • Complete initial Tech Review (QA of tools/Codio/etc.)

ID + ITG + Faculty

Phase 2 – Review & Prep

Internal reviews and coordination leading up to testing

• Coordinate with the STEM Program Facilitation Manager to assign Technical Facilitator (Tester) • Ensure Dev shell and BugHerd are ready • Confirm readiness for Tech SE Review

Tech SE Review Coordinator + ID (+ STEM Program Facilitation Manager)

Phase 3 – Implementation & Testing

Live facilitator test and follow-up edits

• Conduct Technical Facilitator Review (Tech SE Review) • Implement Technical Review Edits

Tech SE Review Coordinator + Tester / ID / ITG / Faculty

Phase 4 – Faculty Sign-Off & Launch Prep

Final validation and wrap-up

• Conduct Faculty Review • Final QA Check • Update Facilitator Notes • Move course to Master

Faculty + IDD + ITG + Ops

Step-by-Step Process

1. Prep and Coordination
  • Tech SE Review Coordinator and the ID identify the course ready for Tech SE Review.
  • Tech SE Review Coordinator coordinates with the STEM Program Facilitation Manager to select one qualified facilitator.
    • Default: 1 tester
      • Optional: Add a 2nd for a supplementary run if justified.
  • Tech SE Review Coordinator prepares the Canvas Dev course, ensuring:
    • BugHerd project linkage
    • Student + Instructor roles for tester
    • All course components (Codio, Jupyter, etc.) function correctly
2. Kickoff Communication
  • Send the Tech SE Kickoff Email (see below) once the tester is confirmed.
  • Optional: schedule a short Zoom meeting to walk through goals and ensure buy-in.
  • Include in the email:
    • Course URL and login info
    • BugHerd instructions
    • Review goals and expectations
    • Timeline and due dates
    • Tech SE Review Coordinator’s contact for tech support
3. Review Period
  • Tester completes the course as a student, logging issues in BugHerd.
  • Tester switches to the instructor role to validate:
    • Grading rubrics
    • Feedback visibility
    • Submission and completion workflows
  • Tech SE Review Coordinator tracks feedback and follows up as needed.
4. Post-Review Wrap-Up
  • Tech SE Review Coordinator reviews logged issues and coordinates a BugHerd debrief with ID, ITG, and relevant stakeholders.
  • Technical fixes handled by ITG; instructional updates by the ID.
  • Tech SE Review Coordinator confirms that all updates are validated and facilitator notes are refreshed.
  • Course moves to Faculty Sign-Off and QA.

Resources

Resource

Owner

Purpose

Tech SE Review Template

ID / Tech SE Review Coordinator

Outlines timeline, roles, and details for the pilot Tech SE Review

Tech SE Kickoff Email

Tech SE Review Coordinator

Standard communication template for facilitator testers

BugHerd Project

ITG

Issue-tracking and documentation of tester findings

Canvas Dev Course Shell

ITG

Secure testing environment with dual-role access

Facilitator Notes (updated)

ID / ITG

Repository for observations, grading notes, and key takeaways

Review Tips

  • Verify each assignment’s grading and rubric visibility.
  • Test any integrations (Codio, Jupyter, embedded tools) for proper function.
  • Use BugHerd for specific, actionable bugs; email the Tech SE Review Coordinator for access or blocking issues.
  • Focus on user flow, clarity, and accuracy—not copy edits.
  • Avoid duplicate BugHerd tickets; check existing comments before adding.

Tech SE Kickoff Email (Template)

NOTE: Modify this template when sending to the tech SE reviewer. Check all links and dates prior to sending. 

Subject: Tech SE Review – [Course Code + Course Title]

Hi [Tech Reviewer],

Thank you for participating in the Technical Student Experience (Tech SE) Review for [Course Name] in the [Certificate Series Title] program. This review helps us confirm both the student experience and technical functionality before launch.

This streamlined review uses a single technical facilitator (you) to represent both the student and instructor experience, allowing us to capture targeted, high-impact feedback efficiently.

Your Role

You’ll be enrolled in the Canvas Dev course twice:

  • as a Student – to complete all course activities and test assignment submissions
  • as an Instructor – to verify grading rubrics, feedback visibility, and facilitator tools

Your review window runs from [Start Date] through [End Date]; please try to complete your testing and feedback within that timeframe so we can turn around edits quickly.

You’ll receive two Canvas invitations (one per role). Please accept both before your review week begins and confirm that you can switch between accounts. If either invite doesn’t appear or you can’t access both roles, contact me right away so we can fix it before you start.

Please complete the following during your review:

  1. Work through the course content as your student account. Log all bugs or feedback in BugHerd (see BugHerd Quick Start guide below).
  2. Switch to your instructor account to confirm grading and rubric setup. Again, log all bugs or feedback in BugHerd
  3. Facilitator Notes Feedback: As you test, please note anything that would help future facilitators—such as grading nuances, student pain points, or setup tips for tools like Codio or Jupyter. You can log these in BugHerd or email them directly to me. This feedback helps us refine the Facilitator Notes used by future instructors.
  4. Finish your review and log all feedback by [End Date]. 

Using BugHerd for Feedback

BugHerd lets you capture issues directly in the browser with comments and screenshots. Please log:

  • Functional or technical bugs
  • Assignment or grading issues
  • Typos, unclear instructions, or navigation problems
  • Suggestions to improve flow or clarity

👉 [BugHerd Quick Start Guide (link)]

If you don’t receive your BugHerd invite by [Date], please let me know right away. If you don’t see the BugHerd toolbar or can’t log in once the review begins, please contact me right away so I can re-issue your invite.

Support

If you encounter blockers or have any questions, reach out anytime: 📧 [Your Email] 💬 Optional: I’m happy to meet via Zoom at the start of the week if you’d like a quick walkthrough.

Thank you again for your time and expertise — your feedback is vital in ensuring this course launches with a seamless experience for both students and facilitators.

Best,

Jason Carroll

Tech SE Review Coordinator

Instructional Technology AssociateITG | eCornell | Cornell University

How did we do?

7. Conduct Technical Review of Course (STEM-only)

2. Conduct IDD or Sr ID Review

Contact