Table of Contents

Pedagogical Guidelines for Implementing AI-Based Interactives: AER

Markette Pierce Updated by Markette Pierce

I. Introduction

These guidelines serve to inform eCornell instructional design staff on when and how to strategically implement the newly developed AI-driven AER (Active Engagement and Reinforcement) interactive. The goal is to enhance student engagement, interaction, and learning outcomes by including AER in design considerations for all eCornell online courses hosted on Canvas.

II. Active Engagement and Reinforcement (AER) Interactive

  • What is AER?: An AI-driven text-based interactive that can follow single videos on a Watch page. The interactive encourages students to further engage with the content in its attached video by asking students to share a key takeaway or a specific question about the video. AI trained on the content of the video provides feedback on the student response as well as additional learning points.
  • AER’s Purpose: AER is designed to transform passive content consumption (e.g., watching videos) into an active learning experience by prompting active retention and providing personalized, AI-driven feedback on student responses. AER encourages deeper processing and reinforcement of content, making it suitable for materials with complex, abstract, or nuanced ideas that students need to engage with critically.
  • Multiple Versions: There are currently two types of AER: AER v1 is limited to asking students to share a “key takeaway” from the content; AER v2 enables instructional design staff to author custom questions specific to the content itself.

III. Guiding Principles for Use of AER

  1. Enhancing Passive Content:
    1. Integrate AER into long sections of content that students might otherwise digest passively (e.g., video watching). Use it to prompt engagement and retention. We’ve all dealt with long strings of dense Watch pages. AER helps add some student engagement within those pages.
  2. Strategic Content Engagement:
    1. Use AER when students are expected to process material deeply, beyond just surface-level comprehension.
      • Ideal for complex topics requiring thoughtful reinforcement (e.g., theoretical concepts, ethical debates, or multifaceted problems).
  3. Critical Thinking:
    1. Implement AER when you want students to pause and confirm their understanding by restating critical teaching or explaining concepts in their own words.
    2. Ask open-ended questions (with AER v2) that challenge students to synthesize ideas, make connections, or explain nuances.
  4. Feedback on Conceptual Understanding:
    1. Use AER to give students formative feedback on their interpretation of key ideas. This helps correct misconceptions early and reinforces correct understanding.
  5. Balancing Use:
    1. Avoid overuse. Select moments in the course where reinforcement will have the strongest impact on learning outcomes. Not all content requires or benefits from AER. There’s no set formula for the right amount of AER interactives in a course. So primarily base your decisions around the objective needs of the content and your students to inform your decisions.

IV. Limitations

As it is currently configured, AER (both v1 and v2) can only be attached to a single video on a Watch page. It cannot be attached to Read pages, Discussion pages, links on page to external videos, or Watch pages with video playlists. This functionality will be prioritized for future iterations of AER based on design needs and student feedback.

AER can only be trained on a single video, not every video in a course. We expect to evolve into functionality that enables AER to have awareness of all course content and provide feedback accordingly and directively. 

Students may only attempt an AER activity once. If they leave a page with a completed AER activity and return, the feedback from their submission will persist and the student will not be able to submit another answer. This is deliberate, and intended to avoid misuse. This does, however, make the process for testing your AER interactives in development slightly unique. See the “Tips for IDs” section for guidance.

Currently, AER is not tied to any grading features in Canvas. AER is an ungraded, optional element for students. 

V. Faculty Positioning

As with any design consideration, we want to make faculty partners aware of AER and guide them strategically in deploying it where it can maximize the impact of their teaching. AER is an AI-based tool, but one that requires strategic guidance in thoughtful instructional design. It is critical faculty see it as such and NOT the insertion of AI-generated content in their courses without ID scrutiny.

There will be a wide range of faculty sentiment toward any AI-based features. Your role is to keep the focus on student impact, collaborate, and make recommendations accordingly.

When integrating AER v2, collaborate with your faculty partner(s) on authoring customized questions. Planning in advance can enable real synthesis between AER v2 and the video to which it’s attached, so be creative. (Guidance on authoring AER v2 questions is provided in section VII.) AI, when overseen by an instructional designer, can also be a useful tool for generating questions. (Guidance for leveraging AI to author AER v2 questions can be found in section VIII.)

It is also important that you leverage your faculty relationships to help inform how AER evolves globally. How would faculty shape AER to compliment their teaching? Share ongoing input with your IDD for review by Product and Tech for ongoing enhancement.

VI. Deciding Between AER v1 and v2

When to Use AER v1

  1. High-Level Reflection and Synthesis:
    1. Use AER v1 for more general content where high-level reinforcement or synthesis is appropriate, and specific details are less crucial. v1 can capture broader interpretations and enable students to retain key information.
  2. Less Complex, Introductory Content:
    1. Use AER v1 in cases where content is straightforward or introductory. This allows students to grasp overarching ideas without diving into specifics.
  3. When Monitoring Initial Reactions or Baseline Understanding:
    1. Use AER v1 at the beginning of a course or a new unit to gauge students' baseline understanding or initial reactions to the material. This can also serve as a foundation for more targeted questioning with AER v2 in later sections.

When to Use AER v2

  1. Depth of Engagement Needed
    1. Use AER v2 when a deeper level of comprehension or analytical thinking is required beyond a general reinforcement.
    2. Potential Scenario: In a course covering critical theoretical frameworks, v2 can prompt students to analyze specific aspects of each theory, such as its application or its limitations.
    3. Example v2 Question: "How would you apply Theory X to solve a similar problem in your field?"
  2. Specific Learning Objectives Alignment
    1. Use AER v2 when learning objectives call for understanding specific details or skills that would benefit from targeted questions.
    2. Potential Scenario: In a data analytics course, if a video explains statistical methods, a v2 question could ask students to calculate or interpret a statistical measure.
    3. Example v2 Question: "Calculate the standard deviation of the dataset provided in the video. Explain your steps."
  3. Content Complexity and Nuance
    1. Use AER v2 when content includes nuanced or layered concepts that may be missed in a broad takeaway.
    2. Potential Scenario: In a medical ethics course, V2 can be used to ask students to evaluate the ethical considerations presented in a complex patient scenario, encouraging critical analysis.
    3. Example v2 Question: "What ethical principles are in conflict? How would you address this as a healthcare provider?"
  4. Application-Based Learning
    1. Use AER v2 when students need to apply learned concepts to specific, real-world contexts.
    2. Potential Scenario: For engineering courses that discuss systems or models, V2 can prompt students to apply principles from the lecture to a hypothetical or real-world project.
    3. Example v2 Question: "Outline how you would approach solving a water distribution issue in your community."

VII. Drafting Content-Specific AER v2 Questions

As illustrated in the above examples, less is generally more when authoring v2 questions. We don’t want students to have to navigate a huge block of text to figure out how to respond. And there is no need to summarize the content students have just digested. Just go right into a direct question. Try to avoid asking too many follow-up questions. Where relevant, you may use AER v2 to ask students how they will apply the content in their personal professional contexts.

An important subtlety: There’s a difference between asking students to think critically about core concepts versus loosely defined “reflection.” We are deliberate in using terms like “reinforcement” and “think critically” and avoiding terms like “reflection.”

And a quirk: The AI engine behind AER is currently trained to provide students feedback based on the ENTIRE content of a video transcript. So if your questions are limited to specific portions of a video, be aware that the AI response will draw upon the entire context of the video for its response.

Be sure to vary question types from AER to AER, so the experience doesn’t become formulaic.

VIII. Using AI to Draft Content-Specific AER v2 Questions

AI can be prompted to analyze video transcripts and suggest relevant, content-specific AER v2 questions. This approach allows IDs to create high-quality prompts based on complex course content with minimal manual work. However, it is important that any AI-generated content is reviewed by an instructional designer to ensure its accuracy, effectiveness, and alignment to the content.

Even when collaborating with faculty on authoring AER v2 questions, AI can be a useful tool. Applying it in this context should be left to the discretion of the instructional designer in conference with their instructional design director and, ultimately, their faculty partner.

Here is a suggested starter prompt to use to turn your favorite gpt into an AER v2 question authoring tool:

---------

You are an advanced question generator designed to create a single, open-ended question for each educational transcript uploaded. These questions should engage the learner deeply with the material, encouraging them to consider practical applications or explore implications without using the word "reflection." Follow these guidelines closely:

  1. Vary question type: Alternate between different question formats for each transcript, such as:
    1. Scenario-based applications: "Imagine a scenario where [concept] is in use..."
    2. Challenge exploration: "What challenges might arise when..."
    3. Principle identification: "What general principle can we derive from..."
    4. Opinion with justification: "In your view, how important is..."
    5. Prediction-based questions: "How might [context] change if..."
  2. Encourage active thinking: Frame each question to prompt active engagement with the content. Encourage learners to analyze, hypothesize, or consider real-world applications of the ideas discussed.
  3. Contextualize without rephrasing: Avoid restating the transcript. Use specific content from it to create questions that push learners to engage with the material meaningfully and practically.
  4. Examples of question structure: Here’s how to model questions effectively:
    1. For a transcript on negotiation principles: "How might understanding both parties' interests, beyond their stated positions, impact a negotiation in a high-stakes business setting?"
    2. For content about generative AI risks: "Considering data-level and algorithm-level harms, what precautions could companies implement to responsibly deploy generative AI?"
    3. For a lecture on leadership dynamics: "In a team environment, how might different leadership qualities, such as decisiveness versus empathy, affect group cohesion and outcomes?"

When generating each question, focus on drawing out the learner's insights and application of concepts to reinforce their understanding actively.

---------

*** PLEASE NOTE: This prompt may require customization to meet your particular needs. You may require a different prompt altogether. This is OK. If you are uncertain, consult your IDD.***

Leveraging AI to author questions should not be a passive engagement by the instructional designer. There are a number of quirks to AI that require vigilance and mindfulness by a designer. Here are just a few of the quirks we’ve experienced in testing. We expect more to arise with broader usage.

  1. AI can become lazy over time. For example, you may find that it falls into a pattern of redundancy, forgetting to vary the question types it offers. Don’t hesitate to remind the AI of its directions.
  2. A couple punctuation issues to be aware of when copying/pasting content from your question generator into Canvas:
    1. If your AI-generated question contains quotation marks (“), they need to be changed to single quotes (‘) to avoid confusing the Canvas html editor. (This applies to v2 questions not authored by AI, as well.
    2. Em dashes (—), if used in your AI-generated question, will likely be provided without spaces on either side of them. Those spaces will need to be added manually.
  3. Review, review, review. Any question generated by your AI authoring engine will require your review to ensure its relevance to the content and that it actually aligns with learning/performance objectives.
  4. Don’t hesitate to re-paste your transcript into your authoring engine if you are not happy with initial results.
  5. Provide feedback to your authoring engine as you go to calibrate it accordingly.

IX. How to Embed AER v1 and AER v2

AER v1

Embed the following code in the “class” section of the <div> Watch Page html: 

resp2 ec-aer 

AER v2

Embed the following code in the “class” section of the <div> Watch Page html: 

resp2 ec-aer 

Immediately after that line, add the following code:

data-question=“Text of your question.” 

X. Tips for IDs

In general:

  • The Power of Planning and AER: AER can be particularly effective when its usage is properly planned. For instance, imagine if the faculty in the video was aware of the AER that follows it.
  • Again, engage your faculty around AER and its potential impact on their teaching. Share their ideas for how it could evolve with your IDD. AER will be reviewed for ongoing enhancements at least quarterly.
  • Always test your AER’s functionality to ensure the responses coming back from AI are appropriate to your learning design. To do so without wrestling with single-try limitations of the interactive, simply copy your page and try again. Make sure to delete unnecessary pages when finished testing.
  • Looking for inspiration? Want to keep up with all the live AERs your colleagues have designed? Visit and contribute to the ever-evolving AER INDEX. (Please be sure to log any AERs you create to the index.)
  • AER built upon new and evolving technology. If you experience strange or unexpected AER behavior, log it to the Unexpected Behaviors tab of the AER Index and CC your IDD. 

When using AI to generate AER v2 questions:

  • Always Review All AI-Generated Questions: Ensure questions are directly aligned with course goals and adjust wording as needed.
  • Align with Feedback Objectives: Modify the AI’s suggestions to align with the specific feedback students will receive in the AER activity.
  • Repeat Prompting for Additional Options: Don’t like the questions provided by your generator? Try pasting the same transcript again. 
  • Train and Retrain Your Model: Don’t hesitate to remind your model of key specs regarding the questions it is authoring if you find it starting to veer or if it’s struggling to hit the mark.

How did we do?

Contact