Table of Contents

2. Conduct IDD or Sr ID Review

Jason Carroll Updated by Jason Carroll

1. Prelude

Before it gets to you for review, it should have gone through the ID checklist, which means the ID or IDA has already checked for a lot of standard things.

2. Check charter

Map home page performance objectives against stated charter objectives; we should have alignment (or you, as the IDD, know why changes were made during development, and you can document what/how you're bringing them into alignment with revised expectations now). There shouldn't be a big surprise variance between what was asked for at charter and what we actually produced.

3. Check home page performance objectives ("What you'll do") against project parts

Make sure the student is actually and literally doing in the project what we say they're going to do on the home page. No purple prose, making it sound like more than it is. The home page objectives are the promise; the project is the fulfillment of that promise. The project has to give the student a thorough chance to meet objectives. Example: if the home page says "You will achieve compliance with ADA laws," but the project is "Create an action plan to use strategies for compliance," those are not the same thing. No dice. We have to force the project practice to agree with the objectives.

4. Check that module titles, modules intros & wrapups, and project titles match

  • Force compliance with this to validate that the integrity of the design is tight.
  • Check module intros and wrap-ups: do they effectively set the scene instructionally, or is the writing just a dry list of things they'll do in the module? This is where storytelling comes into play.
  • Module title is a verb.
  • Project takes the -ing form of the same verb.

5. Review project

  • Review the course project; make sure it's robust, well-reasoned, and logical. Check to ensure that the target audience can actually complete the project within 2 hours (remembering the course timeframe is 2 weeks), and that there are reasonable options for participants who may not be employed or are in transition, as appropriate. For example, a project that asks them to interview every member of their Exec Leadership team at their company within a 2-week timeframe may not be reasonable or possible.
  • See if you could complete this course project without completing the course. That's a sign it's not hard enough.
  • All course project pages get the rubric; only the final piece gets submission settings.
  • Project instructions are clear, and expected quality standards are well-defined (rubric).
  • Make sure that the grading structure is clear and the Canvas implementation is accurate.
see these Grading Knowledge Base articles for more info

6. Review Discussions

  • They should prompt real group discussion and not just multiple monologues. Standard settings. For catalog standard, there should be two, and only two, graded.
  • Typically, one good question that tries to pull from students' personal experiences/emotions is more than enough to generate a robust discussion. If we're asking them to answer 4-5 questions, that will yield multiple long posts and not back-and-forth engaged discussion.

7. Review supporting activities and tools

  • Are they necessary? Valuable? Easy to follow?
  • Are there any missing that we could add?
  • Are they well-formatted and up to our graphic design standards?
  • Course activities provide opportunity for immediate practice of complex and critical teaching content.
  • Activity instructions are clear and expected quality standards are well-defined.
  • Course keeps learners active at appropriate intervals.

8. Review beginning to end, page by page

  • Check: You're looking for big-picture coherence, integrity, rigor, connective tissue, and strong writing, but also the details of every element as noted below.
  • Click through page by page: check videos, animations, engagement opportunities, tools, downloadables, any supportive writing, graphics, etc.
  • Check to see if the videos directly support the student practice (they should).

9. Home page video and home page elements

  • Check: The intro video should be engaging and address the 'WIFM' (What's In It For Me) for the student, and not just be a boring 'Hello, my name is, welcome to blah blah blah.' (It's too late to redo the intro video now, so this is a note for next time.)
  • Check elements, bio/course description, start course button: all elements, all writing.

10. Watch pages

  • Check animation: look for clear visuals that enhance key points
  • Check video length (limit is 4 minutes - if it's way longer that's a conversation you'd want to have to find out why)
  • Check to see if the videos directly support the student practice (they should).
  • Check vid title - should be action oriented, which typically means it starts with a verb (i.e. Choosing the Right Data > Data Types)
  • Check page layout

Video intros

  • Check: Do they repeat verbatim what the professor says? (That's a no no). They should entice the learner without giving away critical teaching content such that the learner can skip the videos
  • Check: Are the intros learner-centered? Preferred: you will now explore key concepts... vs non-preferred: Professor X will tell you key concepts...
  • Check: They don't use the term "learn."
  • Check: Do they smooth over possible confusion the learner might have at that point in time? I.e. maybe the faculty member skipped over something accidentally, does the vid intro provide that hand holding for the learner so they're not lost or confused as to why they're watching what they're watching at that point in the course

11. Read pages

  • Check page layout - a long and tedious page of just text should be converted to click and reveals (h5P) or have formatting that breaks it up for learners
  • Check writing (coherence, quality, etc etc)
  • Check teaching (always check teaching)

12. Other

  • Student Lounge discussion (required in all non-credit courses)
  • Check glossary
  • Thank you and Farewell
  • Additional Reading
  • Any other elements (interactives, etc)
  • Action plans are always required unless the course project is in effect serving the same purpose; IDs make them from standard template, they're an added Tool, they take no more than 5 minutes
  • Qualtrics are illegal per Karen

13. Writing Style

  • Course text is action-oriented and explains how course concepts will help participants achieve goals and solve problems.
  • Course text provides conversational (second person), non-academic guidance to learners.
  • Course text builds a progressive storyline and clearly explains how learning components fit together and culminate in useful outcomes.
  • Course text provides instructional rationale, especially for unexpected elements (i.e. if a faculty member is repeating themselves in a vid, explain the repetition in the copy).

14. Tools

  • Helps the learner achieve learning outcomes on the job
  • ALT text included for all non-decorative graphics and photos?
  • Tools require active input from the learner OR provides information that participants would want/need after they lose access to Canvas order to apply on the job.
  • Tools do not provide brand new teaching concepts

15. Community

  • Course promotes a sense of community through peer sharing about personal experiences or opinions on open-ended questions.
  • Discussion boards provide clear instructions for participation. (These should be boilerplate from the eCornell master course and include the plagiarism note)
  • Discussions enable deeper reflection and/or learning from peers and/or experts.
  • Discussions avoid long, multi-part questions that lead to multiple monologues from individuals rather than a lively give-and-take.

16. User Experience

  • Animations enhance video elements that are critical for the student to remember or know.
  • Course content is presented using a variety of modes (videos, graphics, text, etc.) based on how best the students will retain the information.
    • Example: a list of formulas should be presented within a downloadable reference tool, not merely said out loud in video
  • Information is categorized and presented in a consistent and easy-to-understand manner.
  • Page design is clean and visually appealing.

17. Additional/External Resources

  • External resources provide a missing or unique perspective.
  • OR they reinforce and further validate information provided by faculty (i.e. supporting research).
  • OR they provide images or information that are cost prohibitive or unreasonable for eCornell to reproduce (require permission).

18. Scope

  • Scope is consistent with budget, instructor burden expectations, and deadlines.
  • Target audience can complete the course within the number of hours promised (including ALL components) and feel challenged but not overwhelmed
  • For non-credit, this standard was traditionally 6-10 hours over two weeks, but note this course's mileage may vary
  • For some Tech/STEM courses there is often more variation: in some the course length is increased to 3 weeks each to accomodate more complex and dense content (ex: Machine Learning); in others, the certificate courses are combined into a single course (often with "units" corresponding to the previous courses) offered in more of a semester-style model (ex.: Spaceflight Mechanics; Autonomous Mobile Robots). The length and offering structure of STEM/Tech courses is often adjusted based on responses received during the Alpha Test.
For more the on Alpha Test process, see this Knowledge Base article: https://knowledge.ecornell.com/article/3i5n1rks3m-alpha-review-process.

How did we do?

7. Conduct Technical Review of Course (STEM-only)

Contact