Mocha/Selenium Autograding

Nick Updated by Nick

What is Mocha?

Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple to run. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.

What is Selenium?

Selenium is an open-source suite of tools and libraries that is used for browser automation. Selenium is used to:

  • Allow users to test their websites functionally on different browsers. 
  • Perform Cross browser testing to check if the website functions consistently across different browsers. 

It provides a single interface that lets you write test scripts in programming languages like Ruby, Java, NodeJS, PHP, Perl, Python, JavaScript, and C#, among others. Selenium is very extensible and can be integrated with other tools and frameworks like TestNG, JUnit, Cucumber, etc.

For web-based projects, we can use the Selenium Webdriver to drive a browser natively, as a user would, either locally or on a remote machine using the Selenium server.

JavaScript Development

Starter Pack: Mocha/Selenium JS Auto Grading

Stack: Mocha/Selenium Auto Grading

In Codio, we'll want to use Headless Chrome for developing auto graded tests. Headless Chrome lets you run the browser in an unattended environment without any visible UI.

To start off our auto graded tests, we'll need a few lines of code in our script to require Headless Chrome, the Selenium WebDriver and the assert library that will be used in our tests. The assert libraries provide developers with a set of tools to validate the behavior and output of their code. They enable the creation of expressive and human-readable test cases that can affirm whether a given piece of code behaves as intended.

We'll begin with the following lines of code:

const {Options} = require('selenium-webdriver/chrome');
const options = new Options();
const {By, Builder} = require('selenium-webdriver');
const assert = require("assert");

Next, we need to set up the test suite, which can support multiple tests along with a before function. The before function is an asynchronous function which will simulate the activity the user/student will be doing when interacting with the webpage. Important Note: Using this method, as long as the solution code is correct, any and all interactivity done for the purpose of autograding will take place using the before function's code. Neither the student nor the facilitator will need to manually interact with the web page in order to successfully run an auto graded test.

The following code is the outer layer code which while house the individual tests:

describe('Assessment returned that - ', function () { 
    this.timeout(0); // No time limit 
    let driver; 
    // Setting up the driver and declaring the test file to be used in the test suite
    // Simulate student interactivity - user actions OR to make an item accessible 

    before(async function () { 
        driver = await new Builder().forBrowser('chrome').setChromeOptions( 
            // Uncommment the next line before publishing, otherwise it will stop the browser. Commenting out is ONLY used for debugging 
            options.addArguments('--headless=new') 
            ).build(); 
        await driver.get('file:///home/codio/workspace/exercise_files/index.html'); 

        // opens the item we want to focus on ...so then we can get info from it (ID per element, etc)
        //Find the element by it's ID called "openGallery"
        let openGalleryBtn = await driver.findElement(By.id('openGallery'));

        //Simulate a click on the button we just found
        await openGalleryBtn.click(); 

        //We can add additional interactive steps here as needed
    });

    // Clean the execution 
    // Uncommment the next line before publishing, otherwise it will stop the browser. Commenting out is ONLY used for debugging 
    after(async () => await driver.quit()); // Closes the browser

});

Important note:

Note the following lines that are in the code block above:

  • options.addArguments('--headless=new')

and

  • after(async () => await driver.quit()); // Closes the browser

These two lines must be uncommented before publishing the Codio unit. These two lines are only to be commented out during the debug process. Debugging will be discussed later in this article.

Now, considering the next line:

await driver.get('file:///home/codio/workspace/exercise_files/index.html');

Simply change the file path in this line to the file that needs to be tested.

Now we are ready to add our individual unit tests to our auto graded script. We will be using the revised CIS560s course for our example tests. In the examples used, we will be using an application that when clicking a button, opens up a modal lightbox for an image gallery. Please see the example below:

Let's begin our first unit test:

it('The "thumbnail" class is selected', async function () { 
    // Search for the ribbon 
    let ribbon = await driver.findElement(By.id("thumbnailRibbon")); 

     // Get all elements with the tag "img" inside the ribbon 
    let images = await ribbon.findElements(By.tagName("img")).then(async function(elements){ 

        // Simulate a click on any of the elements   
        elements[2].click(); 

        // Get the class attribute of the element
        let elementClasses = await elements[2].getAttribute("class"); 
    
        // Now, assert the class of the element that was clicked is "selected"
        assert.match(elementClasses, /selected/); // Match selected 
    }); 
})

As seen from the snippet above, we can do specific simulated actions pertaining to each individual test. Once we get the image ribbon stored in our images object, we can simulate a click on any of the elements just like a student user would do. At the very end, we use the assert library by doing an assert.match() to test if the class of the element is "selected". Node.js handles the assert statements. We can find the full list of possible assert statements here.

At the very top of the snippet, we have the it() function, followed by a string as the first parameter. Whatever is written in the string, will be what shows to the user to denote what the test is trying to assert in readable text.

The following is an example of what will be displayed to the student after clicking the "Test Code" button:

As we can see from the image above, the first two tests have failed, while the second two tests have passed. The student will receive 50% as their score from this assessment.

Setting up the Codio JavaScript Unit
  • Configure the preview button in the Codio menu. After the preview button is configured, we can click the button newly to be named "Mocha" to run the auto grader first inside the Codio terminal for development purposes. To enable this, make sure the following code is present:
    {
        // Configure your Run and Preview buttons here.
        // Run button configuration 
        "commands": { 
            "Mocha": "mocha .guides/tests/testJS.js --reporter .guides/secure/score.js 2>&1 || true"      
         },
        // Preview button configuration 
        "preview": { 
            "Project Index (static)": "https://{{domain}}/{{index}}", 
            "Current File (static)": "https://{{domain}}/{{filepath}}", 
            "Box URL": "https://{{domain3000}}/" 
        }
    }

To run the unit tests inside the terminal without the reporter, the following command only should be run:

mocha .guides/tests/testJS.js

  • Set up the reporter. Under .guides/secure/, add a file called "score.js", for example. and paste in the following script. The reporter does two things. It passes the grade the student receives from the automated tests as well as presents the output in a clean format that we're able to manage in the file.
  • Set up the package.json file. Create this file in the /workspace directory with the following code pasted in:
{ 
    "scripts": { 
        "test": "mocha .guides/tests/testJS.js --reporter .guides/score.js 2>&1 || true" 
    }, 
    "dependencies": { 
        "assert": "^2.0.0", 
        "chai": "^4.3.10", 
        "chai-dom": "^1.11.0", 
        "jsdom": "^22.1.0", 
        "jsdom-global": "^3.0.2", 
        "selenium-webdriver": "^4.14.0" 
    }, 
    "devDependencies": { 
        "mocha": "^10.2.0" 
    }
}

Using the terminal command npm install, the above dependancies will be installed.

  • Create the Advanced Code Test. The command to run will be "npm test" and partial points will need to be turned on if there are more than one test to be run.
Debugging JavaScript Code with Mocha and Selenium
  1. First, in the auto grade script, comment out the following two lines:
    1. options.addArguments('--headless=new') 
    2. after(async () => await driver.quit()); // Closes the browser
  2. In the Codio menu, click on Project Index (Static) and change it to Box Url. A new tab will appear showing a subdomain name along with a port number (3000).
  3. Change the port number in the URL from 3000 to 3050. Doing this will allow you to interact with the browser that Selenium is using.
  4. In Codio, click on the Mocha button to open the browser in the new tab. Note: Each time the Mocha button is clicked, it will open a new instance of the Chrome browser. Close each instance of the Chrome browser before opening up a new one with the Mocha button. If there are too many instance of Chrome running in this method, Mocha will run out of memory and fail.
  5. Now we will be able to interact with the browser just like the code put into the before() function. Note: Selenium will interpret this fairly quickly, so it's possible the use of timers might be needed.

How did we do?

R Studio - Exclusion List for R Code

Starter Packs in Codio

Contact