Table of Contents

Mathpix: Accessible STEM

Brock Schmutzler Updated by Brock Schmutzler

Overview

Mathpix uses artificial intelligence (AI) and optical character recognition (OCR) to capture math images and generate MathML code, which is the key to creating math that is accessible to people using assistive technology. Mathpix also generates LaTeX, PNG, and SVG code that is useful in producing STEM content.

Here are some helpful links to Mathpix documentation:

Snip Desktop and Snip Web are the two Mathpix apps you will use most often. When building STEM materials with accessible math, these two apps complement each other but also have distinct functionality which suit different workflows:

  • Snip Desktop - Take screenshots of math then copy/paste the "MathML (MS Word)" code from Mathpix into a Microsoft Word document. (Mathpix also has its own documentation on using Mathpix Snip Desktop with Word.)
  • Snip Web - Create content in a Mathpix Note (written in Mathpix Markdown) then export to a Microsoft Word document (or HTML file) with built-in MathML code.

This article will explain how to use the "snipping tool" to take screenshots of math with the Snip Desktop app then move on to discuss the Snip Web app.

Mathpix Snip Desktop

With the Mathpix Snip Desktop app, you can build accessible math in essentially two steps:

  1. Use the snipping tool to capture a math image on your computer screen.
  2. Copy and paste the MathML code into Microsoft Word (or any other MathML-aware editor).

The details are given in the subsections below.

Snip Math Image

The snipping tool can be used to capture any static math image on your computer screen (e.g., PDF, website, paused video) using Mathpix OCR technology.

  1. Download Mathpix Desktop (mathpix.com > Downloads > Desktop) and install it (you will need to contact it-help@ecornell.com for installation if you do not have administrator privileges).
  2. Click the Mathpix icon in the menu bar (upper-right corner of your screen):
  3. Click the snipping tool (monitor icon) in the upper-left corner of the Mathpix window (keyboard shortcut: CTRL+CMD+M) and use your cursor to draw a box around the math image. To draw the box, left-click near the upper-left corner of the math image and, while holding the mouse button down, drag your cursor to the lower-right corner of the math image to create the box, then release your mouse button to "snip" what is in the box:
  4. When you release your mouse button, Mathpix OCRs the math inside the box and converts it to code in the Snip Desktop app, where you can copy and/or edit the LaTeX code that Mathpix has generated for the math image. In the example below, the first image shows the original image along with the LaTeX code and output, which is not quite correct. Clicking the Edit button in the first image allows us to make the corrections shown in the second image. After clicking the Save button shown in the second image, we are shown the new LaTeX output in the third image.

At this stage, the correct LaTeX code for this math image has been copied to your clipboard. The default "OCR" tab (above) will show you the LaTeX code for the math image. The "Data" tab will give you access to other code formats for your math image (e.g., MathML). This is discussed in the next two subsections.

Snip Desktop to Word

If you are creating accessible STEM content in a Microsoft (MS) Word document, follow these steps to copy and paste the MathML code from the Snip Desktop app into Word.

  1. Click the Copy MS Word button to copy MathML code optimized for Word documents:
  2. Paste — as unformatted text — the "MathML (MS Word)" code into the appropriate place in the Word document. Word should automatically render the MathML code in a manner which is similar (but not exactly the same) as the rendering in a LaTeX editor or PDF (see second screenshot below).
  3. The image you see in your Word document should now be embedded with MathML code, which assistive technologies can use, for example, to create speech. One way to verify that the MathML code is in the Word document is to highlight the image, copy it, and paste it into a text editor:
In order for the MathML to be copied to your clipboard, you must select the Copy MathML to the clipboard as plain text option in Word. To check whether or not this setting has been turned on in Word, go to Format > Equation Options… as shown in the two images below.
Useful Output Formats

After using the Mathpix snipping tool to capture a math image, you have access to the math image in different output formats in the Snip Desktop app. We mention three of those formats here:

  • MathML code that has been optimized for producing high-quality accessible math in Microsoft Word documents
  • MathML code (i.e., standard MathML) used for building high-quality accessible math outside of Word documents
  • PNG format which generates high-resolution images that can be inserted into videos and other creative products

Depending on the context and need, each of these formats can be handed off to the Creative team or Product team.

MathML (Math Markup Language)

Mathpix generates two versions of MathML code that will make math accessible to people using assistive technology. MathML (MS Word) is a version of MathML code that has been optimized for use in accessible Word documents, whereas MathML is the "standard" version of MathML code that should be used anywhere other than Word documents.

MathML (MS Word) is the same code obtained by clicking the Copy MS Word button in the "OCR" tab.

Both types of MathML are available in the "Data" tab of the Snip Desktop app after you have used the snipping tool:

The difference between the two types of MathML is subtle (and not worth trying to describe in detail). If you need MathML code for a software platform other than Microsoft Word, copy the the standard MathML:

PNG (Portable Network Graphics)

If you need the math image encoded in PNG format, click the Open PNG button located in the "OCR" tab:

This will automatically create a high-resolution PNG file and open it in a separate window. Once open, you can save the PNG file with an appropriate name in a designated folder. You can then insert the PNG file wherever it is needed:

Mathpix Snip Web

The Mathpix Snip Web app has an HTML editor that understands LaTeX and Markdown code. Whether you are using Snip Web or Snip Desktop, Mathpix automatically equips the math with accessibility code (i.e., MathML). Moreover, the Snip Web app is synchronized with the Snip Desktop app so you can insert your snips into Mathpix Notes, where math and text can be edited and formatted simultaneously. You can also export Notes to several file formats, including Word and HTML. 

To build accessible STEM content with the Mathpix Snip Web app, there are two basic options:

  • If you have PDF content you want to make accessible, upload your PDF to Snip Web and convert the PDF to a Mathpix Note. (The "nicer" the PDF, the better the conversion.)
  • Use the snipping tool in the Mathpix Snip Desktop app to capture a math image then go to Snip Web to insert the math into a Mathpix Note.
Snip Web Settings

To open the Mathpix Snip Web app for the first time, you can click this Mathpix Snip Web link or you can go to mathpix.com and select Downloads > Web:

In either case, click the Launch Snip Web button:

The first time you sign in to Snip Web, you will see a screen that looks something like this:

Once you have signed in to Snip Web, bookmark the web page for future visits.
Before using the Snip Web app, there are a few settings you should make first (see below).
  1. Click the Account button (icon with the first initial of your name) in the left sidebar menu:
  2. Select the Account option:
  3. Go to Personal Settings > Account and deselect the "Mathpix can use my images..." button:
  4. Go back to Step 1, click the Account button, and this time select the Preferences option:
  5. Select the following options in Preferences:
Mathpix Notes

With Snip Web, it is easy to build accessible STEM content in a Mathpix Note because all the math has embedded MathML code behind it. You can either convert a PDF to a Note (as discussed in the PDF to Note to Word section) or you can follow the steps below to create a blank note and build from there.

  1. Create a new folder in your Snip Web file tree to store the note.
    1. Go to the Files tab on the left sidebar menu:
    2. Click the Create new folder button:
  2. Click the horizontal three-dot menu for More actions:
  3. Select the Create new note option:
Snip Web + Snip Desktop

Snip Web synchronizes with Snip Desktop so you have access to the snips you create with the snipping tool in the Snip Desktop app (as described in the Snip Math Image section above).

  1. In the Snip Desktop app, use the snipping tool to OCR a math image (as described above). Then click the Snips button in the left sidebar menu of Snip Web:
  2. From your list of snips, hover over the one you want to insert, click the Insert into current document button, and select the desired option (e.g., "LaTeX" or "Inline"):

PDF to Note to Word

One of the main advantages of the Snip Web app is its ability to convert PDFs to Notes then export Notes to other file formats, such as Microsoft Word documents. The general process is as follows:

  1. Upload the PDF and convert it to a Note.
  2. Edit the Note.
  3. Review the Note.
  4. Export the Note to Word (or HTML if needed).

Details about each step in this process can be found in the subsections below.

PDF Upload and Conversion

Mathpix makes it easy to upload a PDF file and convert it to a Mathpix Note. The steps below will walk you through the process. The Mathpix website has more information about the PDF conversion process.

  1. Create a new folder in your Snip Web file tree to store the PDF and Note.
    1. Go to the Files tab on the left sidebar menu:
    2. Click the Create new folder button:
  2. Upload your PDF to the new folder you created.
    1. Click the horizontal three-dot menu for More actions:
    2. Select the Upload PDF file option:
    3. Select the PDF file you want to upload:
    4. Click the Upload button:
  3. Convert the PDF to a Mathpix Note by clicking the Convert to note button:
Edit the Note

Once the PDF has been uploaded and converted to a Mathpix Note, you will need to do a bit of editing to fix any conversion errors and make sure the headers, figures, and tables/arrays are formatted correctly. A few pointers are listed below. For more comprehensive coverage, please see the Mathpix Guide at the end of this article.

  • Use the following HTML code for your title header (level 1 header):
<h1 style="font-family: Arial">
    <span style="color: #393F47; font-size: 16pt; font-weight: bold">
        TYPE OF DOCUMENT (e.g., TOOL, CHEAT SHEET)
    </span>
    <span style="color: #B31B1B">
        Name of Document
    </span>
</h1>

<hr style="height: 2px; border: none; background-color: #CECECE" />
<br />

  • Here's an example of what a title header looks like in Mathpix Snip Web:

  • Use hashtag Markdown syntax for all lower-level headers (level 2, 3, 4):
    • ## Header 2
    • ### Header 3
    • #### Header 4
  • Use the Markdown syntax **text** for bold text and *text* for italicized text.
  • Remove extra spaces that Mathpix may have erroneously inserted during the PDF conversion. Errant extra spaces tend to occur most often between numbers and punctuation, as well as between LaTeX math delimiters (e.g., $...$ and \(...\)) and punctuation. In cases when there is a repetitive edit you want to make, try using the "find and replace" functionality by pressing CMD+F (keystroke: command and F) while you are in the Editor mode pane:
  • Use the code &mdash; to render an em-dash (as opposed to a regular dash -)
  • Break up inline math separated by commas that would be easier to understand if read aloud separately; e.g., the single math object $x(i,j), i=1,\ldots,m, j=1,\ldots,n$ could be separated into three pieces of math: $x(i,j)$, $i=1,\ldots,m$,  $j=1,\ldots,n$. Or, perhaps even better: $x(i,j)$, where $i=1,\ldots,m$ and  $j=1,\ldots,n$. 
  • For math names (e.g., functions, abbreviations), use the LaTeX syntax $\text{name}(x)$ instead of $\operatorname{name}(x)$ or $\mathrm{name}(x)$. Using \text{name} helps screen readers know to read the name aloud like it would regular text (instead of maybe reading "name" aloud one letter at a time). Even abbreviations like "RHS" typically sound more natural if they are typeset by the command \text{}. Here are some examples:
    • $\text{Closest}(x)$
    • $\text{Label}(x)$
    • $\text{RHS}_x$
    • $\text{grad}(x)$
  • Put the alignment character & before the equals sign =, or whatever symbol you want to align on. In other words, use & = instead of = &. For example:
$$
\begin{aligned}
\text{Total Shipment}
& = \sum_{i=1}^m \sum_{j=1}^n x(i, j) \\
& \leqslant \sum_{i=1}^m s(i) \\
& < \sum_{j=1}^n d(j) \\
& \leqslant \sum_{j=1}^n \sum_{i=1}^m x(i, j)
\end{aligned}
$$

  • For images that do not have a figure caption, add Alt text: below the image using the Markdown syntax **Alt text:**, i.e.:
![](Mathpix image URL)
**Alt text:** Put your alternative text describing the image here.

  • If you want to export to HTML (does not apply to Word exports), put your alternative text between the brackets [...] in addition to below the image:
![Put your alternative text describing the image here.](Mathpix image URL)

**Alt text:** Put your alternative text describing the image here.

  • If you want to export to HTML (does not apply to Word exports), wrap images in appropriate HTML tags. In other words, for images that are labeled as figures with captions, wrap the Mathpix image(s) with HTML with a <figure> tag and put a <figcaption> tag below the image(s):
<figure>

![Figure #: caption text](Mathpix image URL)

<figcaption style="text-align: center">
    Figure #: caption text
</figcaption>

</figure>

  • If you want to export to HTML (does not apply to Word exports), use the LaTeX syntax $\displaystyle ...$ for inline math that you want to be rendered in display mode; e.g., $\displaystyle \int_a^b \frac{f(x)}{x} \, dx$.
Review the Note

Mathpix Notes can be shared with faculty authors for review.

  1. Click the Share button in the upper-right corner of your screen:
  2. Go to the Collaborative editing field, enter the faculty member's email address, and click the Send invitation button:
Export to Word (or HTML)

After you have finished editing your Note, you can export the Note to different file formats. The native Note format is Mathpix Markdown (file extension .mmd). For general accessibility, Word documents are typically the best option. Once your Note is in Word format, you can do further editing as needed. 

You may alternatively want to export your Note to an HTML file. In that case, you will need to do a more extensive editing job of your Note because the exported HTML file should generally not be edited (though exceptions could be made under certain circumstances).

The export process itself is rather quick. It takes a little longer for the Word export than it does for the HTML export because the Snip Web editor is essentially an HTML editor with additional features for supporting STEM content. For the same reason, you'll notice that an HTML export is a more accurate representation of what you see in Preview mode, compared to a Word export.

  1. Click the Export file button in the upper-right corner of your screen:
  2. Select DOCX for a Word file (or HTML for an HTML file):
  3. Save the exported file with an appropriate name in a designated folder.
  4. Notify the Creative Services Group (CSG) when the Word files are ready for styling.

Mathpix Guide

When you're getting started with Mathpix, you may find it helpful to have a few more tips, tricks, and templates at your disposal. For that, please see this Mathpix Guide. You may want to occasionally check the link for any updates to the guide. For your convenience, you can copy this guide into your own instance of Mathpix Snip Web as follows:

  1. Click the Raw View button in the upper-right corner of your screen:
    Raw View button
  2. Press CMD+A to highlight all the Mathpix Markdown code.
  3. Press CMD+C to copy the highlighted code to your clipboard.
  4. Click the Rendered View button to return to the original view:
    Rendered View button
  5. Create a new Note and press CMD+V to paste the highlighted code into your Note.

You can right-click a math image in the Mathpix Guide to copy the LaTeX or MathML code for that image. You can also use the Chrome extension called Screen Reader to hear the math read aloud.

How did we do?

Creating Accessible Microsoft Files

Design and Development General Approach to Accessibility

Contact