How it Works

Syntax highlighters for the web all follow the same principles.

1. Identify the code elements in the DOM to evaluate by inspecting the existence of a language- class
2. Tokenize the text content inside the code element
3. Convert each token into an individual span element and apply a class such as token keyword or token operator
4. Replace the innerHTML content of the code element with the set of spans for each token

The Phio highlighter applies the same CSS classes as the Prism highlighter which means that any Prism CSS stylesheet can also be used with the Phio highlighter. A number of available themes can be found here : https://cdnjs.com/libraries/prism-themes

An example of how the Prism CSS classes are applied to token types. This will apply the color to all span elements that have the token operator class.

.token.operator {
	color: #d4d4d4
}

You can also test different themes at : https://phioautomation.com/highlighter

Tokenizing Highlighters

While the majority of web syntax highlighters are regex based, this often introduces edge cases that are not captured or context that a tokenizer does not have.

One such example with the Prism highlighter (which currently highlights the docs site). This does not properly capture that underscores are allowed within numbers.

a : INT := 2_000;

An example where a tokenizing highlighter would lack context would be in defining a type. Often types, including built-in types are highlighted the same color. A simple tokenizer, however, sees MyUdt as just another identifier.

TYPE MyUdt
  a : INT;
END_TYPE

Phio Highlighter

Lezer is a parser generator designed to generate a tree structure that can be used for syntax highlighting. This parser additionally plugs in to CodeMirror which runs the editor at https://phioautomation.com.

Given a grammar for the IEC Structured Text language, Lezer is able to generate a Javascript module for an LR parser. With an additional highlightCode function, this provides everything necessary for a syntax highlighter (without needing Codemirror)

Tradeoffs

Of course, in order to ship a complete parser, the bundle size is slightly larger.

Higlighter	Bundle size (kB)	Source
Phio	26.9	https://highlighter.phioautomation.com/iecst.js
Prism	4.6	https://highlighter.phioautomation.com/prism.js
Prettify	68.3 + 6.6	https://infosys.beckhoff.com/includes/prettify/prettify.js
Highlight.js	36.7	https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js

This feels like a reasonable price to pay given :

• A complete grammar can describe all aspects of the language and provide similar highlighting to IDEs
• Prism.js is no longer accepting PRs to address updates to v1