Introduction

I write notes using Markdown and occasionally I need to share notes that explain concepts or progress with other people. While Markdown is easy to write, it is not suitable to be shared in such situations. The solution then is to convert the Markdown note to a more socially acceptable format which for my use cases is PDF or HTML. Often I prefer HTML for its superior presentation quality, wide portability through web browser support on a large array of devices—including mobile phones—and better support for embedding multimedia and other web based content. (It would not be the case if you had written a book of course.) Furthermore, many tools such as email clients or team messaging platforms may be able to render HTML natively in the application.

The translation of Markdown to the target format is accomplished by the Pandoc utility. The tool further satisfies the following requirements important for my use cases:

• Generate a standalone document by embedding any required resources into the final document. The receiving client must be able to render the document offline without running a webserver or the need to resolve other dependencies.
• Support for Mathematics using $\text{LATEX}$ markup syntax.
• Ability to modify elements of a document by access to its internal abstract syntax tree representation.

Pandoc allows to generate a PDF document in a simple straightforward manner. The math markup will require a working $\text{LATEX}$ distribution installed on the system which Pandoc will use to render the PDF. External resources such as images will be embedded in the PDF naturally.

Generating a static HTML document is a bit more involved due to the math requirement above. Math rendering on the web is typically done by offloading the render task to the client-side which will depend on some JavaScript math library. Mathematical content is static by nature and due to the offline requirement above, dynamic client-side rendering with JavaScript is not an option. To end up with static HTML math markup, the following approach will make use of the $\text{KATEX}$ library at compile time of the document (server-side math rendering).

I use a Bash shell script called rendernote to convert Markdown to the target formats mentioned above. The Git repository including that script and CSS style sheets is located here. It uses some Bash only features and was tested on a Linux system. Some commands in the script may not be available in this form on a BSD or MacOS system. The following sections explain some details for my Pandoc document conversion approach. All of the code is located in the rendernote script.

Markdown to PDF Translation

As mentioned above, translation to PDF is straightforward but requires a working $\text{LATEX}$ distribution. Pandoc will then make use of it when rendering Markdown to PDF. Alternative PDF engines are described in the documentation which may work but have not been tested. The render_pdf function in the rendernote script performs this task by calling

1pandoc \
2    --from markdown --to pdf \
3    --highlight-style pygments \
4    --output "${1%.*}.pdf" "${input}"

The --highlight-style option defines the pygments style for code highlighting. Pandoc further supports metadata headers for specific document settings which will be interpreted by the corresponding translation engine. The render_pdf function checks for the presence of such a (YAML) header and adds a default header in case none is found. This default header sets the page geometry, default heading font family and possibly other settings using standard $\text{LATEX}$ commands typically found in the preamble (specified by the header-include sequence) or at the beginning of a document (values in the include-before sequence). See the Pandoc man-page for further documentation. The default header in the script specifies the values:

1fontsize: 12pt
2papersize: a4
3linkcolor: blue
4header-includes:
5  - \usepackage[top=60pt,bottom=60pt,left=80pt,right=80pt]{geometry}
6  - \usepackage{bm}
7  - \usepackage{sectsty}
8include-before:
9  - \allsectionsfont{\sffamily}

This example turbulence.md Markdown document contains some common elements such as math, hyperlinks, block quotes, code blocks as well as images and can be converted to a turbulence.pdf document with the command

1rendernote -pdf turbulence.md

Markdown to Static HTML Translation

Pandoc supports options --standalone and --embed-resources. For PDF translation the former is implied and resources such as images are embedded into the PDF by default. For HTML translation the former ensures that the generated HTML file includes proper HTML markup (html, head and body tags) such that it could be viewed in a web browser by simply opening it. Rendering content or fetching remote CSS style sheets would still require a running webserver (for example python -m http.server). While the generated HTML file is standalone, it is not self-contained and may depend on external files such as images specified by a file system path or a network connection to fetch remote content. The --embed-resources option will attempt to fetch such external resources (local or remote) and encode them within the HTML file. Specifying both of these options will then result in a true self-contained HTML file for offline rendering at the cost of a larger file size due to embedded payload that ensures the file is self-contained.

Pandoc has several options for rendering math in HTML and supports $\text{KATEX}$ natively with the --katex option. Specifying this option in addition to the two options discussed above would be sufficient to produce a self-contained HTML file with support for math. However, the resulting file has several defects which make this solution approach undesirable:

• Math rendering is deferred to the client which introduces an unnecessary JavaScript dependency.
• The --embed-resources option will resolve this dependency by embedding a large amount of JavaScript code in the HTML file, which in turn blows up the file size unnecessarily.
• The client-side rendering introduces an overhead which may result in slow page loading performance for notes with significant amount of math.

For example, a tiny.md Markdown file with the content $a^2 + b^2 = c^2$ is 18 bytes in size. The tiny.html file generated with

1pandoc --to html --standalone --embed-resources --katex -o tiny.html tiny.md

is 1731663 bytes in size, almost 100'000 times the size of the Markdown file. The large file size is due to the encoded payload for JavaScript and font files required for rendering the math. The JavaScript dependency can be fully eliminated using server-side math rendering at the time the HTML document is created. Some further compression can be achieved by selecting only desired formats for font files (may not support all browsers however). To achieve this, the steps implemented in the rendernote script are:

1. Tag math in the input Markdown file with special labels by modifying the abstract syntax tree (AST) representation in Pandoc.
2. Generate rendered HTML for the labeled math using the $\text{KATEX}$ library in a Node.js application.
3. Convert the intermediate document to a self-contained HTML document.

These steps are implemented in the render_html function and are described in more detail in the sections below.

Math Pre-Processing using Lua Filters

Pandoc supports JSON and Lua filters which can be used for AST transformations of a given input. The idea for this pre-processing step is to wrap inline and display math in between HTML tags that can later be used to identify math nodes by parsing the HTML DOM. Fortunately, adding these tags can be achieved easily in Pandoc using a Lua filter. Such a filter is simply a Lua function with the same name as the node of the target object in the AST. Every node in AST with the same name as the filter will then be replaced with the return value of the filter call. The argument passed to the filter is the value of the currently existing node. Math objects in Pandoc are simply given the name Math. The filter used in the rendernote script to transform math nodes is given by the following Lua code:

 1function Math(elem)
 2    assert(FORMAT:match('html'))
 3    local wrap
 4    if elem.mathtype == 'InlineMath' then
 5        wrap = '<latexinline>' .. elem.text .. '</latexinline>'
 6    else
 7        wrap = '<latexdisplay>' .. elem.text .. '</latexdisplay>'
 8    end
 9    return pandoc.RawInline('html', wrap)
10end

The function must be named Math and takes one argument which will take the value of the current node the filter is applied to. All that this filter does is to wrap the $\text{LATEX}$ code contained in elem.text within HTML tags which are either <latexinline> for inline math or <latexdisplay> for display math. The pre-processed math is then returned in a new Pandoc node for raw HTML code (therefore, this filter only works for HTML targets). The rendernote script then generates intermediate HTML code for the Markdown input using Pandoc:

1pandoc \
2    --from markdown --to html \
3    --metadata title="$(basename --suffix=.md -- "${1}")" \
4    --highlight-style pygments \
5    --lua-filter "${lua_filter}" \
6    --template "${html_template}" \
7    "${1}" >"${raw_html}"

The Lua filter is defined in the file pointed to by the variable lua_filter. The Pandoc call further uses a HTML template stored in the file pointed to by html_template (see this command for the details) and writes the intermediate HTML to the file pointed to by raw_html. The example Markdown input

1Some inline math $a^2 + b^2 = c^2$ in a sentence.

is then filtered to HTML that looks like

1<body>
2<p>Some inline math <latexinline>a^2 + b^2 = c^2</latexinline> in a
3sentence.</p>
4</body>

For comparison, the default HTML generated without the Lua filter looks like the following:

1<body>
2<p>Some inline math <span class="math inline"><em>a</em><sup>2</sup> +
3<em>b</em><sup>2</sup> = <em>c</em><sup>2</sup></span> in a sentence.</p>
4</body>

The next step is to render the math in the intermediate HTML code using $\text{KATEX}$ .

Server-Side Math Rendering with KaTeX

If there were filtered math nodes in the AST, they will now be rendered to valid HTML using the renderToString function from the $\text{KATEX}$ API. This is done with a small JavaScript code in the render_latex function that is executed with Node.js. A here document is used for this which is fed into the node command using a pipe. The paths for the input and output HTML files are substituted in the here document with variable expansions. The JavaScript used for the server-side math rendering looks as follows:

 1const katex = require('katex');
 2const {parseHTML} = require('linkedom');
 3const fs = require('node:fs');
 4fs.readFile('${1}', 'utf8', (err, content) => {
 5    const {document} = parseHTML(content.toString());
 6    const inline_items = document.querySelectorAll('latexinline');
 7    const display_items = document.querySelectorAll('latexdisplay');
 8    inline_items.forEach((item) => {
 9        const katex_code = katex.renderToString(item.innerHTML, {
10            output: 'html',
11            displayMode: false,
12        });
13        item.outerHTML = katex_code;
14    });
15    display_items.forEach((item) => {
16        const katex_code = katex.renderToString(item.innerHTML, {
17            output: 'html',
18            displayMode: true,
19        });
20        item.outerHTML = katex_code;
21    });
22    fs.writeFile('${1}', document.toString(), err => {});
23});

This reads the intermediate HTML file from the previous step and parses it into HTML DOM using the linkedom package (line 5). The custom tags for the math nodes can then be queried and processed with forEach loops where the katex.renderToString function is used to replace the tags with valid rendered HTML math. Finally, the processed HTML is written back to the same file as specified for the input, which is OK since fs.readFile reads the full file into memory before the callback is executed. The function further downloads the latest katex and linkedom modules using npm. These modules will be stored in the directory where the rendernote script is located and are only downloaded if they do not exist.

Encoding Math Fonts using Data URIs

After the rendering pass, the static_katex function is executed to generate static CSS and font data for $\text{KATEX}$ . The function fetches the data from a CDN server and stores it in a static directory relative to the location of the rendernote script. The function creates static font files by converting the CDN fonts to base64 encoded data URIs. It only fetches woff fonts by default. Alternatively, other font formats could be defined here. $\text{KATEX}$ related CSS is stored in the directory static/css/katex/${katex_version}, where ${katex_version} is determined by npm.

Creating the Final Standalone HTML File

The final step is to run Pandoc in a second pass with the intermediate HTML code as input. This time the --standalone and --embed-resources options are passed to Pandoc as well. The same HTML template is used for this second pass as was already used during the first pass. In addition, this call adds the --include-in-header option to pass the CSS style sheets prepared for the final document. These style sheets include a default style sheet and possibly $\text{KATEX}$ related style sheets discussed in the previous section. A Markdown document without math will only include the default style sheet.

Following up with the same tiny.md example used at the beginning of the Markdown to Static HTML Translation section, the command

1rendernote tiny.md

generates a tiny.html file with a size of 431666 bytes that is 4 times smaller than the standalone HTML file generated by the native Pandoc approach. The file is further free of JavaScript. While almost half a megabyte is still quite large for such tiny content, 94% of the total file size is attributed to the base64 encoded math fonts included in the payload of the standalone HTML file. Removing the inline math $ markers from the tiny.md file results in a file that is only 4282 bytes in size.

Finally, the example turbulence.pdf file generated in the previous section Markdown to PDF Translation amounts to 3.7 MB. The same Markdown input converted to HTML amounts to 2.7 MB (versus 3.9 MB using the native Pandoc approach). The payload for the two images and encoded math fonts corresponds to 74% (2.0 MB). For comparison, the standalone turbulence.html version can be viewed by following the link. The file is created with the command

1rendernote turbulence.md

Introduction

My workflow is command-line driven and the tools I use to process email are either local daemons, scripts or some ncurses based application. Conceptually, my email data flow looks like this:

A new account required communication with a Microsoft Exchange server on which the IT department disabled POP and IMAP protocols. Since neither getmail nor mutt know how to do that, the additional davmail node in the diagram above was required to be added. Davmail runs as a daemon that enables POP and SMTP protocols on the localhost which it then translates to the Microsoft Exchange protocol. Apart from POP and SMTP, Davmail also supports IMAP, LDAP, CalDav and CardDav protocols. With the Davmail proxy in place, any application that is using these protocols can communicate with a server that is using the Microsoft Exchange protocol.

Davmail Configuration

The following configuration is intended to use Davmail as a server daemon mainly using POP, SMTP and CalDav protocols. Currently evolution-ews is one of the few email client implementations on Linux that work with Microsoft Exchange services. The IT department which manages the Microsoft Exchange server I am connecting to supports this application and typically there is a high level of resistance when asked to support another one.

Authentication on the remote is done with OAuth2. Following the path of least resistance, I am using the clientId and tenantId intended for evolution-ews. This information can typically be found on the help pages provided by the IT department. The persistToken flag (its default value is true) will append a refresh token at the end of the properties file after connecting to the remote server for the first time. The token will be used for automatic authentication for subsequent connections to the server. The steps to obtain a refresh token are the following:

0. If davmail is already running as a daemon, use systemctl to stop the service.
1. Set davmail.mode=O365Manual in your davmail properties file.
2. Run davmail manually with the properties file as argument.
3. Force a connection on the port davmail is listening, for example by fetching mail or sending an email with msmtp.
4. Follow the instructions in the stdout found in the terminal where davmail was started.
5. Set davmail.mode=O365Modern in your davmail properties file and restart the systemd service. From now on authentication happens via the token that has been appended in the properties file used in item 2.

By default, communication with the local Davmail daemon is not encrypted which is fine in most cases (it can be changed if desired, see the documentation).

Since local communication is not encrypted, you would technically not need a password when authenticating with Davmail. Nevertheless, you should still use a password when connecting to Davmail with your local applications. Davmail will use this password to encrypt the refresh token with a symmetric AES cipher before it is written to the properties file (you have to use the same password with all your local applications for this to work). Additionally, you may want to change the file access permissions of the properties file for further security measures.

The listener ports can be chosen freely from the available user ports. For initial setup and testing, the logging level in line 51 may need to be set to DEBUG. In order to get a refresh token, you will need to paste an access code that is obtained from the Microsoft Exchange server back into Davmail. To achieve this, execute Davmail directly from the command-line with a properties file argument where the mode is set to O356Manual. Once the initial connection was successful the refresh token will be appended to the file and the mode can be set back to O356Modern for subsequent automatic authentication running Davmail as a systemd user service.

 1# server and mode settings
 2davmail.server=true
 3davmail.enableKeepAlive=true
 4# STEPS:
 5# 0. Adjust davmail.logFilePath below.
 6# 1. Use O365Manual for initial connection (this should append a refresh token
 7#    to this file, davmail must be executed from the command line).
 8# 2. Use O365Modern for headless server (e.g. systemd service) operation.
 9#    Obtaining an access token should now be automated until refresh token
10#    expires.
11davmail.mode=O365Manual
12# davmail.mode=O365Modern
13davmail.url=https://outlook.office365.com/EWS/Exchange.asmx
14
15# oauth evolution mock settings
16davmail.oauth.clientId=<client id>
17davmail.oauth.tenantId=common
18davmail.oauth.redirectUri=https://login.microsoftonline.com/common/oauth2/nativeclient
19davmail.oauth.persistToken=true
20
21# listener ports
22davmail.caldavPort=5000
23davmail.popPort=5001
24davmail.imapPort=5002
25davmail.ldapPort=5003
26davmail.smtpPort=5004
27
28# network proxy settings
29davmail.enableProxy=false
30davmail.useSystemProxies=false
31davmail.allowRemote=false
32
33# disable SSL for specified listeners
34davmail.ssl.nosecurecaldav=false
35davmail.ssl.nosecureimap=false
36davmail.ssl.nosecureldap=false
37davmail.ssl.nosecurepop=false
38davmail.ssl.nosecuresmtp=false
39
40# POP settings
41davmail.keepDelay=0
42davmail.sentKeepDelay=0
43davmail.popMarkReadOnRetr=false
44
45# SMTP settings
46davmail.smtpSaveInSent=false
47
48# logging
49davmail.logFilePath=/path/to/logfile
50log4j.rootLogger=WARN
51log4j.logger.davmail=INFO
52
53# davmail appended items

Patching Davmail for Forced Message Deletion on Remote

I prefer POP to permanently delete messages when successfully fetched such that none of my email resides on external mail servers (on mobile devices I typically use IMAP to receive messages while on the go). Also note that smtpSaveInSent is set to false in the Davmail configuration above which will prevent saving a copy of sent mail on the server. Default Davmail code is conservative and moves messages to trash when it receives the DELE command via POP (which is probably the sensible thing to do in most cases). Permanent message deletion on the server can be enforced by applying this patch to the source code. For Arch Linux users, an AUR package that applies this patch during build can be obtained with

1git clone https://git.0xfab.ch/davmail-git.git

SMTP Configuration in Mutt

Local communication with Davmail is not encrypted and SMTP configuration in clients such as mutt must be setup accordingly. In particular, SSL settings must be disabled for the account that connects to Davmail:

1set ssl_starttls  = "no"
2set ssl_force_tls = "no"
3set smtp_url      = "smtp://<username>@localhost:<davmail smtp port>/"

Test Kernel

The kernel I am going to benchmark is a variation of the general matrix-vector multiplication (GEMV) found in BLAS level 2. The simplified kernel takes the form

where summation over index $j$ is assumed. The values $x_i$ and $y_i$ are elements of vectors $\mathbf{x},\mathbf{y}\in {\mathbb{R}}^n$ , respectively, and $a_{ij}$ are elements of matrix $A\in {\mathbb{R}}^{n\times n}$ for some integer number $n>1$ . For typical problems $n$ is large.

Operational Intensity

The operational intensity¹ of a compute kernel is the ratio between arithmetic operations and total number of bytes transferred due to memory accesses. To simplify the discussion, I am assuming the destination of all memory transactions is DRAM. For the simplified GEMV kernel above, there are $n^2$ multiplications and $n(n-1)+n$ additions for a total of $2n^2$ flop. The memory accesses to DRAM are trickier to estimate because of cache memory hierarchies found in all modern CPU architectures. An upper bound can be estimated by assuming no cache, where every memory access results in a capacity miss.² In that case there are $n$ reads for $y_i$ , $n^2$ reads for the matrix elements $a_{ij}$ , $n^2$ reads for $x_i$ due to the $n$ inner products and another $n$ writes for $y_i$ , resulting in a total of $2n^2+2n$ memory accesses. For a lower bound estimate we assume an infinite cache where only compulsory misses are relevant.² Here every required read and write is performed exactly once, resulting in a total of $n^2+3n$ memory accesses. For both estimates the leading order term of memory accesses is $n^2$ , indicating that the GEMV kernel is memory bound. Indeed, all BLAS level 1 and level 2 kernels are memory bound.

Since $n$ is assumed large, it is reasonable to pick the upper bound memory access estimate, where the cache capacity is assumed small compared to the problem size. Assuming 32-bit single precision floating point data, the operational intensity for this test kernel computes as

The approximate value indicated corresponds to the expected value in the limit when $n$ is large.

Naive Implementation

A straightforward implementation of the GEMV kernel discussed in the previous section could look like:

1void gemv(const float *A, const float *x, float *y, const size_t n)
2{
3    for (size_t i = 0; i < n; ++i)
4        for (size_t j = 0; j < n; ++j)
5            y[i] += A[i * n + j] * x[j];
6}

Assuming the code is built for the same target architecture, a Fortran compiler is generally able to emit more efficient code than a C/C++ compiler for this particular implementation. This is because a Fortran compiler assumes that array y must not alias x or A . A C/C++ compiler does not make this assumption which will prevent it from performing certain optimizations during code generation. It is often the reason for the claim that Fortran is faster than C/C++.

Strict Aliasing Rule in C/C++

The concept of pointers in certain programming languages allows to declare variables that “point to” a specific memory location by assigning to it a corresponding memory address. This concept does generally not prevent any two pointers from pointing to the same location in memory. It may also be possible that these pointers point to ranges in memory that overlap. Pointers do not exist in earlier releases of Fortran and so aliasing of memory regions is not a concern. This is not true for programs written in C/C++. However, to limit the scope of possible aliasing, some rules apply broadly known as strict aliasing:

Pointers with incompatible types (e.g. int and float) do not alias in memory. Dereferencing a pointer that points to an incompatible type is undefined behavior.

Consider the following function for which the strict aliasing rule applies to the pointer parameter a and b:

1int foo(int *a, int *b)
2{
3    *a = 1;
4    *b = 2;
5    return *a; /* what is the value of *a? */
6}

The compiler does not know a priori where a and b will point to when foo is called. The return value may be 1 or 2 depending on what arguments are passed into foo. If pointers alias, the compiler cannot reorder read and write operations which inflicts a performance penalty in cases where strict aliasing is not desired. In the example above, the compiler must issue a read in line 5 because the write in line 4 may change its value assigned in line 3. If the pointers do not alias, the additional read instruction is not necessary. The code below shows the optimized assembly code generated by GCC 14.2.1 (note that the -fstrict-aliasing flag is implied for -O2, -O3 and -Os optimizations). The values of pointers a and b are stored in registers rdi and rsi, respectively. If the values in these two registers are the same, line 2 and 3 below write to the same memory location and consequently the compiler is required to perform a fresh read from address rdi when moving the result into return register eax in line 4. A total of three memory operations are required to ensure correctness of this optimized code.

1foo: ; gcc 14.2 with flags -O3 -fstrict-aliasing
2        mov     DWORD PTR [rdi], 1   ; *a = 1
3        mov     DWORD PTR [rsi], 2   ; *b = 2
4        mov     eax, DWORD PTR [rdi] ; store *a in register eax
5        ret

The compiler must be conservative with respect to optimizations when aliasing is possible. A performance trade-off for the dynamic flexibility that pointers offer.

Sometimes strict aliasing may not be desired, especially when performance is a concern (the compiler must have the freedom to reorder memory operations). In that case a pointer can be qualified as restrict which then establishes a contract between the compiler and the programmer that allows to transfer responsibility to the programmer.³ More freedom (for the person that writes the code) typically means more responsibility. The restrict type qualifier tells the compiler that a pointer will not alias other memory in the block scope it is defined (from the compilers’ perspective it is more of a liberation than restriction). If we agree (assuming understood) with this contract, we can rewrite the code using the restrict type qualifier in the public API of the function

1int foo(int *restrict a, int *restrict b)
2{
3    *a = 1;
4    *b = 2;
5    return *a; /* the value of *a is definitely 1! */
6}

Because of our contract, the compiler can be absolutely certain that the return value is 1. If something else would be expected, then it is not the compiler’s fault. The optimized assembly now looks as follows:

1foo: ; gcc 14.2 with flags -O3 -fstrict-aliasing
2        mov     DWORD PTR [rdi], 1   ; *a = 1
3        mov     eax, 1               ; return value is 1, no question!
4        mov     DWORD PTR [rsi], 2   ; *b = 2
5        ret

This code requires only two memory operations compared to the three required when the strict aliasing rule must be enforced. Aliasing requires additional memory transactions (write instructions) to guarantee correct code. These additional memory operations are one of the causes for performance degradation in C/C++. Fortran does not have to deal with this.

Memory Aware Implementation

The naive implementation of the GEMV kernel shown at the beginning of the previous section is subject to the strict aliasing rule. The writes to memory locations pointed to by y may overwrite memory that is subsequently read by A and/or x. Note that the const qualifier applied to these two pointers in the function signature is not sufficient to prevent from aliasing. The optimized assembly code for this naive implementation looks like:

 1gemv: ; gcc 14.2 with flags -O3 -ftree-vectorize
 2        test    rcx, rcx
 3        je      .L1
 4        lea     r8, [0+rcx*4]
 5        lea     r9, [rdx+r8]
 6.L3:
 7        movss   xmm1, DWORD PTR [rdx]
 8        xor     eax, eax
 9.L4:
10        movss   xmm0, DWORD PTR [rdi+rax*4]
11        mulss   xmm0, DWORD PTR [rsi+rax*4]
12        add     rax, 1
13        addss   xmm1, xmm0
14        movss   DWORD PTR [rdx], xmm1
15        cmp     rcx, rax
16        jne     .L4
17        add     rdx, 4
18        add     rdi, r8
19        cmp     r9, rdx
20        jne     .L3
21.L1:
22        ret

The highlighted section of lines 9–16 corresponds to the inner-most loop. This code iterates over index $j$ to compute the inner-product $a_{ij}{x}_j$ and add the result to $y_i$ . For every $j$ , the value $a_{ij}$ is loaded into register xmm0 (line 10), then multiplied with $x_j$ (line 11, the second operand is loaded from memory) and finally added to $y_i$ which is represented by the accumulator register xmm1 in line 13. Because of the strict aliasing rule, the compiler is required to emit a store instruction in line 14 that writes the current value of $y_i$ back to memory before continuing with reading again in lines 10 and 11. This store is unnecessary if we can guarantee that y does not alias A and/or x (these are typically distinct arrays for the GEMV operation). Furthermore, note that this code is trivial to vectorize but pointer aliasing prevents the compiler from doing this optimization. This is another cause for performance degradation in C/C++ when pointer aliasing is present. Slight modifications to this kernel implementation will help the compiler emit more efficient code.

Qualify the pointer to the destination array as restrict

As seen in the previous section, one simple way to fix this problem is to add the restrict qualifier to the pointer argument for the destination array y (the one being written to). Changing the function signature of the previous GEMV implementation to

1void gemv(const float *A, const float *x, float *restrict y, const size_t n)

removes the unnecessary store instruction in the inner-most loop and allows the compiler to emit SIMD instructions for vectorized code. The assembly code for the inner-most loop now looks like:⁴

 1.L4: ; gcc 14.2 with flags -O3 -ftree-vectorize
 2        movups   xmm0, XMMWORD PTR [rsi+rax]
 3        movups   xmm3, XMMWORD PTR [rdx+rax]
 4        add      rax, 16
 5        mulps    xmm0, xmm3
 6        movaps   xmm1, xmm0
 7        addss    xmm1, xmm2
 8        movaps   xmm2, xmm0
 9        shufps   xmm2, xmm0, 85
10        addss    xmm1, xmm2
11        movaps   xmm2, xmm0
12        unpckhps xmm2, xmm0
13        shufps   xmm0, xmm0, 255
14        addss    xmm1, xmm2
15        movaps   xmm2, xmm0
16        addss    xmm2, xmm1
17        cmp      rax, rcx
18        jne      .L4

Lines 2 and 3 perform 128-bit loads for $x_j$ and $a_{ij}$ , respectively, and store the result in registers xmm0 and xmm3. These are vector loads and operate on 4 32-bit floats simultaneously (4-way SSE SIMD for the given compiler flags, other SIMD instruction set extensions would generate a similar code path for the given compiler flags). Line 5 then executes a vector multiplication for the product $a_{ij}{x}_j$ and lines 5–16 correspond to a horizontal reduction into the accumulator register xmm2 (the value $y_i$ ). The required store instruction that was observed for the case with strict aliasing is optimized away for this implementation.

The auto-vectorized code generated by GCC 14.2.1 is not ideal because it performs the horizontal reduction for every loop iteration $j$ . A better approach would be to use a vector register for the accumulation of $a_{ij}{x}_j$ products using FMA instructions and then perform the horizontal reduction of the accumulator register into $y_i$ once after the $j$ accumulation steps. For comparison, compiling this code using Clang 19.1.0 with the same optimization flags generates different scalar (non-SIMD) code with 4-way loop-unrolling.

Use Thread-Private Memory

For the GEMV algorithm discussed here, using the restrict qualifier on the destination array y is not necessary. The strict aliasing rule is not a problem for this kernel if we write the code in a way that will impose our intent more clear on the compiler. In the initial implementation, it is clear that the loop index i is constant in the inner-most loop. There is no need at all to index into array y for every iteration j.

1void gemv(const float *A, const float *x, float *y, const size_t n)
2{
3    for (size_t i = 0; i < n; ++i)
4        for (size_t j = 0; j < n; ++j)
5            y[i] += A[i * n + j] * x[j];
6}

Instead of writing the initial code as shown above, it is better to use an accumulator for which the compiler will allocate a register (which is thread-private). There is no need to use a shared resource such as y[i] to perform the accumulation. A better GEMV implementation would thus be

 1void gemv(const float *A, const float *x, float *y, const size_t n)
 2{
 3    for (size_t i = 0; i < n; ++i) {
 4        float inner_product = 0.0f;
 5        for (size_t j = 0; j < n; ++j) {
 6            inner_product += A[i * n + j] * x[j];
 7        }
 8        y[i] += inner_product;
 9    }
10}

Note that no restrict qualifier is used for this implementation. The resulting assembly code is identical to the one obtained using the restrict qualified destination array as in the previous section. The code is more elegant because intention is expressed clearly by the use of an accumulator in thread-private memory. Using thread-private memory whenever possible is furthermore important for writing multi-threaded code where race-conditions may be an issue.

Fortran Implementation

Now it is time to compare against the Fortran equivalent of the GEMV kernel. Because Fortran is using column-major storage, it is assumed that the matrix $A$ is symmetric for the remainder of this post. The following code is taken from the file sgemv.f of the reference BLAS 3.12.0 library with some code stripped off to meet the simplified GEMV variant discussed in the Test Kernel section:

 1      SUBROUTINE SGEMV(A,X,Y,N)
 2      INTEGER(4) N,I,J
 3      REAL(4) A(N,*),X(*),Y(*)
 4      REAL(4) TEMP
 5#ifdef TRANSPOSE
 6! Form  y := A^T*x + y.
 7      DO 100 J = 1,N
 8      TEMP = 0.0
 9      DO 90 I = 1,N
10      TEMP = TEMP + A(I,J)*X(I)
11   90             CONTINUE
12      Y(J) = Y(J) + TEMP
13  100         CONTINUE
14#else
15! Form  y := A*x + y.
16      DO 60 J = 1,N
17      TEMP = X(J)
18      DO 50 I = 1,N
19      Y(I) = Y(I) + TEMP*A(I,J)
20   50             CONTINUE
21   60         CONTINUE
22#endif
23      RETURN
24      END

As mentioned above, the equivalent Fortran algorithm is when TRANSPOSE is defined (lines 7–13). Compiling this code with GFortran 14.2.1 results again in identical assembly code as obtained for the memory aware C/C++ versions. Different for Fortran of course is the absence of aliasing, such that a loop structure without the TEMP accumulator

1      DO 100 J = 1,N
2      DO 90 I = 1,N
3      Y(J) = Y(J) + A(I,J)*X(I)
4   90             CONTINUE
5  100         CONTINUE

still yields identical assembly code.

Benchmark

In this section I am going to benchmark the different C/C++ implementations discussed above and compare to the BLAS Fortran implementation. The main metric of interest is the operational intensity discussed in a previous section. The necessary instruction counts for flops and memory accesses are determined using hardware counters via the PAPI library. The benchmark results shown below are obtained using an Intel Core i7-8700K CPU running at 3.70 GHz and the problem size is set to $n=10000$ . The benchmark code can be found here.

Assuming the upper bound estimate discussed in an earlier section, a total of $2n^2+2n=200020000$ memory instructions are to be expected. The naive C/C++ kernel with strict aliasing generates the following benchmark output (compiled with GCC 14.2.1 and -O2 flag):

 1Total cycles:                 427373621
 2Total instructions:           700071682
 3Instructions per cycle (IPC): 1.64
 4L1 cache size:                32 KB
 5L2 cache size:                256 KB
 6L3 cache size:                12288 KB
 7Total problem size:           390703 KB
 8Total L1 data misses:         12515647
 9Total load/store:             300010748 (expected: 200020000)
10Operational intensity:        1.666690e-01 (expected: 2.499875e-01)
11Performance [GFlop/s]:        2.204652e+00
12Wall-time   [micro-seconds]:  9.072179e+04

The measured number of load/store instructions exceeds the expected value by 100 Million because of the additional store instruction required in the inner-most loop due to the strict aliasing rule (see Memory Aware Implementation). The additional store instructions reduce the observed operational intensity accordingly. The measurement for the kernel without strict aliasing produces the expected values (compiled with GCC 14.2.1 and -O2 flag):

 1Total cycles:                 421014858
 2Total instructions:           600101681
 3Instructions per cycle (IPC): 1.43
 4L1 cache size:                32 KB
 5L2 cache size:                256 KB
 6L3 cache size:                12288 KB
 7Total problem size:           390703 KB
 8Total L1 data misses:         12520995
 9Total load/store:             200020748 (expected: 200020000)
10Operational intensity:        2.499866e-01 (expected: 2.499875e-01)
11Performance [GFlop/s]:        2.237932e+00
12Wall-time   [micro-seconds]:  8.937267e+04

The roofline plot below summarizes measurements obtained for single core execution using different compilers as well as the C and Fortran implementations of the GEMV test kernel. The following compilers have been used on an Arch Linux system with kernel 6.12.4:

All executables are built using -O2 optimization targeting scalar code without SIMD (AVX2) or FMA instructions (additional flags used with the Nvidia Fortran compiler are -acc=host and -mno-fma):

The roofline shows that none of the test executables reach the hardware limit for the given level of optimization. The C implementations correspond to the gcc-14 and clang-18 legend entries, where the naive implementations are indicated with strict aliasing. The executable generated with the Nvidia Fortran compiler exhibits 4x more load/store instructions than what is expected which results in slightly worse performance compared to the other test executables. (It is not clear why the compiler does this but removing the -mno-fma flag immediately results in aggressively optimized code, see next section.)

The overall performance for all test executables is roughly identical at this level of optimization, irrespective of the strict aliasing rule. Given the upper bound estimate for the operational intensity of the GEMV test kernel, the fastest executable is limited by the bandwidth ceiling which corresponds to about 10.4 GFlop/s on this Intel architecture. A C compiler that is compliant with the strict aliasing rule (even when aggressive optimizations are enabled) is unlikely to generate better code than what is observed for gcc-14 above. Therefore, the naive C implementation of the GEMV test kernel is expected to exhibit 5x slower performance due to the strict aliasing rule in C/C++.

With Compiler Optimization

More aggressive optimizations can be enabled by allowing the compiler to use SIMD instructions via auto-vectorization as well as FMA instructions and other optimization techniques of which some may disregard strict standard compliance. The optimization flags used for the following measurements are:

These are rather aggressive optimization flags which may not always be suitable for production code. The benchmark output for the C kernel implementation with strict aliasing now looks like (compiled with GCC 14.2.1 and the optimization flags for gcc-14 shown in the table above):

 1Total cycles:                 421750341
 2Total instructions:           337611682
 3Instructions per cycle (IPC): 0.80
 4L1 cache size:                32 KB
 5L2 cache size:                256 KB
 6L3 cache size:                12288 KB
 7Total problem size:           390703 KB
 8Total L1 data misses:         12520574
 9Total load/store:             300010663 (expected: 200020000)
10Operational intensity:        1.666691e-01 (expected: 2.499875e-01)
11Performance [GFlop/s]:        2.230897e+00
12Wall-time   [micro-seconds]:  8.965453e+04

The total number of cycles remains about the same while the total number of instructions executed has roughly halved due to optimization passes, reducing the average number of instructions per cycle (IPC) by the same factor. The number of load/store instructions remains the same, indicating that the optimized version is still issuing scalar load/stores which is a consequence of the strict aliasing rule. The numbers show that the GNU C compiler honors the strict aliasing rule even when aggressive optimizations are enabled and thus does not have much freedom to optimize the code. Furthermore, IPC is not an accurate metric for this memory bound code.

The benchmark output for the C kernel implementation using thread private memory for the loop accumulator looks like the following (compiled with GCC 14.2.1 and the optimization flags for gcc-14 shown in the table above):

 1Total cycles:                 83715295
 2Total instructions:           30031680
 3Instructions per cycle (IPC): 0.36
 4L1 cache size:                32 KB
 5L2 cache size:                256 KB
 6L3 cache size:                12288 KB
 7Total problem size:           390703 KB
 8Total L1 data misses:         12537289
 9Total load/store:             25020760 (expected: 200020000)
10Operational intensity:        1.998440e+00 (expected: 2.499875e-01)
11Performance [GFlop/s]:        1.121076e+01
12Wall-time   [micro-seconds]:  1.784090e+04

Here the compiler was able to use AVX2 SIMD instructions that operate on 256-bit vector registers. Since the test code is using 32-bit (single precision) floating point data there are 8 SIMD lanes per register. This is the reason for 8x fewer load/store instructions since each load/store instruction operates on 8 elements simultaneously. The fewer memory instructions obfuscate the observed operational intensity which appears to be 8x larger than expected (because it is calculated assuming scalar memory instructions). The total number of bytes transferred remains the same however, which is the metric used to define the denominator of the operational intensity.

The roofline for the measurements with optimized executables looks like this:

The optimized executables reach the hardware ceiling with the exception of the naive C implementation that enforces the strict aliasing rule. It is interesting to note that the latter only applies to the GNU C compiler, the LLVM C compiler seems to ignore the strict aliasing rule for the optimization flags specified in the table above (which is one of these optimizations that is not compliant with the standard mentioned above). These results show that the performance limiting factor should be the hardware rather than the programming language or compiler. Therefore, Fortran is not faster than C.