Posts tagged "haskell" on abhinavsarkar.nethttps://abhinavsarkar.net/tags/haskell/feed.atom2023-06-03T00:00:00ZAbhinav Sarkarhttps://abhinavsarkar.net/about/abhinav@abhinavsarkar.nethttps://abhinavsarkar.net/images/favicon.ico© 2017–2023, Abhinav Sarkarhttps://abhinavsarkar.net/posts/implementing-co-4/Implementing Co, a Small Language With Coroutines #4: Adding Channels2023-06-03T00:00:00ZAbhinav Sarkarhttps://abhinavsarkar.net/about/abhinav@abhinavsarkar.net<p>In the <a href="https://abhinavsarkar.net/posts/implementing-co-3/?mtm_campaign=feed">previous post</a>, we added coroutines to <span class="fancy">Co</span>, the small language we are implementing in this series of posts. In this post, we add channels to it to be able to communicate between coroutines.</p>
<ol type="1">
<li><a href="https://abhinavsarkar.net/posts/implementing-co-1/?mtm_campaign=feed">Implementing Co #1: The Parser</a></li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-2/?mtm_campaign=feed">Implementing Co #2: The Interpreter</a></li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-3/?mtm_campaign=feed">Implementing Co #3: Adding Coroutines</a></li>
<li>Implementing Co #4: Adding Channels</li>
<li>Implementing Co #5: Adding Sleep</li>
</ol>
<p>This post was originally published on <a href="https://abhinavsarkar.net/posts/implementing-co-4/?mtm_campaign=feed">abhinavsarkar.net</a>.</p><!--more-->
<nav id="toc" class="right-toc"><h3>Contents</h3><ol><li><a href="#introduction">Introduction</a></li><li><a href="#channel-design">Channel Design</a></li><li><a href="#channel-operations">Channel Operations</a></li><li><a href="#adding-channels">Adding Channels</a></li><li><a href="#wiring-channels">Wiring Channels</a></li><li><a href="#sending-and-receiving">Sending and Receiving</a></li><li><a href="#pubsub-using-channels">Pubsub using Channels</a></li><li><a href="#bonus-round-emulating-actors">Bonus Round: Emulating Actors</a></li></ol></nav>
<h2 data-track-content data-content-name="introduction" data-content-piece="implementing-co-4" id="introduction">Introduction</h2>
<p>With coroutines, we can now have multiple <em>Threads of Computation</em> (ToCs) in a <span class="fancy">Co</span> program. However, right now these <abbr title="Threads of computation">ToCs</abbr> work completely independent of each other. Often in such concurrent systems, we need to communicate between these <abbr title="Threads of computation">ToCs</abbr>, for example, one coroutine may produce some data that other coroutines may need to consume. Or, one coroutine may need to wait for some other coroutine to complete some task before it can proceed. For that, we need <a href="https://en.wikipedia.org/wiki/Synchronization_(computer_science)#Thread_or_process_synchronization" target="_blank" rel="noopener">Synchonization</a> between coroutines.</p>
<p>There are various ways to synchronize <abbr title="Threads of computation">ToCs</abbr>: <a href="https://en.wikipedia.org/wiki/Lock_(computer_science)" target="_blank" rel="noopener">Locks</a>, <a href="https://en.wikipedia.org/wiki/Semaphore_(programming)" target="_blank" rel="noopener">Semaphores</a>, <a href="https://en.wikipedia.org/wiki/Futures_and_promises" target="_blank" rel="noopener">Promises</a>, <a href="https://en.wikipedia.org/wiki/Actor_model" target="_blank" rel="noopener">Actors</a>, <a href="https://en.wikipedia.org/wiki/Channel_(programming)" target="_blank" rel="noopener">Channels</a>, <a href="https://en.wikipedia.org/wiki/Software_transactional_memory" target="_blank" rel="noopener">Software Transactional Memory</a>, etc. In particular, channels are generally used with coroutines for synchronization in many languages like <a href="https://gobyexample.com/channels" target="_blank" rel="noopener">Go</a>, <a href="https://kotlinlang.org/docs/coroutines-and-channels.html" target="_blank" rel="noopener">Kotlin</a>, <a href="https://docs.python.org/3/library/asyncio-queue.html" target="_blank" rel="noopener">Python</a> etc, and we are going to do the same.</p>
<p><a href="https://en.wikipedia.org/wiki/Channel_(programming)" target="_blank" rel="noopener"><em>Channels</em></a> are a synchronization primitive based on <a href="https://en.wikipedia.org/wiki/Communicating_Sequential_Processes" target="_blank" rel="noopener">Communicating Sequential Processes</a><sup><a href="#ref-Hoare1986-ih" class="citation" title="Hoare, Communicating Sequential Processes.
">@1</a></sup> (CSP). <abbr title="Communicating Sequential Processes">CSP</abbr> is a formal language for describing patterns of interaction between concurrent processes. In <abbr title="Communicating Sequential Processes">CSP</abbr>, processes communicate with each other by sending and receiving messages over channels.</p>
<p>A process can send a message to a channel only if the channel is not full, and blocks otherwise. Similarly, a process can receive a message from a channel only if the channel is not empty, blocking otherwise. Thus, channels provide a way for processes to synchronize with each other, and at the same time, communicate by passing messages.</p>
<p>Before we implement channels, we have to decide how they are going to work.</p>
<h2 data-track-content data-content-name="channel-design" data-content-piece="implementing-co-4" id="channel-design">Channel Design</h2>
<p>There are various design decisions that we need to make while implementing channels. Depending on what we choose, we end up with different kinds. Some of the major design decisions are:</p>
<dl>
<dt><em>Buffered</em> vs <em>Unbuffered</em></dt>
<dd>
A buffered channel has a buffer to store messages. A send operation on a buffered channel succeeds if the buffer is not full, even if there are no pending receive operations. On the other hand, a send operation on an unbuffered channel blocks until the message is received by some other process. For example, in Java <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/LinkedBlockingQueue.html" target="_blank" rel="noopener"><code>LinkedBlockingQueue</code></a> is a buffered channel, while <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/SynchronousQueue.html" target="_blank" rel="noopener"><code>SynchronousQueue</code></a> is an unbuffered channel<a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a>.<br />
<br />
</dd>
<dt><em>Bounded</em> vs <em>Unbounded</em></dt>
<dd>
A bounded channel has a buffer of fixed capacity, and can hold only a fixed number of messages at maximum. A send operation on a bounded channel blocks if the buffer is full and there are no pending receive operations. An unbounded channel has a buffer with no fixed capacity, and can hold any number of messages. A send operation on an unbounded channel never blocks. For example, in Java <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/ArrayBlockingQueue.html" target="_blank" rel="noopener"><code>ArrayBlockingQueue</code></a> is a bounded channel, while <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/LinkedBlockingQueue.html" target="_blank" rel="noopener"><code>LinkedBlockingQueue</code></a> is an unbounded one.<br />
<br />
</dd>
<dt><em>Synchronous</em> vs <em>Asynchronous</em></dt>
<dd>
A synchronous channel blocks on send until the message is received by some other process, even if the channel has an unbounded buffer. An asynchronous channel does not block on send if the channel’s buffer has space. For example, in Java <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/LinkedTransferQueue.html" target="_blank" rel="noopener"><code>LinkedTransferQueue</code></a> is a synchronous channel, while <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/ArrayBlockingQueue.html" target="_blank" rel="noopener"><code>ArrayBlockingQueue</code></a> is an asynchronous channel.<br />
<br />
</dd>
<dt><em>Blocking</em> vs <em>Non-blocking</em></dt>
<dd>
A blocking channel blocks on send if the channel’s buffer is full, or on receive if it is empty. A non-blocking channel never blocks on send or receive, and instead returns a sentinel value (usually the <a href="https://en.wikipedia.org/wiki/Null_pointer" target="_blank" rel="noopener">Null</a> value), or throws an error to indicate that the operation could not be executed. For example, in Java <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/BlockingQueue.html#put(E)" target="_blank" rel="noopener"><code>BlockingQueue.put</code></a> is a blocking send operation, while <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/BlockingQueue.html#offer(E)" target="_blank" rel="noopener"><code>BlockingQueue.offer</code></a> is a non-blocking send operation.<br />
<br />
</dd>
<dt><em>Fair</em> vs <em>Unfair</em></dt>
<dd>
A fair channel ensures that the order of sends and receives is preserved. That means, if there are multiple pending sends and receives, they are executed in the order they were requested. An unfair channel does not guarantee any order. For example, in Java, <code>ArrayBlockingQueue</code> supports <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/ArrayBlockingQueue.html#%3Cinit%3E(int,boolean)" target="_blank" rel="noopener">fair and unfair modes</a> by passing a boolean flag to its constructor.<br />
<br />
</dd>
<dt><em>Locking</em> vs <em>Lock-free</em></dt>
<dd>
A locking channel uses locks to synchronize access to the channel. A lock-free channel uses atomic operations for the same. For example, in Java <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/LinkedBlockingQueue.html" target="_blank" rel="noopener"><code>LinkedBlockingQueue</code></a> is a locking channel, while <a href="https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/ConcurrentLinkedQueue.html" target="_blank" rel="noopener"><code>ConcurrentLinkedQueue</code></a> is a lock-free channel.<br />
<br />
</dd>
<dt>Selectable vs Non-selectable</dt>
<dd>
A selectable channel can be used in a <a href="https://en.wikipedia.org/wiki/Select_(Unix)" target="_blank" rel="noopener"><em>Select</em></a> like operation to wait for a message on multiple channels at once. A non-selectable channel cannot be used in such an operation. For example, channels in <a href="https://gobyexample.com/channels" target="_blank" rel="noopener">Go</a> and <a href="https://github.com/clojure/core.async" target="_blank" rel="noopener">Clojure core.async</a> are selectable, while aforementioned channels in Java are not.
</dd>
</dl>
<p>In our implementation for <span class="fancy">Co</span>, we have both buffered and unbuffered channels. The buffered channels are bounded, with a fixed capacity. The channels are asynchronous, blocking, fair, lock-free, and non-selectable.</p>
<p>Enough of theory, let’s see how channels work in <span class="fancy">Co</span>.</p>
<h2 data-track-content data-content-name="channel-operations" data-content-piece="implementing-co-4" id="channel-operations">Channel Operations</h2>
<p>In this section, we explore the various scenarios for send and receive operations on a channel in <span class="fancy">Co</span> using diagrams. These diagrams are for buffered channels. For unbuffered channels, the send operation acts as for a fully buffered channel, and the receive operation acts as for an empty buffered channel.</p>
<p>Each channel has three internal queues: a send queue, a receive queue, and a buffer<a href="#fn2" class="footnote-ref" id="fnref2" role="doc-noteref"><sup>2</sup></a>. The send and receive queues are used to store pending send and receive operations (as coroutines) respectively. The buffer is used to store data of the messages. The send and receive queues are always bounded, because otherwise any number of send and receive operations can be blocked on a channel, thus defeating the point of bounded buffer. In extreme cases, it can cause the program to run out of memory.</p>
<p>The invariants we must maintain for the channel operations are:</p>
<ol type="1">
<li>There can never be pending send operations while there are pending receive operations, and vice versa. This is because a send operation will complete immediately if there are pending receive operations, and vice versa.</li>
<li>There can never be pending receive operations while there are messages in the buffer. This is because a receive operation will complete immediately by dequeuing the oldest message in the buffer.</li>
<li>There can never be pending send operations while there is room in the buffer. This is because a send operation will complete immediately by enqueuing the message in the buffer.</li>
</ol>
<p>With these invariants in mind, let’s look at the different scenarios in detail:</p>
<ul>
<li>When a program tries to receive from a channel, and the channel has nothing in its buffer and there are no pending sends, the program blocks. The programs’s continuation is captured as a coroutine, and is enqueued to the receive queue. Note that the coroutine is not queued into the interpreter’s global coroutine queue.</li>
</ul>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 873 321'%3E%3C/svg%3E" class="lazyload ascii-art nolink extra-width" style="--image-aspect-ratio: 2.7196261682242993" data-src="/images/implementing-co-4/receive1.svg" alt="Receive when no pending sends and buffer empty"></img>
<noscript><img src="/images/implementing-co-4/receive1.svg" class="ascii-art nolink extra-width" alt="Receive when no pending sends and buffer empty"></img></noscript>
<figcaption>Receive when no pending sends and buffer empty</figcaption>
</figure>
<ul>
<li>The corresponding scenario for a send operation is when the channel has pending receives. In this case, the send operation completes immediately, and the first coroutine in the receive queue is dequeued and resumed with the message.</li>
</ul>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 857 321'%3E%3C/svg%3E" class="lazyload ascii-art nolink extra-width" style="--image-aspect-ratio: 2.6697819314641746" data-src="/images/implementing-co-4/send1.svg" alt="Send when pending receives"></img>
<noscript><img src="/images/implementing-co-4/send1.svg" class="ascii-art nolink extra-width" alt="Send when pending receives"></img></noscript>
<figcaption>Send when pending receives</figcaption>
</figure>
<ul>
<li>When there are no pending receives and the buffer is not full, the message is enqueued to the buffer, and the send operation completes immediately.</li>
</ul>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 841 321'%3E%3C/svg%3E" class="lazyload ascii-art nolink extra-width" style="--image-aspect-ratio: 2.61993769470405" data-src="/images/implementing-co-4/send2.svg" alt="Send when no pending receives and buffer not full"></img>
<noscript><img src="/images/implementing-co-4/send2.svg" class="ascii-art nolink extra-width" alt="Send when no pending receives and buffer not full"></img></noscript>
<figcaption>Send when no pending receives and buffer not full</figcaption>
</figure>
<ul>
<li>In the corresponding scenario for a receive operation, when there are no pending sends, and there are messages in the buffer, the oldest message is dequeued, and the receive operation completes immediately with it.</li>
</ul>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 873 321'%3E%3C/svg%3E" class="lazyload ascii-art nolink extra-width" style="--image-aspect-ratio: 2.7196261682242993" data-src="/images/implementing-co-4/receive2.svg" alt="Receive when no pending sends and buffer not empty"></img>
<noscript><img src="/images/implementing-co-4/receive2.svg" class="ascii-art nolink extra-width" alt="Receive when no pending sends and buffer not empty"></img></noscript>
<figcaption>Receive when no pending sends and buffer not empty</figcaption>
</figure>
<ul>
<li>When the buffer is full, the program trying to do a send operation is blocked and its continuation is captured as a coroutine and queued into the send queue. Note that the coroutine is not queued into the interpreter’s global coroutine queue.</li>
</ul>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 841 321'%3E%3C/svg%3E" class="lazyload ascii-art nolink extra-width" style="--image-aspect-ratio: 2.61993769470405" data-src="/images/implementing-co-4/send3.svg" alt="Send when buffer full"></img>
<noscript><img src="/images/implementing-co-4/send3.svg" class="ascii-art nolink extra-width" alt="Send when buffer full"></img></noscript>
<figcaption>Send when buffer full</figcaption>
</figure>
<ul>
<li>In the corresponding scenario for a receive operation, when the buffer is full, the oldest message is dequeued from the buffer, and the receive operation completes immediately with it. If there are pending sends, the oldest coroutine in the send queue is dequeued and resumed, and its message is enqueued to the buffer.</li>
</ul>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 1025 321'%3E%3C/svg%3E" class="lazyload ascii-art nolink extra-width" style="--image-aspect-ratio: 3.1931464174454827" data-src="/images/implementing-co-4/receive3.svg" alt="Receive when pending sends and buffer full"></img>
<noscript><img src="/images/implementing-co-4/receive3.svg" class="ascii-art nolink extra-width" alt="Receive when pending sends and buffer full"></img></noscript>
<figcaption>Receive when pending sends and buffer full</figcaption>
</figure>
<ul>
<li>When the send queue is full and the buffer is full as well, an error is thrown when trying to do a send operation.</li>
</ul>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 849 321'%3E%3C/svg%3E" class="lazyload ascii-art nolink extra-width" style="--image-aspect-ratio: 2.6448598130841123" data-src="/images/implementing-co-4/send4.svg" alt="Send when send queue and buffer full"></img>
<noscript><img src="/images/implementing-co-4/send4.svg" class="ascii-art nolink extra-width" alt="Send when send queue and buffer full"></img></noscript>
<figcaption>Send when send queue and buffer full</figcaption>
</figure>
<ul>
<li>Similarly, when the receive queue is full and the buffer is empty, an error is thrown when a receive operation is attempted.</li>
</ul>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 865 321'%3E%3C/svg%3E" class="lazyload ascii-art nolink extra-width" style="--image-aspect-ratio: 2.694704049844237" data-src="/images/implementing-co-4/receive4.svg" alt="Receive when receive queue full and buffer empty"></img>
<noscript><img src="/images/implementing-co-4/receive4.svg" class="ascii-art nolink extra-width" alt="Receive when receive queue full and buffer empty"></img></noscript>
<figcaption>Receive when receive queue full and buffer empty</figcaption>
</figure>
<p>That captures all scenarios for send and receive operations on a channel. In the next section, we implement channels in <span class="fancy">Co</span>.</p>
<h2 data-track-content data-content-name="adding-channels" data-content-piece="implementing-co-4" id="adding-channels">Adding Channels</h2>
<p>Let’s start with defining the <code>Channel</code> type:</p>
<div class="sourceCode" id="cb1" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Channel</span> <span class="ot">=</span> <span class="dt">Channel</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> {<span class="ot"> channelCapacity ::</span> <span class="dt">Int</span>,</span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="ot"> channelBuffer ::</span> <span class="dt">Queue</span> <span class="dt">Value</span>,</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="ot"> channelSendQueue ::</span> <span class="dt">Queue</span> (<span class="dt">Coroutine</span> (), <span class="dt">Value</span>),</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="ot"> channelReceiveQueue ::</span> <span class="dt">Queue</span> (<span class="dt">Coroutine</span> <span class="dt">Value</span>)</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="ot">newChannel ::</span> <span class="dt">Int</span> <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Channel</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a>newChannel size <span class="ot">=</span> <span class="dt">Channel</span> size <span class="op"><$></span> newQueue <span class="op"><*></span> newQueue <span class="op"><*></span> newQueue</span></code></pre></div>
<p>A channel has a buffer, a send queue, and a receive queue. The buffer is a queue of <span class="fancy">Co</span> values, the receive queue is a queue of coroutines, and the send queue is a queue of coroutine and value pair. A channel also has a capacity, which is the capacity of the buffer<a href="#fn3" class="footnote-ref" id="fnref3" role="doc-noteref"><sup>3</sup></a>.</p>
<p>Now, we add <code>Channel</code> to the <code>Value</code> type:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="8-8"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Value</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">Null</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Boolean</span> <span class="dt">Bool</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Str</span> <span class="dt">String</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Num</span> <span class="dt">Integer</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Function</span> <span class="dt">Identifier</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>] <span class="dt">Env</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">BuiltinFunction</span> <span class="dt">Identifier</span> <span class="dt">Int</span> ([<span class="dt">Expr</span>] <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span>)</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op">|</span> <span class="dt">Chan</span> <span class="dt">Channel</span></span></span></code></pre></div>
<p>Finally, we introduce some new built-in functions to create channels:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="4-7" data-deemphasize="8-10"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">builtinEnv ::</span> <span class="dt">IO</span> <span class="dt">Env</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>builtinEnv <span class="ot">=</span> Map.fromList <span class="op"><$></span> <span class="fu">traverse</span> (<span class="fu">traverse</span> newIORef) [</span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> (<span class="st">"print"</span>, <span class="dt">BuiltinFunction</span> <span class="st">"print"</span> <span class="dv">1</span> executePrint)</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> , (<span class="st">"newChannel"</span>,</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">BuiltinFunction</span> <span class="st">"newChannel"</span> <span class="dv">0</span> <span class="op">$</span> <span class="fu">fmap</span> <span class="dt">Chan</span> <span class="op">.</span> <span class="fu">const</span> (newChannel <span class="dv">0</span>))</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> , (<span class="st">"newBufferedChannel"</span>,</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">BuiltinFunction</span> <span class="st">"newBufferedChannel"</span> <span class="dv">1</span> executeNewBufferedChannel)</span></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> , (<span class="st">"sleep"</span>, <span class="dt">BuiltinFunction</span> <span class="st">"sleep"</span> <span class="dv">1</span> executeSleep)</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> , (<span class="st">"getCurrentMillis"</span>,</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">BuiltinFunction</span> <span class="st">"getCurrentMillis"</span> <span class="dv">0</span> executeGetCurrentMillis)</span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> ]</span></code></pre></div>
<p>The <code>newChannel</code> function creates an unbuffered channel, and the <code>newBufferedChannel</code> function creates a buffered channel with the given capacity:</p>
<div class="sourceCode" id="cb2" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="ot">executeNewBufferedChannel ::</span> [<span class="dt">Expr</span>] <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span></span>
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a>executeNewBufferedChannel argEs <span class="ot">=</span> evaluate (<span class="fu">head</span> argEs) <span class="op">>>=</span> \<span class="kw">case</span></span>
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">Num</span> capacity <span class="op">|</span> capacity <span class="op">>=</span> <span class="dv">0</span> <span class="ot">-></span> <span class="dt">Chan</span> <span class="op"><$></span> newChannel (<span class="fu">fromIntegral</span> capacity)</span>
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a> _ <span class="ot">-></span> throw <span class="st">"newBufferedChannel call expected a positive number argument"</span></span></code></pre></div>
<h2 data-track-content data-content-name="wiring-channels" data-content-piece="implementing-co-4" id="wiring-channels">Wiring Channels</h2>
<p>Moving on to wiring the channels into the existing interpreter implementation. First we add a new constructor for send statements to the <code>Stmt</code> type:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="11-11"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Stmt</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">ExprStmt</span> <span class="dt">Expr</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">VarStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">AssignStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">IfStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">WhileStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">FunctionStmt</span> <span class="dt">Identifier</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>]</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">ReturnStmt</span> (<span class="dt">Maybe</span> <span class="dt">Expr</span>)</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">YieldStmt</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">SpawnStmt</span> <span class="dt">Expr</span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op">|</span> <span class="dt">SendStmt</span> <span class="dt">Expr</span> <span class="dt">Expr</span></span></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Eq</span>)</span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Program</span> <span class="ot">=</span> [<span class="dt">Stmt</span>]</span></code></pre></div>
<p>And another for receive expressions to the <code>Expr</code> type:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="10-10"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Expr</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">LNull</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LBool</span> <span class="dt">Bool</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LStr</span> <span class="dt">String</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LNum</span> <span class="dt">Integer</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Variable</span> <span class="dt">Identifier</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Binary</span> <span class="dt">BinOp</span> <span class="dt">Expr</span> <span class="dt">Expr</span></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Call</span> <span class="dt">Expr</span> [<span class="dt">Expr</span>]</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Lambda</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>]</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op">|</span> <span class="dt">Receive</span> <span class="dt">Expr</span></span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Eq</span>)</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Identifier</span> <span class="ot">=</span> <span class="dt">String</span></span></code></pre></div>
<p>We have already written the code to parse these statements and expressions in the <a href="https://abhinavsarkar.net/posts/implementing-co-1/?mtm_campaign=feed">first post</a>, so that’s taken care of. We need to modify the <code>execute</code> and <code>evaluate</code> functions to handle these new statements and expressions. Let’s start with <code>execute</code>:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="23-27"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">execute ::</span> <span class="dt">Stmt</span> <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>execute <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">ExprStmt</span> expr <span class="ot">-></span> void <span class="op">$</span> evaluate expr</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">VarStmt</span> name expr <span class="ot">-></span> evaluate expr <span class="op">>>=</span> defineVar name</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">AssignStmt</span> name expr <span class="ot">-></span> evaluate expr <span class="op">>>=</span> assignVar name</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">IfStmt</span> expr body <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> cond <span class="ot"><-</span> evaluate expr</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> when (isTruthy cond) <span class="op">$</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> traverse_ execute body</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> while<span class="op">@</span>(<span class="dt">WhileStmt</span> expr body) <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> cond <span class="ot"><-</span> evaluate expr</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> when (isTruthy cond) <span class="op">$</span> <span class="kw">do</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> traverse_ execute body</span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a> execute while</span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">ReturnStmt</span> mExpr <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a> mRet <span class="ot"><-</span> <span class="fu">traverse</span> evaluate mExpr</span>
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a> throwError <span class="op">.</span> <span class="dt">Return</span> <span class="op">.</span> fromMaybe <span class="dt">Null</span> <span class="op">$</span> mRet</span>
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">FunctionStmt</span> name params body <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a> env <span class="ot"><-</span> State.gets isEnv</span>
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a> defineVar name <span class="op">$</span> <span class="dt">Function</span> name params body env</span>
<span id="cb1-21"><a href="#cb1-21" aria-hidden="true" tabindex="-1"></a> <span class="dt">YieldStmt</span> <span class="ot">-></span> yield</span>
<span id="cb1-22"><a href="#cb1-22" aria-hidden="true" tabindex="-1"></a> <span class="dt">SpawnStmt</span> expr <span class="ot">-></span> spawn expr</span>
<span id="cb1-23"><a href="#cb1-23" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">SendStmt</span> expr chan <span class="ot">-></span> evaluate chan <span class="op">>>=</span> \<span class="kw">case</span></span></span>
<span id="cb1-24"><a href="#cb1-24" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">Chan</span> channel <span class="ot">-></span> <span class="kw">do</span></span></span>
<span id="cb1-25"><a href="#cb1-25" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> val <span class="ot"><-</span> evaluate expr</span></span>
<span id="cb1-26"><a href="#cb1-26" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> channelSend val channel</span></span>
<span id="cb1-27"><a href="#cb1-27" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> v <span class="ot">-></span> throw <span class="op">$</span> <span class="st">"Cannot send to a non-channel: "</span> <span class="op"><></span> <span class="fu">show</span> v</span></span>
<span id="cb1-28"><a href="#cb1-28" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb1-29"><a href="#cb1-29" aria-hidden="true" tabindex="-1"></a> isTruthy <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb1-30"><a href="#cb1-30" aria-hidden="true" tabindex="-1"></a> <span class="dt">Null</span> <span class="ot">-></span> <span class="dt">False</span></span>
<span id="cb1-31"><a href="#cb1-31" aria-hidden="true" tabindex="-1"></a> <span class="dt">Boolean</span> b <span class="ot">-></span> b</span>
<span id="cb1-32"><a href="#cb1-32" aria-hidden="true" tabindex="-1"></a> _ <span class="ot">-></span> <span class="dt">True</span></span></code></pre></div>
<p>To execute a <code>SendStmt</code>, we evaluate its arguments to get the channel and the value to send. Then we call the <code>channelSend</code> function to send the value over the channel.</p>
<p>Similarly, to evaluate a <code>Receive</code> expression, we evaluate its argument to get the channel, and then call the <code>channelReceive</code> function to receive a value from the channel:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="11-13"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">evaluate ::</span> <span class="dt">Expr</span> <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>evaluate <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">LNull</span> <span class="ot">-></span> <span class="fu">pure</span> <span class="dt">Null</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">LBool</span> bool <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Boolean</span> bool</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">LStr</span> str <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Str</span> str</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">LNum</span> num <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Num</span> num</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">Variable</span> v <span class="ot">-></span> lookupVar v</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">Lambda</span> params body <span class="ot">-></span> <span class="dt">Function</span> <span class="st">"<lambda>"</span> params body <span class="op"><$></span> State.gets isEnv</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> binary<span class="op">@</span><span class="dt">Binary</span> {} <span class="ot">-></span> evaluateBinaryOp binary</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> call<span class="op">@</span><span class="dt">Call</span> {} <span class="ot">-></span> evaluateFuncCall call</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">Receive</span> expr <span class="ot">-></span> evaluate expr <span class="op">>>=</span> \<span class="kw">case</span></span></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">Chan</span> channel <span class="ot">-></span> channelReceive channel</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> val <span class="ot">-></span> throw <span class="op">$</span> <span class="st">"Cannot receive from a non-channel: "</span> <span class="op"><></span> <span class="fu">show</span> val</span></span></code></pre></div>
<p>Now comes the core of the implementation: the <code>channelSend</code> and <code>channelReceive</code> functions. Let’s look into them in detail.</p>
<h2 data-track-content data-content-name="sending-and-receiving" data-content-piece="implementing-co-4" id="sending-and-receiving">Sending and Receiving</h2>
<p>The <code>channelSend</code> function takes a value and a channel, and sends the value over the channel, blocking if necessary.</p>
<div class="sourceCode" id="cb3" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="ot">channelSend ::</span> <span class="dt">Value</span> <span class="ot">-></span> <span class="dt">Channel</span> <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a>channelSend value <span class="dt">Channel</span> {<span class="op">..</span>} <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a> bufferSize <span class="ot"><-</span> queueSize channelBuffer</span>
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a> sendQueueSize <span class="ot"><-</span> queueSize channelSendQueue</span>
<span id="cb3-5"><a href="#cb3-5" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb3-6"><a href="#cb3-6" aria-hidden="true" tabindex="-1"></a> dequeue channelReceiveQueue <span class="op">>>=</span> \<span class="kw">case</span></span>
<span id="cb3-7"><a href="#cb3-7" aria-hidden="true" tabindex="-1"></a> <span class="co">-- there are pending receives</span></span>
<span id="cb3-8"><a href="#cb3-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">Just</span> coroutine<span class="op">@</span><span class="dt">Coroutine</span> {<span class="op">..</span>} <span class="ot">-></span></span>
<span id="cb3-9"><a href="#cb3-9" aria-hidden="true" tabindex="-1"></a> scheduleCoroutine <span class="op">$</span> coroutine { corCont <span class="ot">=</span> <span class="fu">const</span> <span class="op">$</span> corCont value }</span>
<span id="cb3-10"><a href="#cb3-10" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb3-11"><a href="#cb3-11" aria-hidden="true" tabindex="-1"></a> <span class="co">-- there are no pending receives and the buffer is not full</span></span>
<span id="cb3-12"><a href="#cb3-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">Nothing</span> <span class="op">|</span> channelCapacity <span class="op">></span> <span class="dv">0</span> <span class="op">&&</span> bufferSize <span class="op"><</span> channelCapacity <span class="ot">-></span></span>
<span id="cb3-13"><a href="#cb3-13" aria-hidden="true" tabindex="-1"></a> enqueue value channelBuffer</span>
<span id="cb3-14"><a href="#cb3-14" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb3-15"><a href="#cb3-15" aria-hidden="true" tabindex="-1"></a> <span class="co">-- there are no pending receives and</span></span>
<span id="cb3-16"><a href="#cb3-16" aria-hidden="true" tabindex="-1"></a> <span class="co">-- (the buffer is full or the channel is unbuffered)</span></span>
<span id="cb3-17"><a href="#cb3-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">Nothing</span> <span class="op">|</span> sendQueueSize <span class="op"><</span> maxSendQueueSize <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb3-18"><a href="#cb3-18" aria-hidden="true" tabindex="-1"></a> env <span class="ot"><-</span> State.gets isEnv</span>
<span id="cb3-19"><a href="#cb3-19" aria-hidden="true" tabindex="-1"></a> callCC <span class="op">$</span> \cont <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb3-20"><a href="#cb3-20" aria-hidden="true" tabindex="-1"></a> coroutine <span class="ot"><-</span> newCoroutine env cont</span>
<span id="cb3-21"><a href="#cb3-21" aria-hidden="true" tabindex="-1"></a> enqueue (coroutine, value) channelSendQueue</span>
<span id="cb3-22"><a href="#cb3-22" aria-hidden="true" tabindex="-1"></a> runNextCoroutine</span>
<span id="cb3-23"><a href="#cb3-23" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb3-24"><a href="#cb3-24" aria-hidden="true" tabindex="-1"></a> <span class="co">-- the send queue is full</span></span>
<span id="cb3-25"><a href="#cb3-25" aria-hidden="true" tabindex="-1"></a> <span class="dt">Nothing</span> <span class="ot">-></span> throw <span class="st">"Channel send queue is full"</span></span>
<span id="cb3-26"><a href="#cb3-26" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb3-27"><a href="#cb3-27" aria-hidden="true" tabindex="-1"></a> maxSendQueueSize <span class="ot">=</span> <span class="dv">4</span></span></code></pre></div>
<p>This is a direct implementation of the algorithm we discussed earlier using diagrams. We dequeue a coroutine from the receive queue. Then:</p>
<ul>
<li>If there is a coroutine, we schedule it to be run with the sent value. The send call <strong>does not block</strong>.</li>
<li>If there is no coroutine, and
<ul>
<li>the channel is buffered and the buffer is not full, we enqueue the sent value to the buffer. The send call <strong>does not block</strong>.</li>
<li>the buffer is full, we create a new coroutine with the current continuation, and enqueue the coroutine and the value to the send queue. The send call <strong>blocks</strong>.</li>
</ul></li>
<li>If the send queue is full, we throw an error.</li>
</ul>
<p>Next, let’s write the <code>channelReceive</code> function:</p>
<div class="sourceCode" id="cb4" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="ot">channelReceive ::</span> <span class="dt">Channel</span> <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span></span>
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a>channelReceive <span class="dt">Channel</span> {<span class="op">..</span>} <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a> mSend <span class="ot"><-</span> dequeue channelSendQueue</span>
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a> mBufferedValue <span class="ot"><-</span> dequeue channelBuffer</span>
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a> recieveQueueSize <span class="ot"><-</span> queueSize channelReceiveQueue</span>
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a> <span class="kw">case</span> (mSend, mBufferedValue) <span class="kw">of</span></span>
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a> <span class="co">-- the channel is unbuffered and there are pending sends</span></span>
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Just</span> (sendCoroutine, sendValue), <span class="dt">Nothing</span>) <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a> scheduleCoroutine sendCoroutine</span>
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> sendValue</span>
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-13"><a href="#cb4-13" aria-hidden="true" tabindex="-1"></a> <span class="co">-- the buffer is full and there are pending sends</span></span>
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Just</span> (sendCoroutine, sendValue), <span class="dt">Just</span> bufferedValue) <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a> scheduleCoroutine sendCoroutine</span>
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a> enqueue sendValue channelBuffer</span>
<span id="cb4-17"><a href="#cb4-17" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> bufferedValue</span>
<span id="cb4-18"><a href="#cb4-18" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-19"><a href="#cb4-19" aria-hidden="true" tabindex="-1"></a> <span class="co">-- the buffer is empty and there are no pending sends</span></span>
<span id="cb4-20"><a href="#cb4-20" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Nothing</span>, <span class="dt">Nothing</span>) <span class="op">|</span> recieveQueueSize <span class="op"><</span> maxReceiveQueueSize <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb4-21"><a href="#cb4-21" aria-hidden="true" tabindex="-1"></a> env <span class="ot"><-</span> State.gets isEnv</span>
<span id="cb4-22"><a href="#cb4-22" aria-hidden="true" tabindex="-1"></a> callCC <span class="op">$</span> \receive <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb4-23"><a href="#cb4-23" aria-hidden="true" tabindex="-1"></a> coroutine <span class="ot"><-</span> newCoroutine env receive</span>
<span id="cb4-24"><a href="#cb4-24" aria-hidden="true" tabindex="-1"></a> enqueue coroutine channelReceiveQueue</span>
<span id="cb4-25"><a href="#cb4-25" aria-hidden="true" tabindex="-1"></a> runNextCoroutine</span>
<span id="cb4-26"><a href="#cb4-26" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> <span class="dt">Null</span></span>
<span id="cb4-27"><a href="#cb4-27" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-28"><a href="#cb4-28" aria-hidden="true" tabindex="-1"></a> <span class="co">-- the receive queue is full</span></span>
<span id="cb4-29"><a href="#cb4-29" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Nothing</span>, <span class="dt">Nothing</span>) <span class="ot">-></span> throw <span class="st">"Channel receive queue is full"</span></span>
<span id="cb4-30"><a href="#cb4-30" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-31"><a href="#cb4-31" aria-hidden="true" tabindex="-1"></a> <span class="co">-- the buffer is not empty and there are no pending sends</span></span>
<span id="cb4-32"><a href="#cb4-32" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Nothing</span>, <span class="dt">Just</span> bufferedValue) <span class="ot">-></span> <span class="fu">return</span> bufferedValue</span>
<span id="cb4-33"><a href="#cb4-33" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb4-34"><a href="#cb4-34" aria-hidden="true" tabindex="-1"></a> maxReceiveQueueSize <span class="ot">=</span> <span class="dv">4</span></span></code></pre></div>
<p>This is also a straightforward implementation of the algorithm. We dequeue a coroutine and its value from the send queue, and another value from the buffer. Then:</p>
<ul>
<li>If there is a coroutine,
<ul>
<li>but no buffered value, we schedule the coroutine to be resumed, and return its value. The returned value becomes the value that is received from the channel. The receive call <strong>does not block</strong>.</li>
<li>and a buffered value, we schedule the coroutine to be resumed, enqueue its value to the buffer, and return the buffered value. The receive call <strong>does not block</strong>.</li>
</ul></li>
<li>If there is no coroutine and no buffered value, and the receive queue is not full, we create a new coroutine with the current continuation, and enqueue it to the receive queue. The receive call <strong>blocks</strong>.</li>
<li>If the receive queue is full, we throw an error.</li>
</ul>
<p>We hardcode the capacity of the send and receive queues to 4.</p>
<p>That’s it for the implementation of channels. Since we broke down the scenarios for send and receive operations, the implementation is not complicated. Let’s see it in action next.</p>
<h2 data-track-content data-content-name="pubsub-using-channels" data-content-piece="implementing-co-4" id="pubsub-using-channels">Pubsub using Channels</h2>
<p>In this demo, we implement a pubsub system using channels. The pubsub system consists of a server and a set of workers. The server sends messages to the workers over a channel. The workers print the messages and send acks back to the server over another channel. After sending all the messages, the server waits for the acks from the workers, and then stops the workers.</p>
<p>Diagrammatically, the pubsub system looks like this:</p>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 713 305'%3E%3C/svg%3E" class="lazyload ascii-art nolink" style="--image-aspect-ratio: 2.3377049180327867" data-src="/images/implementing-co-4/pubsub.svg" alt="Pubsub using channels"></img>
<noscript><img src="/images/implementing-co-4/pubsub.svg" class="ascii-art nolink" alt="Pubsub using channels"></img></noscript>
<figcaption>Pubsub using channels</figcaption>
</figure>
<p>The boxes with double borders are <abbr title="Threads of computation">ToCs</abbr>, and the ones with single borders are channels. The arrows show how the <abbr title="Threads of computation">ToCs</abbr> and channels are connected.</p>
<details>
<summary>
Pubsub code
</summary>
<div class="sourceCode" id="cb5" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="co">// server sends messages to workers.</span></span>
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">startServer</span>(messageCount<span class="op">,</span> messageChan) {</span>
<span id="cb5-3"><a href="#cb5-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"server starting"</span>)<span class="op">;</span></span>
<span id="cb5-4"><a href="#cb5-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb5-5"><a href="#cb5-5" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> messageCount <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb5-6"><a href="#cb5-6" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"server sending: "</span> <span class="op">+</span> i)<span class="op">;</span></span>
<span id="cb5-7"><a href="#cb5-7" aria-hidden="true" tabindex="-1"></a> i <span class="op">-></span> messageChan<span class="op">;</span></span>
<span id="cb5-8"><a href="#cb5-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"server sent: "</span> <span class="op">+</span> i)<span class="op">;</span></span>
<span id="cb5-9"><a href="#cb5-9" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb5-10"><a href="#cb5-10" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb5-11"><a href="#cb5-11" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb5-12"><a href="#cb5-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-13"><a href="#cb5-13" aria-hidden="true" tabindex="-1"></a><span class="co">// workers receive messages over a channel, print them.</span></span>
<span id="cb5-14"><a href="#cb5-14" aria-hidden="true" tabindex="-1"></a><span class="co">// and send a ack back to the sender on a channel.</span></span>
<span id="cb5-15"><a href="#cb5-15" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">worker</span>(name<span class="op">,</span> messageChan<span class="op">,</span> ackChan) {</span>
<span id="cb5-16"><a href="#cb5-16" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"worker "</span> <span class="op">+</span> name <span class="op">+</span> <span class="st">" starting"</span>)<span class="op">;</span></span>
<span id="cb5-17"><a href="#cb5-17" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> message <span class="op">=</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb5-18"><a href="#cb5-18" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (<span class="kw">true</span>) {</span>
<span id="cb5-19"><a href="#cb5-19" aria-hidden="true" tabindex="-1"></a> message <span class="op">=</span> <span class="op"><-</span> messageChan<span class="op">;</span></span>
<span id="cb5-20"><a href="#cb5-20" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"worker "</span> <span class="op">+</span> name <span class="op">+</span> <span class="st">" received: "</span> <span class="op">+</span> message)<span class="op">;</span></span>
<span id="cb5-21"><a href="#cb5-21" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (message <span class="op">==</span> <span class="kw">null</span>) {</span>
<span id="cb5-22"><a href="#cb5-22" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"worker "</span> <span class="op">+</span> name <span class="op">+</span> <span class="st">" stopped"</span>)<span class="op">;</span></span>
<span id="cb5-23"><a href="#cb5-23" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span><span class="op">;</span></span>
<span id="cb5-24"><a href="#cb5-24" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb5-25"><a href="#cb5-25" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"worker "</span> <span class="op">+</span> name <span class="op">+</span> <span class="st">" sending: "</span> <span class="op">+</span> message)<span class="op">;</span></span>
<span id="cb5-26"><a href="#cb5-26" aria-hidden="true" tabindex="-1"></a> message <span class="op">-></span> ackChan<span class="op">;</span></span>
<span id="cb5-27"><a href="#cb5-27" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"worker "</span> <span class="op">+</span> name <span class="op">+</span> <span class="st">" sent: "</span> <span class="op">+</span> message)<span class="op">;</span></span>
<span id="cb5-28"><a href="#cb5-28" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb5-29"><a href="#cb5-29" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb5-30"><a href="#cb5-30" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-31"><a href="#cb5-31" aria-hidden="true" tabindex="-1"></a><span class="co">// start workers.</span></span>
<span id="cb5-32"><a href="#cb5-32" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">startWorkers</span>(workerCount<span class="op">,</span> messageChan<span class="op">,</span> ackChan) {</span>
<span id="cb5-33"><a href="#cb5-33" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"workers starting"</span>)<span class="op">;</span></span>
<span id="cb5-34"><a href="#cb5-34" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb5-35"><a href="#cb5-35" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> workerCount <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb5-36"><a href="#cb5-36" aria-hidden="true" tabindex="-1"></a> <span class="kw">function</span>(name) {</span>
<span id="cb5-37"><a href="#cb5-37" aria-hidden="true" tabindex="-1"></a> spawn <span class="fu">worker</span>(name<span class="op">,</span> messageChan<span class="op">,</span> ackChan)<span class="op">;</span></span>
<span id="cb5-38"><a href="#cb5-38" aria-hidden="true" tabindex="-1"></a> }(i)<span class="op">;</span></span>
<span id="cb5-39"><a href="#cb5-39" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb5-40"><a href="#cb5-40" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb5-41"><a href="#cb5-41" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"workers scheduled to be started"</span>)<span class="op">;</span></span>
<span id="cb5-42"><a href="#cb5-42" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb5-43"><a href="#cb5-43" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-44"><a href="#cb5-44" aria-hidden="true" tabindex="-1"></a><span class="co">// server waits for acks from workers.</span></span>
<span id="cb5-45"><a href="#cb5-45" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">waitForWorkers</span>(messageCount<span class="op">,</span> ackChan<span class="op">,</span> doneChan) {</span>
<span id="cb5-46"><a href="#cb5-46" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"server waiting for acks"</span>)<span class="op">;</span></span>
<span id="cb5-47"><a href="#cb5-47" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb5-48"><a href="#cb5-48" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> message <span class="op">=</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb5-49"><a href="#cb5-49" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> messageCount <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb5-50"><a href="#cb5-50" aria-hidden="true" tabindex="-1"></a> message <span class="op">=</span> <span class="op"><-</span> ackChan<span class="op">;</span></span>
<span id="cb5-51"><a href="#cb5-51" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"server received: "</span> <span class="op">+</span> message)<span class="op">;</span></span>
<span id="cb5-52"><a href="#cb5-52" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb5-53"><a href="#cb5-53" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb5-54"><a href="#cb5-54" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"server received all acks"</span>)<span class="op">;</span></span>
<span id="cb5-55"><a href="#cb5-55" aria-hidden="true" tabindex="-1"></a> <span class="kw">null</span> <span class="op">-></span> doneChan<span class="op">;</span></span>
<span id="cb5-56"><a href="#cb5-56" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb5-57"><a href="#cb5-57" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-58"><a href="#cb5-58" aria-hidden="true" tabindex="-1"></a><span class="co">// stop workers.</span></span>
<span id="cb5-59"><a href="#cb5-59" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">stopWorkers</span>(workerCount<span class="op">,</span> messageChan<span class="op">,</span> doneChan) {</span>
<span id="cb5-60"><a href="#cb5-60" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> done <span class="op">=</span> <span class="op"><-</span> doneChan<span class="op">;</span></span>
<span id="cb5-61"><a href="#cb5-61" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"workers stopping"</span>)<span class="op">;</span></span>
<span id="cb5-62"><a href="#cb5-62" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb5-63"><a href="#cb5-63" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> workerCount <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb5-64"><a href="#cb5-64" aria-hidden="true" tabindex="-1"></a> <span class="kw">null</span> <span class="op">-></span> messageChan<span class="op">;</span></span>
<span id="cb5-65"><a href="#cb5-65" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb5-66"><a href="#cb5-66" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb5-67"><a href="#cb5-67" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"workers scheduled to be stopped"</span>)<span class="op">;</span></span>
<span id="cb5-68"><a href="#cb5-68" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb5-69"><a href="#cb5-69" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-70"><a href="#cb5-70" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> workerCount <span class="op">=</span> <span class="dv">3</span><span class="op">;</span></span>
<span id="cb5-71"><a href="#cb5-71" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> messageCount <span class="op">=</span> <span class="dv">7</span><span class="op">;</span></span>
<span id="cb5-72"><a href="#cb5-72" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> messageBufferSize <span class="op">=</span> <span class="dv">5</span><span class="op">;</span></span>
<span id="cb5-73"><a href="#cb5-73" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> ackBufferSize <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb5-74"><a href="#cb5-74" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> messageChan <span class="op">=</span> <span class="fu">newBufferedChannel</span>(messageBufferSize)<span class="op">;</span></span>
<span id="cb5-75"><a href="#cb5-75" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> ackChan <span class="op">=</span> <span class="fu">newBufferedChannel</span>(ackBufferSize)<span class="op">;</span></span>
<span id="cb5-76"><a href="#cb5-76" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> doneChan <span class="op">=</span> <span class="fu">newChannel</span>()<span class="op">;</span></span>
<span id="cb5-77"><a href="#cb5-77" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-78"><a href="#cb5-78" aria-hidden="true" tabindex="-1"></a><span class="fu">startWorkers</span>(workerCount<span class="op">,</span> messageChan<span class="op">,</span> ackChan)<span class="op">;</span></span>
<span id="cb5-79"><a href="#cb5-79" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">waitForWorkers</span>(messageCount<span class="op">,</span> ackChan<span class="op">,</span> doneChan)<span class="op">;</span></span>
<span id="cb5-80"><a href="#cb5-80" aria-hidden="true" tabindex="-1"></a><span class="fu">startServer</span>(messageCount<span class="op">,</span> messageChan)<span class="op">;</span></span>
<span id="cb5-81"><a href="#cb5-81" aria-hidden="true" tabindex="-1"></a><span class="fu">stopWorkers</span>(workerCount<span class="op">,</span> messageChan<span class="op">,</span> doneChan)<span class="op">;</span></span></code></pre></div>
</details>
<p>Running the program produces this output:</p>
<details>
<summary>
Pubsub output
</summary>
<pre class="plain"><code>workers starting
workers scheduled to be started
server starting
server sending: 1
server sent: 1
server sending: 2
server sent: 2
server sending: 3
server sent: 3
server sending: 4
server sent: 4
server sending: 5
server sent: 5
server sending: 6
worker 1 starting
worker 1 received: 1
worker 1 sending: 1
worker 1 sent: 1
worker 1 received: 2
worker 1 sending: 2
worker 2 starting
worker 2 received: 3
worker 2 sending: 3
worker 3 starting
worker 3 received: 4
worker 3 sending: 4
server waiting for acks
server received: 1
server received: 2
server received: 3
server received: 4
server sent: 6
server sending: 7
server sent: 7
worker 1 sent: 2
worker 1 received: 5
worker 1 sending: 5
worker 1 sent: 5
worker 1 received: 6
worker 1 sending: 6
worker 1 sent: 6
worker 1 received: 7
worker 1 sending: 7
worker 2 sent: 3
worker 3 sent: 4
server received: 5
server received: 6
server received: 7
server received all acks
worker 1 sent: 7
workers stopping
workers scheduled to be stopped
worker 2 received: null
worker 2 stopped
worker 3 received: null
worker 3 stopped
worker 1 received: null
worker 1 stopped</code></pre>
</details>
<p>The output shows how the server and worker coroutines yield control to each other when they are waiting for messages or acks<a href="#fn4" class="footnote-ref" id="fnref4" role="doc-noteref"><sup>4</sup></a>.</p>
<h2 data-track-content data-content-name="bonus-round-emulating-actors" data-content-piece="implementing-co-4" id="bonus-round-emulating-actors">Bonus Round: Emulating Actors</h2>
<p>The <em><a href="https://en.wikipedia.org/wiki/Actor_model" target="_blank" rel="noopener">Actor model</a></em> is a concurrent programming paradigm where computation is carried out by lightweight processes called <em>Actors</em> that can only communicate with each other by sending messages. This makes them ideal for building concurrent and distributed systems.</p>
<p>In this section, we emulate actors in <span class="fancy">Co</span> using channels:</p>
<div class="sourceCode" id="cb7" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">start</span>(<span class="bu">process</span>) {</span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> inbox <span class="op">=</span> <span class="fu">newChannel</span>()<span class="op">;</span></span>
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">spawn</span> (<span class="kw">function</span> () {</span>
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> val <span class="op">=</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (<span class="kw">true</span>) {</span>
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a> val <span class="op">=</span> <span class="op"><-</span> inbox<span class="op">;</span></span>
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (val <span class="op">==</span> <span class="kw">null</span>) { <span class="cf">return</span><span class="op">;</span> }</span>
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a> <span class="bu">process</span>(val)<span class="op">;</span></span>
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a> })()<span class="op">;</span></span>
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="kw">function</span> (message) { message <span class="op">-></span> inbox<span class="op">;</span> }<span class="op">;</span></span>
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb7-13"><a href="#cb7-13" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb7-14"><a href="#cb7-14" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">send</span>(actor<span class="op">,</span> message) { <span class="fu">actor</span>(message)<span class="op">;</span> }</span>
<span id="cb7-15"><a href="#cb7-15" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">stop</span>(actor) { <span class="fu">actor</span>(<span class="kw">null</span>)<span class="op">;</span> }</span></code></pre></div>
<p>Actors are implemented as wrappers around channels. By sending messages to an actor’s channel, we can send messages to the actor. However, we cannot expose the channels directly, so we wrap them in functions.</p>
<p>The <code>start</code> function creates and starts an actor by creating a new channel, and spawning a coroutine that receives messages from the channel in a loop and passes them to the <code>process</code> function taken as a parameter by the <code>start</code> function. Upon receiving a <code>null</code> value, the coroutine returns, which stops the actor.</p>
<p>The <code>start</code> function returns a function to send messages to the actor, which works by sending the messages to the actor’s channel.</p>
<p>The <code>send</code> function is a convenience function to send a message to an actor. The <code>stop</code> function stop an actor by sending it a <code class="sourceCode javascript"><span class="kw">null</span></code> message.</p>
<p>It was easy, wasn’t it? Now let’s use actors in some different ways.</p>
<p>Let’s start with a simple example of an actor that prints the received messages:</p>
<div class="sourceCode" id="cb8" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> printer <span class="op">=</span> <span class="fu">start</span>(print)<span class="op">;</span></span>
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">send</span>(printer<span class="op">,</span> <span class="st">"world"</span>)<span class="op">;</span></span>
<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a><span class="fu">send</span>(printer<span class="op">,</span> <span class="st">"hello"</span>)<span class="op">;</span></span>
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a><span class="fu">stop</span>(printer)<span class="op">;</span></span></code></pre></div>
<p>The <code>process</code> parameter here is the <a href="https://abhinavsarkar.net/posts/implementing-co-2/?mtm_campaign=feed#print-func"><code>print</code></a> function. Running this program produces the following output:</p>
<pre class="plain"><code>hello
world</code></pre>
<p>Next, let’s write an actor that counts. For that, first we need to create a <em><a href="https://en.wikipedia.org/wiki/2-Tuple" target="_blank" rel="noopener">2-Tuple</a></em> data structure using closures, named <code>Pair</code><a href="#fn5" class="footnote-ref" id="fnref5" role="doc-noteref"><sup>5</sup></a>:</p>
<div class="sourceCode" id="cb10" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">Pair</span>(first<span class="op">,</span> second) {</span>
<span id="cb10-2"><a href="#cb10-2" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="kw">function</span> (command) {</span>
<span id="cb10-3"><a href="#cb10-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (command <span class="op">==</span> <span class="st">"first"</span>) { <span class="cf">return</span> first<span class="op">;</span> }</span>
<span id="cb10-4"><a href="#cb10-4" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (command <span class="op">==</span> <span class="st">"second"</span>) { <span class="cf">return</span> second<span class="op">;</span> }</span>
<span id="cb10-5"><a href="#cb10-5" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb10-6"><a href="#cb10-6" aria-hidden="true" tabindex="-1"></a> }<span class="op">;</span></span>
<span id="cb10-7"><a href="#cb10-7" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb10-8"><a href="#cb10-8" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb10-9"><a href="#cb10-9" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">first</span>(pair) { <span class="cf">return</span> <span class="fu">pair</span>(<span class="st">"first"</span>)<span class="op">;</span> }</span>
<span id="cb10-10"><a href="#cb10-10" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">second</span>(pair) { <span class="cf">return</span> <span class="fu">pair</span>(<span class="st">"second"</span>)<span class="op">;</span> }</span></code></pre></div>
<p>Now we implement the counter actor:</p>
<div class="sourceCode" id="cb11" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">makeCounter</span>() {</span>
<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> value <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb11-3"><a href="#cb11-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="fu">start</span>(<span class="kw">function</span> (message) {</span>
<span id="cb11-4"><a href="#cb11-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> command <span class="op">=</span> <span class="fu">first</span>(message)<span class="op">;</span></span>
<span id="cb11-5"><a href="#cb11-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> arg <span class="op">=</span> <span class="fu">second</span>(message)<span class="op">;</span></span>
<span id="cb11-6"><a href="#cb11-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb11-7"><a href="#cb11-7" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (command <span class="op">==</span> <span class="st">"inc"</span>) { value <span class="op">=</span> value <span class="op">+</span> arg<span class="op">;</span> }</span>
<span id="cb11-8"><a href="#cb11-8" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (command <span class="op">==</span> <span class="st">"get"</span>) { <span class="fu">send</span>(arg<span class="op">,</span> value)<span class="op">;</span> }</span>
<span id="cb11-9"><a href="#cb11-9" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
<span id="cb11-10"><a href="#cb11-10" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
<p>The <code>makeCounter</code> function creates a counter actor. The counter actor is started with a processing function that takes a message as a <code>Pair</code>, extracts the command and the argument from the message, and increments the counter value or sends the counter value back depending on the command. We exercise the counter like this:</p>
<div class="sourceCode" id="cb12" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> printer <span class="op">=</span> <span class="fu">start</span>(print)<span class="op">;</span></span>
<span id="cb12-2"><a href="#cb12-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> counter1 <span class="op">=</span> <span class="fu">makeCounter</span>()<span class="op">;</span></span>
<span id="cb12-3"><a href="#cb12-3" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-4"><a href="#cb12-4" aria-hidden="true" tabindex="-1"></a><span class="fu">send</span>(counter1<span class="op">,</span> <span class="fu">Pair</span>(<span class="st">"inc"</span><span class="op">,</span> <span class="dv">1</span>))<span class="op">;</span></span>
<span id="cb12-5"><a href="#cb12-5" aria-hidden="true" tabindex="-1"></a><span class="fu">send</span>(counter1<span class="op">,</span> <span class="fu">Pair</span>(<span class="st">"get"</span><span class="op">,</span> printer))<span class="op">;</span></span>
<span id="cb12-6"><a href="#cb12-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-7"><a href="#cb12-7" aria-hidden="true" tabindex="-1"></a><span class="fu">send</span>(counter1<span class="op">,</span> <span class="fu">Pair</span>(<span class="st">"inc"</span><span class="op">,</span> <span class="dv">2</span>))<span class="op">;</span></span>
<span id="cb12-8"><a href="#cb12-8" aria-hidden="true" tabindex="-1"></a><span class="fu">send</span>(counter1<span class="op">,</span> <span class="fu">Pair</span>(<span class="st">"get"</span><span class="op">,</span> printer))<span class="op">;</span></span>
<span id="cb12-9"><a href="#cb12-9" aria-hidden="true" tabindex="-1"></a><span class="fu">stop</span>(counter1)<span class="op">;</span></span>
<span id="cb12-10"><a href="#cb12-10" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-11"><a href="#cb12-11" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> counter2 <span class="op">=</span> <span class="fu">makeCounter</span>()<span class="op">;</span></span>
<span id="cb12-12"><a href="#cb12-12" aria-hidden="true" tabindex="-1"></a><span class="fu">send</span>(counter2<span class="op">,</span> <span class="fu">Pair</span>(<span class="st">"inc"</span><span class="op">,</span> <span class="dv">5</span>))<span class="op">;</span></span>
<span id="cb12-13"><a href="#cb12-13" aria-hidden="true" tabindex="-1"></a><span class="fu">send</span>(counter2<span class="op">,</span> <span class="fu">Pair</span>(<span class="st">"get"</span><span class="op">,</span> printer))<span class="op">;</span></span>
<span id="cb12-14"><a href="#cb12-14" aria-hidden="true" tabindex="-1"></a><span class="fu">stop</span>(counter2)<span class="op">;</span></span>
<span id="cb12-15"><a href="#cb12-15" aria-hidden="true" tabindex="-1"></a><span class="fu">stop</span>(printer)<span class="op">;</span></span></code></pre></div>
<p>The output of the program is:</p>
<pre class="plain"><code>1
3
5</code></pre>
<p>And for the grand finale, let’s reimplement the <a href="https://abhinavsarkar.net/posts/implementing-co-1/?mtm_campaign=feed#cb5-1">ping-pong</a> program using actors:</p>
<div class="sourceCode" id="cb14" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">makePingPonger</span>(name) {</span>
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> self <span class="op">=</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">function</span> <span class="fu">pingPong</span>(message) {</span>
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> value <span class="op">=</span> <span class="fu">first</span>(message)<span class="op">;</span></span>
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> other <span class="op">=</span> <span class="fu">second</span>(message)<span class="op">;</span></span>
<span id="cb14-6"><a href="#cb14-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb14-7"><a href="#cb14-7" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (value <span class="op">==</span> <span class="st">"done"</span>) {</span>
<span id="cb14-8"><a href="#cb14-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" done"</span>)<span class="op">;</span></span>
<span id="cb14-9"><a href="#cb14-9" aria-hidden="true" tabindex="-1"></a> <span class="fu">spawn</span> (<span class="kw">function</span> () { <span class="fu">stop</span>(self)<span class="op">;</span> } ())<span class="op">;</span></span>
<span id="cb14-10"><a href="#cb14-10" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span><span class="op">;</span></span>
<span id="cb14-11"><a href="#cb14-11" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb14-12"><a href="#cb14-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb14-13"><a href="#cb14-13" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" "</span> <span class="op">+</span> value)<span class="op">;</span></span>
<span id="cb14-14"><a href="#cb14-14" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (value <span class="op">==</span> <span class="dv">0</span>) {</span>
<span id="cb14-15"><a href="#cb14-15" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" done"</span>)<span class="op">;</span></span>
<span id="cb14-16"><a href="#cb14-16" aria-hidden="true" tabindex="-1"></a> <span class="fu">send</span>(other<span class="op">,</span> <span class="fu">Pair</span>(<span class="st">"done"</span><span class="op">,</span> self))<span class="op">;</span></span>
<span id="cb14-17"><a href="#cb14-17" aria-hidden="true" tabindex="-1"></a> <span class="fu">spawn</span> (<span class="kw">function</span> () { <span class="fu">stop</span>(self)<span class="op">;</span> } ())<span class="op">;</span></span>
<span id="cb14-18"><a href="#cb14-18" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span><span class="op">;</span></span>
<span id="cb14-19"><a href="#cb14-19" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb14-20"><a href="#cb14-20" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb14-21"><a href="#cb14-21" aria-hidden="true" tabindex="-1"></a> <span class="fu">send</span>(other<span class="op">,</span> <span class="fu">Pair</span>(value <span class="op">-</span> <span class="dv">1</span><span class="op">,</span> self))<span class="op">;</span></span>
<span id="cb14-22"><a href="#cb14-22" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb14-23"><a href="#cb14-23" aria-hidden="true" tabindex="-1"></a> self <span class="op">=</span> <span class="fu">start</span>(pingPong)<span class="op">;</span></span>
<span id="cb14-24"><a href="#cb14-24" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> self<span class="op">;</span></span>
<span id="cb14-25"><a href="#cb14-25" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
<p>The <code>makePingPonger</code> function creates a ping-ponger actor. The ping-ponger actor is started with a processing function that takes a message as a <code>Pair</code> of the value to print and the other actor to send the next message to. The processing function prints the value, decrements it, and sends it to the other actor. If the value is 0, it sends a <code>done</code> message to the other actor and stops itself. If the value is <code>done</code>, it stops itself.</p>
<p>Upon running it like this:</p>
<div class="sourceCode" id="cb15" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> pinger <span class="op">=</span> <span class="fu">makePingPonger</span>(<span class="st">"ping"</span>)<span class="op">;</span></span>
<span id="cb15-2"><a href="#cb15-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> ponger <span class="op">=</span> <span class="fu">makePingPonger</span>(<span class="st">"pong"</span>)<span class="op">;</span></span>
<span id="cb15-3"><a href="#cb15-3" aria-hidden="true" tabindex="-1"></a><span class="fu">send</span>(pinger<span class="op">,</span> <span class="fu">Pair</span>(<span class="dv">10</span><span class="op">,</span> ponger))<span class="op">;</span></span></code></pre></div>
<p>It produces the same output as the original ping-pong program:</p>
<pre class="plain"><code>ping 10
pong 9
ping 8
pong 7
ping 6
pong 5
ping 4
pong 3
ping 2
pong 1
ping 0
ping done
pong done</code></pre>
<hr></hr>
<p>In this post, we added channels to <span class="fancy">Co</span>, and used them to create a variety of concurrent programs. We learned about <abbr title="Communicating Sequential Processes">CSP</abbr> and how implement it using coroutines and channels. In the next post, we will add support for <em><a href="https://en.wikipedia.org/wiki/Sleep_(system_call)" target="_blank" rel="noopener">sleep</a></em> to <span class="fancy">Co</span>.</p>
<p>The code for complete <span class="fancy">Co</span> interpreter is available <a href="https://abhinavsarkar.net/code/co-interpreter.html?mtm_campaign=feed">here</a>.</p>
<h2 class="notoc" data-track-content data-content-name="acknowledgements" data-content-piece="implementing-co-4" id="acknowledgements">Acknowledgements</h2>
<p>Many thanks to <a href="https://www.deobald.ca/" target="_blank" rel="noopener">Steven Deobald</a> for reviewing a draft of this article.</p>
<div id="refs" class="references csl-bib-body hanging-indent" role="list">
<div id="ref-Hoare1986-ih" class="csl-entry" role="listitem">
Hoare, C A R. <em>Communicating Sequential Processes</em>. Prentice Hall, 1986. <a href="https://doi.org/10.1145/359576.359585" target="_blank" rel="noopener">https://doi.org/10.1145/359576.359585</a>.
</div>
</div>
<section id="footnotes" class="footnotes footnotes-end-of-document" role="doc-endnotes">
<hr></hr>
<ol>
<li id="fn1"><p>Recently, Java added support for <a href="https://openjdk.org/jeps/444" target="_blank" rel="noopener"><em>Virtual Threads</em></a>, which though are not cooperatively scheduled like coroutines, are scheduled by the JVM, and are very lightweight. With virtual threads, the various Java queues can be considered channels as defined in <abbr title="Communicating Sequential Processes">CSP</abbr>.<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn2"><p>The design of channels in <span class="fancy">Co</span> is inspired by the design of channels in <a href="http://cdn.cognitect.com/presentations/2014/insidechannels.pdf" target="_blank" rel="noopener">Clojure core.async</a>. It is a simplified version, not supporting some of the features of core.async, such as <a href="https://clojuredocs.org/clojure.core.async/transduce" target="_blank" rel="noopener">transducers</a>, and <a href="https://clojuredocs.org/clojure.core.async/alt!" target="_blank" rel="noopener">alts</a>.<a href="#fnref2" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn3"><p>Recall that the <a href="https://abhinavsarkar.net/posts/implementing-co-3/?mtm_campaign=feed#queue-ds"><code>Queue</code></a> type is an immutable queue data structure wrapped in an <code>IORef</code>, which we manipulate using atomic operations <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html#v:atomicModifyIORef-39-" target="_blank" rel="noopener"><code>atomicModifyIORef'</code></a>.<a href="#fnref3" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn4"><p>You can try running the program with different values for the <code>workerCount</code>, <code>messageCount</code>, <code>messageBufferSize</code> and <code>ackBufferSize</code> variables to see how it behaves. You can also try changing the order of the function calls at the end of the program, or prefixing them with <code class="sourceCode"><span class="cf">spawn</span></code> to see how it affects the output. In some cases, the program may deadlock and hang, and in some other cases, it may throw an error. Try to understand why.<a href="#fnref4" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn5"><p>We used the same trick to create a <a href="https://abhinavsarkar.net/posts/implementing-co-3/?mtm_campaign=feed#bonus-round-breadth-first-traversal-without-a-queue">binary tree</a> data structure in the previous post.<a href="#fnref5" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
</ol>
</section><p>If you liked this post, please <a href="https://abhinavsarkar.net/posts/implementing-co-4/?mtm_campaign=feed#syndications">leave a comment</a>.</p><img referrerpolicy="no-referrer-when-downgrade" src="https://anna.abhinavsarkar.net/matomo.php?idsite=1&rec=1" style="border:0" alt="" />2023-06-03T00:00:00Z<p>In the <a href="https://abhinavsarkar.net/posts/implementing-co-3/">previous post</a>, we added coroutines to <span class="fancy">Co</span>, the small language we are implementing in this series of posts. In this post, we add channels to it to be able to communicate between coroutines.</p>
<ol type="1">
<li><a href="https://abhinavsarkar.net/posts/implementing-co-1/">Implementing Co #1: The Parser</a></li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-2/">Implementing Co #2: The Interpreter</a></li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-3/">Implementing Co #3: Adding Coroutines</a></li>
<li>Implementing Co #4: Adding Channels</li>
<li>Implementing Co #5: Adding Sleep</li>
</ol>
https://abhinavsarkar.net/posts/implementing-co-3/Implementing Co, a Small Language With Coroutines #3: Adding Coroutines2023-02-11T00:00:00ZAbhinav Sarkarhttps://abhinavsarkar.net/about/abhinav@abhinavsarkar.net<p>In the <a href="https://abhinavsarkar.net/posts/implementing-co-2/?mtm_campaign=feed">previous post</a>, we wrote the interpreter for basic features of <span class="fancy">Co</span>, the small language we are building in this series of posts. In this post, we explore and implement what makes <span class="fancy">Co</span> really interesting: support for lightweight concurrency using Coroutines.</p>
<p>This post was originally published on <a href="https://abhinavsarkar.net/posts/implementing-co-3/?mtm_campaign=feed">abhinavsarkar.net</a>.</p><!--more-->
<p>This is the third post in a series of posts:</p>
<ol type="1">
<li><a href="https://abhinavsarkar.net/posts/implementing-co-1/?mtm_campaign=feed">Implementing Co #1: The Parser</a></li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-2/?mtm_campaign=feed">Implementing Co #2: The Interpreter</a></li>
<li>Implementing Co #3: Adding Coroutines</li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-4/?mtm_campaign=feed">Implementing Co #4: Adding Channels</a></li>
<li>Implementing Co #5: Adding Sleep</li>
</ol>
<p>In this and next two posts, we add support for the following features to the <span class="fancy">Co</span> interpreter:</p>
<ul>
<li><code class="sourceCode javascript"><span class="kw">yield</span></code> statement to yield the current thread of computation (ToC).</li>
<li><code class="sourceCode"><span class="cf">spawn</span></code> statement to start a new <abbr title="Thread of computation">ToC</abbr>.</li>
<li>First class channels with operators to send and receive values over them.</li>
<li><code class="sourceCode"><span class="cf">sleep</span></code> function to sleep the current <abbr title="Thread of computation">ToC</abbr> for a given number of milliseconds.</li>
</ul>
<p>Let’s <span class="fancy">Co</span>!</p>
<!--balha-->
<nav id="toc"><h3>Contents</h3><ol><li><a href="#coroutines">Coroutines</a></li><li><a href="#coroutines-in-various-languages">Coroutines in Various Languages</a></li><li><a href="#implementing-coroutines">Implementing Coroutines</a></li><li><a href="#continuation-passing-style">Continuation-Passing Style</a><ol><li><a href="#continuation-passing-style-in-haskell">Continuation-Passing Style in Haskell</a></li><li><a href="#call-with-current-continuation">Call with Current Continuation</a></li></ol></li><li><a href="#from-continuations-to-coroutines">From Continuations to Coroutines</a></li><li><a href="#scheduling-coroutines">Scheduling Coroutines</a></li><li><a href="#yield-and-spawn">Yield and Spawn</a><ol><li><a href="#implementation">Implementation</a></li></ol></li><li><a href="#waiting-for-termination">Waiting for Termination</a></li><li><a href="#putting-everything-together">Putting Everything Together</a></li><li><a href="#bonus-round-breadth-first-traversal-without-a-queue">Bonus Round: Breadth-First Traversal without a Queue</a></li></ol></nav>
<h2 data-track-content data-content-name="coroutines" data-content-piece="implementing-co-3" id="coroutines">Coroutines</h2>
<p><a href="https://en.wikipedia.org/wiki/Coroutines" target="_blank" rel="noopener">Coroutines</a><sup><a href="#ref-Knuth1997-rv" class="citation" title="Knuth, “Coroutines.”
">@1</a></sup> are computations that support <a href="https://en.wikipedia.org/wiki/Cooperative_multitasking" target="_blank" rel="noopener"><em>Cooperative multitasking</em></a><sup><a href="#ref-Bartel2011-ap" class="citation" title="Bartel, “Non-Preemptive Multitasking.”
">@2</a></sup>. Unlike ordinary <a href="https://en.wikipedia.org/wiki/Subroutines" target="_blank" rel="noopener"><em>Subroutines</em></a> that execute from start to end, and do not hold any state between invocations, coroutines can exit in the middle, and may resume later from the same point while holding state between invocations. They do so by <a href="https://en.wikipedia.org/wiki/Yield_(multithreading)" target="_blank" rel="noopener"><em>yielding</em></a> the control of the current running thread.</p>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 713 337'%3E%3C/svg%3E" class="lazyload ascii-art nolink" style="--image-aspect-ratio: 2.115727002967359" data-src="/images/implementing-co-3/coroutine-vs-subroutine.svg" alt="Subroutines vs. Coroutines"></img>
<noscript><img src="/images/implementing-co-3/coroutine-vs-subroutine.svg" class="ascii-art nolink" alt="Subroutines vs. Coroutines"></img></noscript>
<figcaption>Subroutines vs. Coroutines</figcaption>
</figure>
<p>The above diagram compares the execution of a subroutine and a coroutine, invoked from a caller<a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a>. The rectangles represent instructions, starting at top and ending at bottom. The arrows represent the flow of control.</p>
<p>The subroutine executes from start to end when called. The coroutine can exit in the middle by yielding, and can resume later from the same point. The coroutine state is saved automatically at the point of yielding, and restored when the coroutine resumes. Note that the coroutine may not be resumed, in which case the rest of the coroutine never executes.</p>
<h2 data-track-content data-content-name="coroutines-in-various-languages" data-content-piece="implementing-co-3" id="coroutines-in-various-languages">Coroutines in Various Languages</h2>
<p>Many languages have support for coroutines, either <a href="https://en.wikipedia.org/wiki/Coroutine#Programming_languages_with_native_support" target="_blank" rel="noopener">built-in</a> or <a href="https://en.wikipedia.org/wiki/Coroutine#Implementations" target="_blank" rel="noopener">through libraries or plugins</a>. Here are two examples in <a href="https://kotlinlang.org" target="_blank" rel="noopener">Kotlin</a> and <a href="https://www.python.org/" target="_blank" rel="noopener">Python</a><a href="#fn2" class="footnote-ref" id="fnref2" role="doc-noteref"><sup>2</sup></a>:</p>
<div id="lst:kotlin-coroutine" class="listing numberSource kotlin">
<div class="sourceCode" id="cb1" data-lang="kotlin"><pre class="sourceCode numberSource kotlin"><code class="sourceCode kotlin"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">fun</span> <span class="fu">main</span><span class="op">()</span> <span class="op">=</span> runBlocking <span class="op">{</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> launch <span class="op">{</span> <span class="co">// launch a new coroutine and continue</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> delay<span class="op">(</span><span class="dv">1000L</span><span class="op">)</span> <span class="co">// non-blocking delay for 1 second</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> println<span class="op">(</span><span class="st">"World!"</span><span class="op">)</span> <span class="co">// print after delay</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="op">}</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> println<span class="op">(</span><span class="st">"Hello"</span><span class="op">)</span> <span class="co">// main coroutine</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="op">}</span></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="co">// prints "Hello World!"</span></span></code></pre></div>
<p>Coroutines in Kotlin</p>
</div>
<div id="lst:python-coroutine" class="listing numberSource python">
<div class="sourceCode" id="cb2" data-lang="python"><pre class="sourceCode numberSource python"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> asyncio</span>
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a><span class="cf">async</span> <span class="kw">def</span> say_after(delay, what):</span>
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a> <span class="cf">await</span> asyncio.sleep(delay)</span>
<span id="cb2-5"><a href="#cb2-5" aria-hidden="true" tabindex="-1"></a> <span class="bu">print</span>(what, end<span class="op">=</span><span class="st">""</span>)</span>
<span id="cb2-6"><a href="#cb2-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb2-7"><a href="#cb2-7" aria-hidden="true" tabindex="-1"></a><span class="cf">async</span> <span class="kw">def</span> main():</span>
<span id="cb2-8"><a href="#cb2-8" aria-hidden="true" tabindex="-1"></a> <span class="cf">await</span> asyncio.gather(</span>
<span id="cb2-9"><a href="#cb2-9" aria-hidden="true" tabindex="-1"></a> say_after(<span class="dv">1</span>, <span class="st">'World!</span><span class="ch">\n</span><span class="st">'</span>),</span>
<span id="cb2-10"><a href="#cb2-10" aria-hidden="true" tabindex="-1"></a> say_after(<span class="dv">0</span>, <span class="st">'Hello '</span>))</span>
<span id="cb2-11"><a href="#cb2-11" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb2-12"><a href="#cb2-12" aria-hidden="true" tabindex="-1"></a>asyncio.run(main())</span>
<span id="cb2-13"><a href="#cb2-13" aria-hidden="true" tabindex="-1"></a><span class="co"># prints "Hello World!"</span></span></code></pre></div>
<p>Coroutines in Python</p>
</div>
<p>Now, for a different kind of example, the following <a href="https://en.wikipedia.org/wiki/JavaScript" target="_blank" rel="noopener">JavaScript</a> code prints numbers 11–16 and 1–4 interleaved, using <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Generator" target="_blank" rel="noopener">Generators</a><a href="#fn3" class="footnote-ref" id="fnref3" role="doc-noteref"><sup>3</sup></a>:</p>
<div id="lst:javascript-coroutine" class="listing numberSource javascript">
<div class="sourceCode" id="cb3" data-lang="javascript"><pre class="sourceCode numberSource javascript"><code class="sourceCode javascript"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span><span class="op">*</span> <span class="fu">printNums</span>(start<span class="op">,</span> end) {</span>
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a> <span class="cf">for</span> (<span class="kw">let</span> i <span class="op">=</span> start<span class="op">;</span> i <span class="op"><</span> end <span class="op">+</span> <span class="dv">1</span><span class="op">;</span> i<span class="op">++</span>) {</span>
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a> <span class="bu">console</span><span class="op">.</span><span class="fu">log</span>(i)<span class="op">;</span></span>
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">yield</span><span class="op">;</span></span>
<span id="cb3-5"><a href="#cb3-5" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb3-6"><a href="#cb3-6" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb3-7"><a href="#cb3-7" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb3-8"><a href="#cb3-8" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">run</span>(<span class="op">...</span>gens) {</span>
<span id="cb3-9"><a href="#cb3-9" aria-hidden="true" tabindex="-1"></a> <span class="kw">const</span> queue <span class="op">=</span> [<span class="op">...</span>gens]<span class="op">;</span></span>
<span id="cb3-10"><a href="#cb3-10" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (queue<span class="op">.</span><span class="at">length</span> <span class="op">!=</span> <span class="dv">0</span>) {</span>
<span id="cb3-11"><a href="#cb3-11" aria-hidden="true" tabindex="-1"></a> <span class="kw">const</span> p <span class="op">=</span> queue<span class="op">.</span><span class="fu">shift</span>()<span class="op">;</span></span>
<span id="cb3-12"><a href="#cb3-12" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (<span class="op">!</span>p<span class="op">.</span><span class="fu">next</span>()<span class="op">.</span><span class="at">done</span>) {</span>
<span id="cb3-13"><a href="#cb3-13" aria-hidden="true" tabindex="-1"></a> queue<span class="op">.</span><span class="fu">push</span>(p)<span class="op">;</span></span>
<span id="cb3-14"><a href="#cb3-14" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb3-15"><a href="#cb3-15" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb3-16"><a href="#cb3-16" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb3-17"><a href="#cb3-17" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb3-18"><a href="#cb3-18" aria-hidden="true" tabindex="-1"></a><span class="fu">run</span>(<span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)<span class="op">,</span> <span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>))<span class="op">;</span></span>
<span id="cb3-19"><a href="#cb3-19" aria-hidden="true" tabindex="-1"></a><span class="co">// prints numbers 11–16 and 1–4, interleaved.</span></span></code></pre></div>
<p>Generators in JavaScript</p>
</div>
<p>The next example is in <span class="fancy">Co</span>, and it has the same behaviour as the JavaScript example above, except we don’t have to write the function to schedule and run the coroutines. The runtime for <span class="fancy">Co</span>—the <span class="fancy">Co</span> interpreter—does that implicitly for us.</p>
<div id="lst:co-coroutine" class="listing javascript numberSource">
<div class="sourceCode" id="cb4" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">printNums</span>(start<span class="op">,</span> end) {</span>
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> start<span class="op">;</span></span>
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> end <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(i)<span class="op">;</span></span>
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">yield</span><span class="op">;</span></span>
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)<span class="op">;</span></span>
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a><span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)<span class="op">;</span></span></code></pre></div>
<p>Coroutine example in <span class="fancy">Co</span></p>
</div>
<p>So how are coroutines implemented in <span class="fancy">Co</span>? Let’s find out.</p>
<h2 data-track-content data-content-name="implementing-coroutines" data-content-piece="implementing-co-3" id="implementing-coroutines">Implementing Coroutines</h2>
<p>A coroutine is essentially an <em>Environment</em><sup><a href="#ref-Abelson1996-c32" class="citation" title="Abelson, Sussman, and with Julie Sussman, “The Environment Model of
Evaluation.”
">@6</a></sup> and a <em>Continuation</em><sup><a href="#ref-Reynolds1993-dc" class="citation" title="Reynolds, “The Discoveries of Continuations.”
">@7</a></sup>. The environment is the state of the executing code at the point of yielding. The continuation is the code to be executed when the coroutine is resumed later. If we can capture the environment and the continuation, we can implement coroutines.</p>
<p>Different implementations of coroutines capture the environment and the continuation in different ways<a href="#fn4" class="footnote-ref" id="fnref4" role="doc-noteref"><sup>4</sup></a>:</p>
<ul>
<li>We can capture the environment as the current stack and the continuation as the pointer to the next instruction to be executed at the level of <a href="https://en.wikipedia.org/wiki/machine_code" target="_blank" rel="noopener">machine code</a>. This is how coroutines are implemented in C and C++.</li>
<li>We can transform the code into a state machine as a large switch statement, and use variables to store the environment. This is how Go threads are implemented in the Clojure <a href="https://clojure.github.io/core.async/" target="_blank" rel="noopener">core.async</a><a href="#fn5" class="footnote-ref" id="fnref5" role="doc-noteref"><sup>5</sup></a> library.</li>
<li>We can capture the environment and the continuation as a <a href="https://en.wikipedia.org/wiki/Closure_(computer_programming)" target="_blank" rel="noopener"><em>Closure</em></a>. To do this, we need to first transform the code into <a href="https://en.wikipedia.org/wiki/Continuation-passing_style" target="_blank" rel="noopener"><em>Continuation-passing style</em></a> (CPS), so that we have the handle to the continuation at every point in the code. This is how we are going to implement coroutines in <span class="fancy">Co</span>.</li>
</ul>
<p>Let’s learn what <abbr title="Continuation-passing style">CPS</abbr> is, and how we can use it to implement coroutines.</p>
<h2 data-track-content data-content-name="continuation-passing-style" data-content-piece="implementing-co-3" id="continuation-passing-style">Continuation-Passing Style</h2>
<p>In the usual direct programming style, we write one statement or function call after another, as a sequence of steps to execute. There is another way of thinking about program execution: after returning from executing one statement/function, the rest of the program—which can be thought of as a big statement/function itself—is run. In <abbr title="Continuation-passing style">CPS</abbr>, this is made explicit: each statement/function takes the rest of the program that comes after it as an argument, which it invokes explicitly. For example, if we have a program to get the recommendations for a user and print them, written in direct style like this:</p>
<div class="sourceCode" id="cb5" data-lang="javascript"><pre class="sourceCode numberSource javascript"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">getUserRecommendations</span>(userId) {</span>
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> user <span class="op">=</span> <span class="fu">getUser</span>(userId)<span class="op">;</span></span>
<span id="cb5-3"><a href="#cb5-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> friends <span class="op">=</span> <span class="fu">getFriends</span>(user)<span class="op">;</span></span>
<span id="cb5-4"><a href="#cb5-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> recommendations <span class="op">=</span> <span class="fu">getRecommendations</span>(friends)<span class="op">;</span></span>
<span id="cb5-5"><a href="#cb5-5" aria-hidden="true" tabindex="-1"></a> <span class="fu">recordRecommendations</span>(userId<span class="op">,</span> recommendations)<span class="op">;</span></span>
<span id="cb5-6"><a href="#cb5-6" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> recommendations<span class="op">;</span></span>
<span id="cb5-7"><a href="#cb5-7" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb5-8"><a href="#cb5-8" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-9"><a href="#cb5-9" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">main</span>() {</span>
<span id="cb5-10"><a href="#cb5-10" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> recommendations <span class="op">=</span> <span class="fu">getUserRecommendations</span>(<span class="dv">123</span>)<span class="op">;</span></span>
<span id="cb5-11"><a href="#cb5-11" aria-hidden="true" tabindex="-1"></a> <span class="bu">console</span><span class="op">.</span><span class="fu">log</span>(recommendations)<span class="op">;</span></span>
<span id="cb5-12"><a href="#cb5-12" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
<p>It can be converted to an equivalent <abbr title="Continuation-passing style">CPS</abbr> program like this:</p>
<div class="sourceCode" id="cb6" data-lang="javascript"><pre class="sourceCode numberSource javascript"><code class="sourceCode javascript"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">getUserRecommendationsCPS</span>(userId<span class="op">,</span> cont) {</span>
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">getUserCPS</span>(userId<span class="op">,</span> (user) <span class="kw">=></span> {</span>
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">getFriendsCPS</span>(user<span class="op">,</span> (friends) <span class="kw">=></span> {</span>
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">getRecommendationsCPS</span>(friends<span class="op">,</span> (recommendations) <span class="kw">=></span> {</span>
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a> <span class="fu">recordRecommendationsCPS</span>(userId<span class="op">,</span> recommendations<span class="op">,</span> () <span class="kw">=></span> {</span>
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a> <span class="fu">cont</span>(recommendations)<span class="op">;</span></span>
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">mainCPS</span>() {</span>
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a> <span class="fu">getUserRecommendationsCPS</span>(<span class="dv">123</span><span class="op">,</span> (recommendations) <span class="kw">=></span> {</span>
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a> <span class="bu">console</span><span class="op">.</span><span class="fu">log</span>(recommendations)<span class="op">;</span></span>
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
<p>We see how each function takes the rest of the program after it captured as a function, as a parameter, and calls it explicitly to further the flow of the program. Instead of returning the recommendations, the <code>getUserRecommendationsCPS</code> function now takes a function as an additional parameter, which it calls with the recommendations at the end of all the processing. Same for all the other functions invoked in the program. These functions passed as arguments are known as continuations because they <strong>continue</strong> the execution of the programs when called, and hence this style is called the continuation-passing style. The <code>cont</code> function is the continuation here.</p>
<details>
<summary>
The rest of the functions can be written in <abbr title="Continuation-passing style">CPS</abbr> like this:
</summary>
<div class="sourceCode" id="cb7" data-lang="javascript"><pre class="sourceCode numberSource javascript"><code class="sourceCode javascript"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">getUserCPS</span>(userId<span class="op">,</span> cont) {</span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> user <span class="op">=</span> <span class="fu">getUser</span>(userId)<span class="op">;</span></span>
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">cont</span>(user)<span class="op">;</span></span>
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">getFriendsCPS</span>(user<span class="op">,</span> cont) {</span>
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> friends <span class="op">=</span> <span class="fu">getFriends</span>(user)<span class="op">;</span></span>
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">cont</span>(friends)<span class="op">;</span></span>
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">getRecommendationsCPS</span>(friends<span class="op">,</span> cont) {</span>
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> recommendations <span class="op">=</span> <span class="fu">getRecommendations</span>(friends)<span class="op">;</span></span>
<span id="cb7-13"><a href="#cb7-13" aria-hidden="true" tabindex="-1"></a> <span class="fu">cont</span>(recommendations)<span class="op">;</span></span>
<span id="cb7-14"><a href="#cb7-14" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb7-15"><a href="#cb7-15" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb7-16"><a href="#cb7-16" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">recordRecommendationsCPS</span>(userId<span class="op">,</span> recommendations<span class="op">,</span> cont) {</span>
<span id="cb7-17"><a href="#cb7-17" aria-hidden="true" tabindex="-1"></a> <span class="fu">recordRecommendations</span>(userId<span class="op">,</span> recommendations)<span class="op">;</span></span>
<span id="cb7-18"><a href="#cb7-18" aria-hidden="true" tabindex="-1"></a> <span class="fu">cont</span>()<span class="op">;</span></span>
<span id="cb7-19"><a href="#cb7-19" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
</details>
<p>So, what is the point of all this? Why transform code into <abbr title="Continuation-passing style">CPS</abbr>? Since, in <abbr title="Continuation-passing style">CPS</abbr> the rest of the program is passed as a function, a program can itself explicitly manipulate the flow of control of the program. This lets us do things like<a href="#fn6" class="footnote-ref" id="fnref6" role="doc-noteref"><sup>6</sup></a>:</p>
<ul>
<li>Returning early from a function by calling the continuation with the return value, and not executing the rest of the function.</li>
<li>Implementing exceptions by passing two continuations: one for the normal flow of the program, and another for the exceptional flow.</li>
<li>Implementing non-deterministic programs by passing continuations for backtracking to previous states of the program.</li>
<li>Converting potentially stack-blowing recursive programs into iterative programs by passing the continuation as a parameter to the recursive function.</li>
<li>Suspending the execution of the program by storing the continuation, and resuming it later.</li>
</ul>
<p>We can now begin to see how <abbr title="Continuation-passing style">CPS</abbr> can be used to implement coroutines.</p>
<h3 id="continuation-passing-style-in-haskell">Continuation-Passing Style in Haskell</h3>
<p>It is straightforward to translate the <a href="#cb5-1">above program</a> into Haskell:</p>
<div class="sourceCode" id="cb8" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="ot">getUserRecommendations ::</span> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Int</span> <span class="ot">-></span> m <span class="dt">Recommendations</span></span>
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a>getUserRecommendations userId <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a> user <span class="ot"><-</span> getUser userId</span>
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a> friends <span class="ot"><-</span> getFriends user</span>
<span id="cb8-5"><a href="#cb8-5" aria-hidden="true" tabindex="-1"></a> recommendations <span class="ot"><-</span> getRecommendations friends</span>
<span id="cb8-6"><a href="#cb8-6" aria-hidden="true" tabindex="-1"></a> recordRecommendations userId recommendations</span>
<span id="cb8-7"><a href="#cb8-7" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> recommendations</span>
<span id="cb8-8"><a href="#cb8-8" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb8-9"><a href="#cb8-9" aria-hidden="true" tabindex="-1"></a><span class="ot">main ::</span> <span class="dt">IO</span> ()</span>
<span id="cb8-10"><a href="#cb8-10" aria-hidden="true" tabindex="-1"></a>main <span class="ot">=</span> getUserRecommendations <span class="dv">123</span> <span class="op">>>=</span> <span class="fu">print</span></span></code></pre></div>
<p>And the <abbr title="Continuation-passing style">CPS</abbr> versions:</p>
<div class="sourceCode" id="cb9" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="ot">getUserRecommendationsCPS ::</span></span>
<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Int</span> <span class="ot">-></span> (<span class="dt">Recommendations</span> <span class="ot">-></span> m a) <span class="ot">-></span> m a</span>
<span id="cb9-3"><a href="#cb9-3" aria-hidden="true" tabindex="-1"></a>getUserRecommendationsCPS userId cont <span class="ot">=</span></span>
<span id="cb9-4"><a href="#cb9-4" aria-hidden="true" tabindex="-1"></a> getUserCPS userId <span class="op">$</span> \user <span class="ot">-></span></span>
<span id="cb9-5"><a href="#cb9-5" aria-hidden="true" tabindex="-1"></a> getFriendsCPS user <span class="op">$</span> \friends <span class="ot">-></span></span>
<span id="cb9-6"><a href="#cb9-6" aria-hidden="true" tabindex="-1"></a> getRecommendationsCPS friends <span class="op">$</span> \recommendations <span class="ot">-></span></span>
<span id="cb9-7"><a href="#cb9-7" aria-hidden="true" tabindex="-1"></a> recordRecommendationsCPS userId recommendations <span class="op">$</span> \_ <span class="ot">-></span></span>
<span id="cb9-8"><a href="#cb9-8" aria-hidden="true" tabindex="-1"></a> cont recommendations</span>
<span id="cb9-9"><a href="#cb9-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb9-10"><a href="#cb9-10" aria-hidden="true" tabindex="-1"></a><span class="ot">getUserCPS ::</span> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Int</span> <span class="ot">-></span> (<span class="dt">User</span> <span class="ot">-></span> m a) <span class="ot">-></span> m a</span>
<span id="cb9-11"><a href="#cb9-11" aria-hidden="true" tabindex="-1"></a>getUserCPS userId cont <span class="ot">=</span> getUser userId <span class="op">>>=</span> cont</span>
<span id="cb9-12"><a href="#cb9-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb9-13"><a href="#cb9-13" aria-hidden="true" tabindex="-1"></a><span class="ot">getFriendsCPS ::</span> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">User</span> <span class="ot">-></span> (<span class="dt">Friends</span> <span class="ot">-></span> m a) <span class="ot">-></span> m a</span>
<span id="cb9-14"><a href="#cb9-14" aria-hidden="true" tabindex="-1"></a>getFriendsCPS user cont <span class="ot">=</span> getFriends user <span class="op">>>=</span> cont</span>
<span id="cb9-15"><a href="#cb9-15" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb9-16"><a href="#cb9-16" aria-hidden="true" tabindex="-1"></a><span class="ot">getRecommendationsCPS ::</span></span>
<span id="cb9-17"><a href="#cb9-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Friends</span> <span class="ot">-></span> (<span class="dt">Recommendations</span> <span class="ot">-></span> m a) <span class="ot">-></span> m a</span>
<span id="cb9-18"><a href="#cb9-18" aria-hidden="true" tabindex="-1"></a>getRecommendationsCPS friends cont <span class="ot">=</span></span>
<span id="cb9-19"><a href="#cb9-19" aria-hidden="true" tabindex="-1"></a> getRecommendations friends <span class="op">>>=</span> cont</span>
<span id="cb9-20"><a href="#cb9-20" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb9-21"><a href="#cb9-21" aria-hidden="true" tabindex="-1"></a><span class="ot">recordRecommendationsCPS ::</span></span>
<span id="cb9-22"><a href="#cb9-22" aria-hidden="true" tabindex="-1"></a> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Int</span> <span class="ot">-></span> <span class="dt">Recommendations</span> <span class="ot">-></span> (() <span class="ot">-></span> m a) <span class="ot">-></span> m a</span>
<span id="cb9-23"><a href="#cb9-23" aria-hidden="true" tabindex="-1"></a>recordRecommendationsCPS userId recommendations cont <span class="ot">=</span></span>
<span id="cb9-24"><a href="#cb9-24" aria-hidden="true" tabindex="-1"></a> recordRecommendations userId recommendations <span class="op">>></span> cont ()</span>
<span id="cb9-25"><a href="#cb9-25" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb9-26"><a href="#cb9-26" aria-hidden="true" tabindex="-1"></a><span class="ot">mainCPS ::</span> <span class="dt">IO</span> ()</span>
<span id="cb9-27"><a href="#cb9-27" aria-hidden="true" tabindex="-1"></a>mainCPS <span class="ot">=</span> getUserRecommendationsCPS <span class="dv">123</span> <span class="op">$</span> <span class="fu">print</span></span></code></pre></div>
<p>We can immediately notice a pattern in the type signatures of the functions above: they are all of the form:</p>
<div class="sourceCode" id="cb10" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="ot">f ::</span> <span class="dt">Monad</span> m <span class="ot">=></span> b <span class="ot">-></span> (a <span class="ot">-></span> m r) <span class="ot">-></span> m r</span></code></pre></div>
<p>It is indeed a known pattern, and is captured by the <code class="sourceCode haskell"><span class="dt">ContT</span></code> type:</p>
<div class="sourceCode" id="cb11" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="kw">newtype</span> <span class="dt">ContT</span> r m a <span class="ot">=</span></span>
<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">ContT</span> {<span class="ot"> runContT ::</span> (a <span class="ot">-></span> m r) <span class="ot">-></span> m r }</span></code></pre></div>
<p>Turns out, the <a href="https://hackage.haskell.org/package/mtl/docs/Control-Monad-Cont.html#t:ContT" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">ContT</span></code></a> type is a monad transformer, and we can use it to write the above <abbr title="Continuation-passing style">CPS</abbr> program in a more concise way<a href="#fn7" class="footnote-ref" id="fnref7" role="doc-noteref"><sup>7</sup></a>:</p>
<div class="sourceCode" id="cb12" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="ot">getUserRecommendationsCont ::</span></span>
<span id="cb12-2"><a href="#cb12-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Int</span> <span class="ot">-></span> <span class="dt">ContT</span> r m <span class="dt">Recommendations</span></span>
<span id="cb12-3"><a href="#cb12-3" aria-hidden="true" tabindex="-1"></a>getUserRecommendationsCont userId <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb12-4"><a href="#cb12-4" aria-hidden="true" tabindex="-1"></a> user <span class="ot"><-</span> getUserCont userId</span>
<span id="cb12-5"><a href="#cb12-5" aria-hidden="true" tabindex="-1"></a> friends <span class="ot"><-</span> getFriendsCont user</span>
<span id="cb12-6"><a href="#cb12-6" aria-hidden="true" tabindex="-1"></a> recommendations <span class="ot"><-</span> getRecommendationsCont friends</span>
<span id="cb12-7"><a href="#cb12-7" aria-hidden="true" tabindex="-1"></a> recordRecommendationsCont userId recommendations</span>
<span id="cb12-8"><a href="#cb12-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> recommendations</span>
<span id="cb12-9"><a href="#cb12-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-10"><a href="#cb12-10" aria-hidden="true" tabindex="-1"></a><span class="ot">getUserCont ::</span> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Int</span> <span class="ot">-></span> <span class="dt">ContT</span> r m <span class="dt">User</span></span>
<span id="cb12-11"><a href="#cb12-11" aria-hidden="true" tabindex="-1"></a>getUserCont userId <span class="ot">=</span> <span class="dt">ContT</span> (getUser userId <span class="op">>>=</span>)</span>
<span id="cb12-12"><a href="#cb12-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-13"><a href="#cb12-13" aria-hidden="true" tabindex="-1"></a><span class="ot">getFriendsCont ::</span> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">User</span> <span class="ot">-></span> <span class="dt">ContT</span> r m <span class="dt">Friends</span></span>
<span id="cb12-14"><a href="#cb12-14" aria-hidden="true" tabindex="-1"></a>getFriendsCont user <span class="ot">=</span> <span class="dt">ContT</span> (getFriends user <span class="op">>>=</span>)</span>
<span id="cb12-15"><a href="#cb12-15" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-16"><a href="#cb12-16" aria-hidden="true" tabindex="-1"></a><span class="ot">getRecommendationsCont ::</span></span>
<span id="cb12-17"><a href="#cb12-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Friends</span> <span class="ot">-></span> <span class="dt">ContT</span> r m <span class="dt">Recommendations</span></span>
<span id="cb12-18"><a href="#cb12-18" aria-hidden="true" tabindex="-1"></a>getRecommendationsCont friends <span class="ot">=</span></span>
<span id="cb12-19"><a href="#cb12-19" aria-hidden="true" tabindex="-1"></a> <span class="dt">ContT</span> (getRecommendations friends <span class="op">>>=</span>)</span>
<span id="cb12-20"><a href="#cb12-20" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-21"><a href="#cb12-21" aria-hidden="true" tabindex="-1"></a><span class="ot">recordRecommendationsCont ::</span></span>
<span id="cb12-22"><a href="#cb12-22" aria-hidden="true" tabindex="-1"></a> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Int</span> <span class="ot">-></span> <span class="dt">Recommendations</span> <span class="ot">-></span> <span class="dt">ContT</span> r m ()</span>
<span id="cb12-23"><a href="#cb12-23" aria-hidden="true" tabindex="-1"></a>recordRecommendationsCont userId recommendations <span class="ot">=</span></span>
<span id="cb12-24"><a href="#cb12-24" aria-hidden="true" tabindex="-1"></a> <span class="dt">ContT</span> <span class="op">$</span> \cont <span class="ot">-></span></span>
<span id="cb12-25"><a href="#cb12-25" aria-hidden="true" tabindex="-1"></a> recordRecommendations userId recommendations <span class="op">>></span> cont ()</span>
<span id="cb12-26"><a href="#cb12-26" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-27"><a href="#cb12-27" aria-hidden="true" tabindex="-1"></a><span class="ot">mainCont ::</span> <span class="dt">IO</span> ()</span>
<span id="cb12-28"><a href="#cb12-28" aria-hidden="true" tabindex="-1"></a>mainCont <span class="ot">=</span> runContT (getUserRecommendationsCont <span class="dv">123</span>) <span class="fu">print</span></span></code></pre></div>
<p>So we have come full circle: we started with <a href="#cb8-1">monadic code</a>, and ended with similar <a href="#cb12-1">monadic code</a>, but with a different monad. So what did we gain from this transformation? Well, we can now use the <a href="https://hackage.haskell.org/package/mtl/docs/Control-Monad-Cont.html#v:callCC" target="_blank" rel="noopener"><code class="sourceCode haskell">callCC</code></a> function provided by <code class="sourceCode haskell"><span class="dt">ContT</span></code>.</p>
<h3 id="call-with-current-continuation">Call with Current Continuation</h3>
<p><code>callCC</code>—short for “call with current continuation”—is a function that provides on-demand access to the current continuation at any point in the code, just like we had in the <a href="#cb9-1">CPS version</a> of the program. At the same time, by using <code class="sourceCode haskell"><span class="dt">ConT</span></code> we can write the program again in the concise monadic style<a href="#fn8" class="footnote-ref" id="fnref8" role="doc-noteref"><sup>8</sup></a>.</p>
<p>The following example uses <code>callCC</code> to print the user recommendation twice, instead of once<a href="#fn9" class="footnote-ref" id="fnref9" role="doc-noteref"><sup>9</sup></a>:</p>
<div class="sourceCode" id="cb13" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="ot">getUserRecommendationsCont2 ::</span></span>
<span id="cb13-2"><a href="#cb13-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">Monad</span> m <span class="ot">=></span> <span class="dt">Int</span> <span class="ot">-></span> <span class="dt">ContT</span> r m <span class="dt">Recommendations</span></span>
<span id="cb13-3"><a href="#cb13-3" aria-hidden="true" tabindex="-1"></a>getUserRecommendationsCont2 userId <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb13-4"><a href="#cb13-4" aria-hidden="true" tabindex="-1"></a> user <span class="ot"><-</span> getUserCont userId</span>
<span id="cb13-5"><a href="#cb13-5" aria-hidden="true" tabindex="-1"></a> friends <span class="ot"><-</span> getFriendsCont user</span>
<span id="cb13-6"><a href="#cb13-6" aria-hidden="true" tabindex="-1"></a> recommendations <span class="ot"><-</span> getRecommendationsCont friends</span>
<span id="cb13-7"><a href="#cb13-7" aria-hidden="true" tabindex="-1"></a> logRecommendationsCont userId recommendations</span>
<span id="cb13-8"><a href="#cb13-8" aria-hidden="true" tabindex="-1"></a> callCC <span class="op">$</span> \cont <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb13-9"><a href="#cb13-9" aria-hidden="true" tabindex="-1"></a> cont recommendations</span>
<span id="cb13-10"><a href="#cb13-10" aria-hidden="true" tabindex="-1"></a> cont recommendations</span>
<span id="cb13-11"><a href="#cb13-11" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb13-12"><a href="#cb13-12" aria-hidden="true" tabindex="-1"></a><span class="ot">mainCont2 ::</span> <span class="dt">IO</span> ()</span>
<span id="cb13-13"><a href="#cb13-13" aria-hidden="true" tabindex="-1"></a>mainCont2 <span class="ot">=</span> runContT (getUserRecommendationsCont2 <span class="dv">123</span>) <span class="fu">print</span></span></code></pre></div>
<p>This is the power of <abbr title="Continuation-passing style">CPS</abbr>: it lets the programs manipulate the flow of control explicitly, and in some cases markedly, as we see in the next section.</p>
<h2 data-track-content data-content-name="from-continuations-to-coroutines" data-content-piece="implementing-co-3" id="from-continuations-to-coroutines">From Continuations to Coroutines</h2>
<p>Since continuations are functions, we can store them in data structures. This lets us pause the execution of a <abbr title="Continuation-passing style">CPS</abbr> program at a certain point, and resume it later from the same point. This is exactly what coroutines do.</p>
<p>To implement coroutines in <span class="fancy">Co</span>, first we enhance the <code class="sourceCode haskell"><span class="dt">Interpreter</span></code> monad to be able to capture the current continuation by adding the <code class="sourceCode haskell"><span class="dt">ContT</span></code> monad transformer in the transformer stack:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="4-5,17-17"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">newtype</span> <span class="dt">Interpreter</span> a <span class="ot">=</span> <span class="dt">Interpreter</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> {<span class="ot"> runInterpreter ::</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">ExceptT</span> <span class="dt">Exception</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> (<span class="dt">ContT</span></span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> (<span class="dt">Either</span> <span class="dt">Exception</span> ())</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> (<span class="dt">StateT</span> <span class="dt">InterpreterState</span> <span class="dt">IO</span>))</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> a</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> ( <span class="dt">Functor</span>,</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">Applicative</span>,</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">Monad</span>,</span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">MonadIO</span>,</span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">MonadBase</span> <span class="dt">IO</span>,</span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">MonadState</span> <span class="dt">InterpreterState</span>,</span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">MonadError</span> <span class="dt">Exception</span>,</span>
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">MonadCont</span></span></span>
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a> )</span></code></pre></div>
<p>To be able to pause and resume the <span class="fancy">Co</span> code being interpreted, we need to capture the current interpreter environment as well. The environment contains the bindings that the executing <span class="fancy">Co</span> code sees at any given time. By capturing and later restoring the environment, the code execution resumes with same environment, and hence works correctly.</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="4-4,9-9,10:31-10:36"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Coroutine</span> a <span class="ot">=</span> <span class="dt">Coroutine</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> {<span class="ot"> corEnv ::</span> <span class="dt">Env</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> ,<span class="ot"> corCont ::</span> a <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> ,<span class="ot"> corReady ::</span> <span class="dt">MVar</span> <span class="dt">TimeSpec</span></span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="ot">newCoroutine ::</span> <span class="dt">Env</span> <span class="ot">-></span> (a <span class="ot">-></span> <span class="dt">Interpreter</span> ()) <span class="ot">-></span> <span class="dt">Interpreter</span> (<span class="dt">Coroutine</span> a)</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a>newCoroutine env cont <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> ready <span class="ot"><-</span> newMVar <span class="op">=<<</span> currentSystemTime</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> <span class="op">$</span> <span class="dt">Coroutine</span> env cont <span class="deemphasis">ready</span></span></code></pre></div>
<p>The <code class="sourceCode haskell"><span class="dt">Coroutine</span></code> data type contains the environment and the continuation. The <code>newCoroutine</code> function creates a new coroutine.</p>
<p>Next, we enhance the interpreter state to keep a queue of coroutines to be run.</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="3-3,7:60-7:68"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">InterpreterState</span> <span class="ot">=</span> <span class="dt">InterpreterState</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> {<span class="ot"> isEnv ::</span> <span class="dt">Env</span>,</span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="emphasis"><span class="ot"> isCoroutines ::</span> <span class="dt">Queue</span> (<span class="dt">Coroutine</span> ())</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="ot">initInterpreterState ::</span> <span class="dt">IO</span> <span class="dt">InterpreterState</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a>initInterpreterState <span class="ot">=</span> <span class="dt">InterpreterState</span> <span class="op"><$></span> builtinEnv <span class="op"><*></span> <span class="emphasis">newQueue</span></span></code></pre></div>
<p>We use an <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html#t:IORef" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">IORef</span></code></a> containing a <a href="https://en.wikipedia.org/wiki/min-priority_queue" target="_blank" rel="noopener">min-priority queue</a> to store the coroutines<a href="#fn10" class="footnote-ref" id="fnref10" role="doc-noteref"><sup>10</sup></a>. For now, we use it as a simple <a href="https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics)" target="_blank" rel="noopener">FIFO</a> queue, but we will see in a later post how we use it to implement the <code>sleep</code> functionality in our interpreter.</p>
<p><a id="queue-ds"></a></p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="1:46-1:56,5-5,6:12-6:13,6:21-6:27,9:18-9:19,9:26-9:33"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Queue</span> a <span class="ot">=</span> <span class="dt">IORef</span> (<span class="dt">PQ.MinPQueue</span> <span class="dt">TimeSpec</span> a<span class="deemphasis">, <span class="dt">TimeSpec</span></span>)</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="ot">newQueue ::</span> <span class="dt">MonadBase</span> <span class="dt">IO</span> m <span class="ot">=></span> m (<span class="dt">Queue</span> a)</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a>newQueue <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> now <span class="ot"><-</span> liftBase currentSystemTime</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> newIORef <span class="deemphasis">(</span>PQ.empty<span class="deemphasis">, now)</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="ot">queueSize ::</span> <span class="dt">MonadBase</span> <span class="dt">IO</span> m <span class="ot">=></span> <span class="dt">Queue</span> a <span class="ot">-></span> m <span class="dt">Int</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a>queueSize <span class="ot">=</span> <span class="fu">fmap</span> <span class="deemphasis">(</span>PQ.size<span class="deemphasis"> <span class="op">.</span> <span class="fu">fst</span>)</span> <span class="op">.</span> readIORef</span></code></pre></div>
<p>Now that we know how coroutines are stored in the interpreter, let’s see how we schedule them.</p>
<h2 data-track-content data-content-name="scheduling-coroutines" data-content-piece="implementing-co-3" id="scheduling-coroutines">Scheduling Coroutines</h2>
<p>First step in scheduling coroutines is to write functions to enqueue and dequeue from a queue:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="2:56-2:57,2:58-2:74,3:4-3:5,3:26-3:27,4-4,5:3-5:5,16:45-16:46,16:47-16:63,18:11-18:12,18:13-18:29,20:14-20:15,20:17-20:33"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">enqueueAt ::</span> <span class="dt">TimeSpec</span> <span class="ot">-></span> a <span class="ot">-></span> <span class="dt">Queue</span> a <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>enqueueAt time val queue <span class="ot">=</span> atomicModifyIORef' queue <span class="op">$</span> \<span class="deemphasis">(</span>q<span class="deemphasis">, maxWakeupTime)</span> <span class="ot">-></span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> (<span class="deemphasis">(</span> PQ.insert time val q<span class="deemphasis">,</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="kw">if</span> time <span class="op">></span> maxWakeupTime <span class="kw">then</span> time <span class="kw">else</span> maxWakeupTime</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="deemphasis"> )</span>, ())</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="ot">enqueue ::</span> a <span class="ot">-></span> <span class="dt">Queue</span> a <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a>enqueue val queue <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> now <span class="ot"><-</span> currentSystemTime</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> enqueueAt now val queue</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a><span class="ot">currentSystemTime ::</span> <span class="dt">MonadIO</span> m <span class="ot">=></span> m <span class="dt">TimeSpec</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a>currentSystemTime <span class="ot">=</span> liftIO <span class="op">$</span> getTime <span class="dt">Monotonic</span></span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a><span class="ot">dequeue ::</span> <span class="dt">Queue</span> a <span class="ot">-></span> <span class="dt">Interpreter</span> (<span class="dt">Maybe</span> a)</span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a>dequeue queue <span class="ot">=</span> atomicModifyIORef' queue <span class="op">$</span> \<span class="deemphasis">(</span>q<span class="deemphasis">, maxWakeupTime)</span> <span class="ot">-></span></span>
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a> <span class="kw">if</span> PQ.null q</span>
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a> <span class="kw">then</span> (<span class="deemphasis">(</span>q<span class="deemphasis">, maxWakeupTime)</span>, <span class="dt">Nothing</span>)</span>
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a> <span class="kw">else</span> <span class="kw">let</span> ((_, val), q') <span class="ot">=</span> PQ.deleteFindMin q</span>
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a> <span class="kw">in</span> (<span class="deemphasis">(</span>q'<span class="deemphasis">, maxWakeupTime)</span>, <span class="dt">Just</span> val)</span></code></pre></div>
<p>To use the min-priority queue as a FIFO queue, we use the current system time—which is a monotonically increasing value—as the priority of the values in the queue. This way, the coroutines are scheduled in the order they are enqueued.</p>
<p>The <code>enqueueAt</code> function enqueues the given value at the given time in the queue. The <code>enqueue</code> function enqueues the value at the current time, thus scheduling it to run immediately.</p>
<p>The <code>dequeue</code> function dequeues the value with the lowest priority from the queue, which in this case, is the value that is enqueued first.</p>
<p>The <code>currentSystemTime</code> function returns the monotonically increasing current system time.</p>
<p>Over these queuing primitives, we build the coroutine scheduling functions:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="10-10"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">scheduleCoroutine ::</span> <span class="dt">Coroutine</span> () <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>scheduleCoroutine coroutine <span class="ot">=</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> State.gets isCoroutines <span class="op">>>=</span> enqueue coroutine</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="ot">runNextCoroutine ::</span> <span class="dt">Interpreter</span> ()</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a>runNextCoroutine <span class="ot">=</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> State.gets isCoroutines <span class="op">>>=</span> dequeue <span class="op">>>=</span> \<span class="kw">case</span></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">Nothing</span> <span class="ot">-></span> throwError <span class="dt">CoroutineQueueEmpty</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">Just</span> <span class="dt">Coroutine</span> {<span class="op">..</span>} <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> void <span class="op">$</span> takeMVar corReady</span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> setEnv corEnv</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> corCont ()</span></code></pre></div>
<p>The <code>scheduleCoroutine</code> function takes a coroutine, and schedules it by enqueuing it in the coroutine queue in the interpreter state.</p>
<p>The <code>runNextCoroutine</code> function dequeues the next coroutine from the queue, and runs it. It first restores the environment of the coroutine in the interpreter state, and then runs the continuation of the coroutine. If the queue is empty, it throws a <code class="sourceCode haskell"><span class="dt">CoroutineQueueEmpty</span></code> exception, which we add in the <code class="sourceCode haskell"><span class="dt">Exception</span></code> data type:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="4-4"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Exception</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">Return</span> <span class="dt">Value</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">RuntimeError</span> <span class="dt">String</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op">|</span> <span class="dt">CoroutineQueueEmpty</span></span></span></code></pre></div>
<p>The <code>runNextCoroutine</code> function is the heart of the coroutine scheduling. It is called at the end of every function related to coroutines in the interpreter, and that’s how the coroutines are run one-after-another. Next, we see how we use these functions to implement the <code class="sourceCode javascript"><span class="kw">yield</span></code> and <code class="sourceCode"><span class="cf">spawn</span></code> statements in <span class="fancy">Co</span>.</p>
<h2 data-track-content data-content-name="yield-and-spawn" data-content-piece="implementing-co-3" id="yield-and-spawn">Yield and Spawn</h2>
<p>Let’s recall the program we used to demonstrate coroutines:</p>
<div class="sourceCode" id="cb14" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">printNums</span>(start<span class="op">,</span> end) {</span>
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> start<span class="op">;</span></span>
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> end <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(i)<span class="op">;</span></span>
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">yield</span><span class="op">;</span></span>
<span id="cb14-6"><a href="#cb14-6" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb14-7"><a href="#cb14-7" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb14-8"><a href="#cb14-8" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb14-9"><a href="#cb14-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb14-10"><a href="#cb14-10" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)<span class="op">;</span></span>
<span id="cb14-11"><a href="#cb14-11" aria-hidden="true" tabindex="-1"></a><span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)<span class="op">;</span></span></code></pre></div>
<p>Running this program with the interpreter produces the following output:</p>
<pre class="plain"><code>11
1
12
2
13
3
14
4
15
16</code></pre>
<p>As we see, the numbers printed by the <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)</code> function call are interleaved with the ones printed by the <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)</code> call.</p>
<p>This is how the code is interpreted:</p>
<ol type="1">
<li>First, the definition of the function <code class="sourceCode javascript">printNums</code> executes. The function gets stored in the environment as a <code class="sourceCode haskell"><span class="dt">Function</span></code> value.</li>
<li>The <code class="sourceCode javascript">spawn <span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)</code> statement executes. The <code class="sourceCode"><span class="cf">spawn</span></code> statement creates a new coroutine for the function call <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)</code> and schedules it.</li>
<li><details>
<summary>
The <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)</code> function call executes, prints <code class="sourceCode javascript"><span class="dv">11</span></code> and yields.
</summary>
<ol type="i">
<li>The <code class="sourceCode javascript"><span class="cf">while</span></code> loop executes, and the <code>print</code> statement prints the value of the variable <code>i</code>, which is <code class="sourceCode javascript"><span class="dv">11</span></code> at this point.</li>
<li>The <code class="sourceCode javascript"><span class="kw">yield</span></code> statement executes. This creates a new coroutine for the rest of the call execution, and schedules it. The call execution suspends at this point.</li>
</ol>
</details></li>
<li><details>
<summary>
The <code>runNextCoroutine</code> function executes, which dequeues the coroutine for the <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)</code> call, and runs it. This prints <code class="sourceCode javascript"><span class="dv">1</span></code> and yields.
</summary>
<ol type="i">
<li>The <code class="sourceCode javascript"><span class="cf">while</span></code> loop executes, and the <code>print</code> statement prints the value of the variable <code>i</code>, which is <code class="sourceCode javascript"><span class="dv">1</span></code> at this point.</li>
<li>The <code class="sourceCode javascript"><span class="kw">yield</span></code> statement executes. This creates a new coroutine for the rest of the call execution, and schedules it. The call execution suspends at this point.</li>
</ol>
</details></li>
<li><details>
<summary>
The <code>runNextCoroutine</code> function executes again, which dequeues the coroutine for the <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)</code> call, and runs it. This prints <code class="sourceCode javascript"><span class="dv">12</span></code> and yields.
</summary>
<ol type="i">
<li>The call resumes after the <code class="sourceCode javascript"><span class="kw">yield</span></code> statement. The <code class="sourceCode javascript"><span class="cf">while</span></code> loop executes again, and the <code>print</code> statement prints the value of the variable <code>i</code>, which is <code class="sourceCode javascript"><span class="dv">12</span></code> at this point.</li>
<li>The function execution suspends at the <code class="sourceCode javascript"><span class="kw">yield</span></code> statement again.</li>
</ol>
</details></li>
<li><details>
<summary>
The <code>runNextCoroutine</code> function executes again, which dequeues the coroutine for the <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)</code> call, and runs it. This prints <code class="sourceCode javascript"><span class="dv">2</span></code> and yields.
</summary>
<ol type="i">
<li>The call resumes after the <code class="sourceCode javascript"><span class="kw">yield</span></code> statement. The <code class="sourceCode javascript"><span class="cf">while</span></code> loop executes again, and the <code>print</code> statement prints the value of the variable <code>i</code>, which is <code class="sourceCode javascript"><span class="dv">2</span></code> at this point.</li>
<li>The function execution suspends at the <code class="sourceCode javascript"><span class="kw">yield</span></code> statement again.</li>
</ol>
</details></li>
<li>This back-and-forth process of suspension and resumption of function executions continues until the <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)</code> call returns after printing the number <code class="sourceCode javascript"><span class="dv">4</span></code>.</li>
<li>After that, the call <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)</code> resumes to print the numbers and yields, again and again, until it returns after printing the number <code class="sourceCode javascript"><span class="dv">16</span></code>.</li>
<li>Interpretation ends.</li>
</ol>
<p>The diagram below depicts this process in abstract:</p>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 673 849'%3E%3C/svg%3E" class="lazyload ascii-art nolink w-80pct" style="--image-aspect-ratio: 0.7926972909305064" data-src="/images/implementing-co-3/coroutine-scheduling.svg" alt="Spawning, yielding, and running coroutines"></img>
<noscript><img src="/images/implementing-co-3/coroutine-scheduling.svg" class="ascii-art nolink w-80pct" alt="Spawning, yielding, and running coroutines"></img></noscript>
<figcaption>Spawning, yielding, and running coroutines</figcaption>
</figure>
<p>With the understanding of how they work, let’s see how to implement the <code class="sourceCode javascript"><span class="kw">yield</span></code> and <code class="sourceCode"><span class="cf">spawn</span></code> statements in <span class="fancy">Co</span>.</p>
<h3 id="implementation">Implementation</h3>
<p>First, we add the <code class="sourceCode haskell"><span class="dt">YieldStmt</span></code> and <code class="sourceCode haskell"><span class="dt">SpawnStmt</span></code> constructors to the <code class="sourceCode haskell"><span class="dt">Stmt</span></code> data type:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="9-10" data-deemphasize="11-11"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Stmt</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">ExprStmt</span> <span class="dt">Expr</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">VarStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">AssignStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">IfStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">WhileStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">FunctionStmt</span> <span class="dt">Identifier</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>]</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">ReturnStmt</span> (<span class="dt">Maybe</span> <span class="dt">Expr</span>)</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op">|</span> <span class="dt">YieldStmt</span></span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op">|</span> <span class="dt">SpawnStmt</span> <span class="dt">Expr</span></span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op">|</span> <span class="dt">SendStmt</span> <span class="dt">Expr</span> <span class="dt">Expr</span></span></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Eq</span>)</span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Program</span> <span class="ot">=</span> [<span class="dt">Stmt</span>]</span></code></pre></div>
<p>Then, we enhance the <code>stmt</code> parser to parse these statements:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="6-7" data-deemphasize="14-14"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">stmt ::</span> <span class="dt">Parser</span> <span class="dt">Stmt</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>stmt <span class="ot">=</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">IfStmt</span> <span class="op"><$></span> (reserved <span class="st">"if"</span> <span class="op">*></span> parens expr) <span class="op"><*></span> braces (many stmt)</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">WhileStmt</span> <span class="op"><$></span> (reserved <span class="st">"while"</span> <span class="op">*></span> parens expr) <span class="op"><*></span> braces (many stmt)</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">VarStmt</span> <span class="op"><$></span> (reserved <span class="st">"var"</span> <span class="op">*></span> identifier) <span class="op"><*></span> (symbol <span class="st">"="</span> <span class="op">*></span> expr <span class="op"><*</span> semi)</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op"><|></span> <span class="dt">YieldStmt</span> <span class="op"><$</span> (reserved <span class="st">"yield"</span> <span class="op"><*</span> semi)</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op"><|></span> <span class="dt">SpawnStmt</span> <span class="op"><$></span> (reserved <span class="st">"spawn"</span> <span class="op">*></span> expr <span class="op"><*</span> semi)</span></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">ReturnStmt</span> <span class="op"><$></span> (reserved <span class="st">"return"</span> <span class="op">*></span> optional expr <span class="op"><*</span> semi)</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">FunctionStmt</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> <span class="op"><$></span> try (reserved <span class="st">"function"</span> <span class="op">*></span> identifier)</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="op"><*></span> parens (sepBy identifier <span class="op">$</span> symbol <span class="st">","</span>)</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> <span class="op"><*></span> braces (many stmt)</span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> try (<span class="dt">AssignStmt</span> <span class="op"><$></span> identifier <span class="op"><*></span> (symbol <span class="st">"="</span> <span class="op">*></span> expr <span class="op"><*</span> semi))</span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op"><|></span> try (<span class="dt">SendStmt</span> <span class="op"><$></span> expr <span class="op"><*></span> (symbol <span class="st">"->"</span> <span class="op">*></span> expr <span class="op"><*</span> semi))</span></span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">ExprStmt</span> <span class="op"><$></span> expr <span class="op"><*</span> semi</span></code></pre></div>
<p>Next, we implement the <code>execute</code> function for the <code class="sourceCode haskell"><span class="dt">YieldStmt</span></code> and <code class="sourceCode haskell"><span class="dt">SpawnStmt</span></code> statements:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="21-22" data-deemphasize="23-27"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">execute ::</span> <span class="dt">Stmt</span> <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>execute <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">ExprStmt</span> expr <span class="ot">-></span> void <span class="op">$</span> evaluate expr</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">VarStmt</span> name expr <span class="ot">-></span> evaluate expr <span class="op">>>=</span> defineVar name</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">AssignStmt</span> name expr <span class="ot">-></span> evaluate expr <span class="op">>>=</span> assignVar name</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">IfStmt</span> expr body <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> cond <span class="ot"><-</span> evaluate expr</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> when (isTruthy cond) <span class="op">$</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> traverse_ execute body</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> while<span class="op">@</span>(<span class="dt">WhileStmt</span> expr body) <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> cond <span class="ot"><-</span> evaluate expr</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> when (isTruthy cond) <span class="op">$</span> <span class="kw">do</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> traverse_ execute body</span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a> execute while</span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">ReturnStmt</span> mExpr <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a> mRet <span class="ot"><-</span> <span class="fu">traverse</span> evaluate mExpr</span>
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a> throwError <span class="op">.</span> <span class="dt">Return</span> <span class="op">.</span> fromMaybe <span class="dt">Null</span> <span class="op">$</span> mRet</span>
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">FunctionStmt</span> name params body <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a> env <span class="ot"><-</span> State.gets isEnv</span>
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a> defineVar name <span class="op">$</span> <span class="dt">Function</span> name params body env</span>
<span id="cb1-21"><a href="#cb1-21" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">YieldStmt</span> <span class="ot">-></span> yield</span></span>
<span id="cb1-22"><a href="#cb1-22" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">SpawnStmt</span> expr <span class="ot">-></span> spawn expr</span></span>
<span id="cb1-23"><a href="#cb1-23" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">SendStmt</span> expr chan <span class="ot">-></span> evaluate chan <span class="op">>>=</span> \<span class="kw">case</span></span></span>
<span id="cb1-24"><a href="#cb1-24" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">Chan</span> channel <span class="ot">-></span> <span class="kw">do</span></span></span>
<span id="cb1-25"><a href="#cb1-25" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> val <span class="ot"><-</span> evaluate expr</span></span>
<span id="cb1-26"><a href="#cb1-26" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> channelSend val channel</span></span>
<span id="cb1-27"><a href="#cb1-27" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> v <span class="ot">-></span> throw <span class="op">$</span> <span class="st">"Cannot send to a non-channel: "</span> <span class="op"><></span> <span class="fu">show</span> v</span></span>
<span id="cb1-28"><a href="#cb1-28" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb1-29"><a href="#cb1-29" aria-hidden="true" tabindex="-1"></a> isTruthy <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb1-30"><a href="#cb1-30" aria-hidden="true" tabindex="-1"></a> <span class="dt">Null</span> <span class="ot">-></span> <span class="dt">False</span></span>
<span id="cb1-31"><a href="#cb1-31" aria-hidden="true" tabindex="-1"></a> <span class="dt">Boolean</span> b <span class="ot">-></span> b</span>
<span id="cb1-32"><a href="#cb1-32" aria-hidden="true" tabindex="-1"></a> _ <span class="ot">-></span> <span class="dt">True</span></span></code></pre></div>
<p>All the scaffolding is now in place. Next, we implement the <code class="sourceCode javascript"><span class="kw">yield</span></code> and <code class="sourceCode"><span class="cf">spawn</span></code> functions. First comes <code class="sourceCode"><span class="cf">spawn</span></code>:</p>
<div class="sourceCode" id="cb16" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb16-1"><a href="#cb16-1" aria-hidden="true" tabindex="-1"></a><span class="ot">spawn ::</span> <span class="dt">Expr</span> <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb16-2"><a href="#cb16-2" aria-hidden="true" tabindex="-1"></a>spawn expr <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb16-3"><a href="#cb16-3" aria-hidden="true" tabindex="-1"></a> env <span class="ot"><-</span> State.gets isEnv</span>
<span id="cb16-4"><a href="#cb16-4" aria-hidden="true" tabindex="-1"></a> coroutine <span class="ot"><-</span> newCoroutine env (<span class="fu">const</span> <span class="op">$</span> evaluate expr <span class="op">>></span> runNextCoroutine)</span>
<span id="cb16-5"><a href="#cb16-5" aria-hidden="true" tabindex="-1"></a> scheduleCoroutine coroutine</span></code></pre></div>
<p>The <code class="sourceCode"><span class="cf">spawn</span></code> statement creates a new coroutine for the expression <code class="sourceCode haskell">expr</code> and schedules it. The coroutine captures the current environment, and evaluates the expression <code class="sourceCode haskell">expr</code> when it is run. The <code>runNextCoroutine</code> function is called after the expression is evaluated to run the next coroutine in the queue<a href="#fn11" class="footnote-ref" id="fnref11" role="doc-noteref"><sup>11</sup></a>.</p>
<p>Next up is <code class="sourceCode javascript"><span class="kw">yield</span></code>:</p>
<div class="sourceCode" id="cb17" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a><span class="ot">yield ::</span> <span class="dt">Interpreter</span> ()</span>
<span id="cb17-2"><a href="#cb17-2" aria-hidden="true" tabindex="-1"></a>yield <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb17-3"><a href="#cb17-3" aria-hidden="true" tabindex="-1"></a> env <span class="ot"><-</span> State.gets isEnv</span>
<span id="cb17-4"><a href="#cb17-4" aria-hidden="true" tabindex="-1"></a> callCC <span class="op">$</span> \cont <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb17-5"><a href="#cb17-5" aria-hidden="true" tabindex="-1"></a> newCoroutine env cont <span class="op">>>=</span> scheduleCoroutine</span>
<span id="cb17-6"><a href="#cb17-6" aria-hidden="true" tabindex="-1"></a> runNextCoroutine</span></code></pre></div>
<p>The <code class="sourceCode javascript"><span class="kw">yield</span></code> function is the essence of coroutines in <span class="fancy">Co</span>. This is where we use the continuations that we added to the interpreter. First, we capture the current environment from the interpreter state. Then, we invoke <code class="sourceCode haskell">callCC</code> to get the current continuation. This continuation represents the rest of the program execution that lies in future after the <code class="sourceCode javascript"><span class="kw">yield</span></code> statement<a href="#fn12" class="footnote-ref" id="fnref12" role="doc-noteref"><sup>12</sup></a>. We create a new coroutine with the captured environment and the continuation, and schedule it. Finally, we run the next coroutine in the queue.</p>
<p>By capturing the environment and the continuation in a coroutine, and scheduling it to be run later, we are able to suspend the current program execution, and resume it later. At the same time, by running the next coroutine in the queue, we cause the interleaved execution of function calls that we saw in the previous section.</p>
<h2 data-track-content data-content-name="waiting-for-termination" data-content-piece="implementing-co-3" id="waiting-for-termination">Waiting for Termination</h2>
<p>There is one last thing we need to implement. If we were to run the following program with the interpreter as we have it now, it would terminate prematurely without printing anything:</p>
<div class="sourceCode" id="cb18" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb18-1"><a href="#cb18-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">printNums</span>(start<span class="op">,</span> end) {</span>
<span id="cb18-2"><a href="#cb18-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> start<span class="op">;</span></span>
<span id="cb18-3"><a href="#cb18-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> end <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb18-4"><a href="#cb18-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(i)<span class="op">;</span></span>
<span id="cb18-5"><a href="#cb18-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">yield</span><span class="op">;</span></span>
<span id="cb18-6"><a href="#cb18-6" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb18-7"><a href="#cb18-7" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb18-8"><a href="#cb18-8" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb18-9"><a href="#cb18-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb18-10"><a href="#cb18-10" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)<span class="op">;</span></span></code></pre></div>
<p>That’s because <code class="sourceCode"><span class="cf">spawn</span></code> schedules a coroutine for the <code class="sourceCode javascript"><span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)</code> function call, but the interpreter does not wait for all scheduled coroutines to finish executing before terminating. So, we add a mechanism for the same:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="3:3-3:4,3:14-3:30,4-4,5:32-5:43,6-6,7:5-7:9"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">awaitTermination ::</span> <span class="dt">Interpreter</span> ()</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>awaitTermination <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="deemphasis">(</span>coroutines<span class="deemphasis">, maxWakeupTime)</span> <span class="ot"><-</span> readIORef <span class="op">=<<</span> State.gets isCoroutines</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> dur <span class="ot"><-</span> calcSleepDuration maxWakeupTime</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> unless (PQ.null coroutines) <span class="op">$</span><span class="deemphasis"> <span class="kw">if</span> dur <span class="op">></span> <span class="dv">0</span></span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="kw">then</span> sleep dur <span class="op">>></span> awaitTermination</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="kw"><span class="deemphasis">else</span></span> yield <span class="op">>></span> awaitTermination</span></code></pre></div>
<p>The <code>awaitTermination</code> function checks if the coroutine queue is empty. If it is not, it yields and calls itself again to redo the check. Calling <code class="sourceCode javascript"><span class="kw">yield</span></code> causes the next coroutine in the queue to be run. <code>awaitTermination</code> keeps checking the queue, and yielding until the queue is empty. Then, it finally returns.</p>
<h2 data-track-content data-content-name="putting-everything-together" data-content-piece="implementing-co-3" id="putting-everything-together">Putting Everything Together</h2>
<p>Finally, we put everything together in the <code>interpret</code> function:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="5-5,8:34-8:53,12-12"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">interpret ::</span> <span class="dt">Program</span> <span class="ot">-></span> <span class="dt">IO</span> (<span class="dt">Either</span> <span class="dt">String</span> ())</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>interpret program <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> state <span class="ot"><-</span> initInterpreterState</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> retVal <span class="ot"><-</span> <span class="fu">flip</span> evalStateT state</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op">.</span> <span class="fu">flip</span> runContT <span class="fu">return</span></span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> runExceptT</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> runInterpreter</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="op">$</span> (traverse_ execute program <span class="emphasis"><span class="op">>></span> awaitTermination</span>)</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="kw">case</span> retVal <span class="kw">of</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">Left</span> (<span class="dt">RuntimeError</span> err) <span class="ot">-></span> <span class="fu">return</span> <span class="op">$</span> <span class="dt">Left</span> err</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">Left</span> (<span class="dt">Return</span> _) <span class="ot">-></span> <span class="fu">return</span> <span class="op">$</span> <span class="dt">Left</span> <span class="st">"Cannot return from outside functions"</span></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="dt">Left</span> <span class="dt">CoroutineQueueEmpty</span> <span class="ot">-></span> <span class="fu">return</span> <span class="op">$</span> <span class="dt">Right</span> ()</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">Right</span> _ <span class="ot">-></span> <span class="fu">return</span> <span class="op">$</span> <span class="dt">Right</span> ()</span></code></pre></div>
<p>We add <code>awaitTermination</code> at the end of the program to be interpreted so that the interpreter waits for the coroutine queue to be empty before terminating.</p>
<p>We use <code>runContT</code> to run the program with the initial continuation, which is just <code class="sourceCode haskell"><span class="fu">return</span></code>. This causes the interpreter to terminate when the program returns.</p>
<p>Lastly, we catch the <code class="sourceCode haskell"><span class="dt">CoroutineQueueEmpty</span></code> exception, ignore it, and terminate the interpreter.</p>
<p>That’s it! We have implemented coroutines in <span class="fancy">Co</span>. For an interesting example of usage of coroutines, we are going to implement the breadth-first traversal of a binary tree without using a queue in the next section.</p>
<h2 data-track-content data-content-name="bonus-round-breadth-first-traversal-without-a-queue" data-content-piece="implementing-co-3" id="bonus-round-breadth-first-traversal-without-a-queue">Bonus Round: Breadth-First Traversal without a Queue</h2>
<p><a href="https://en.wikipedia.org/wiki/Breadth-first_traversal" target="_blank" rel="noopener">Breadth-first traversal</a> is a common algorithm for traversing a tree. It traverses the tree level-by-level, from left to right. It uses a queue to keep track of the nodes that are yet to be traversed. However, with coroutines, we can implement a breadth-first traversal without using a queue.</p>
<p>First, we need to define a binary tree data structure in <span class="fancy">Co</span>. Remember, however, that <span class="fancy">Co</span> does not have a built-in data structure for trees, neither does it support user-defined data structures. So, we are going to borrow <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-14.html#%_sec_2.1.3" target="_blank" rel="noopener">a trick from the Wizard book</a>, and implement it using closures:</p>
<div class="sourceCode" id="cb19" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb19-1"><a href="#cb19-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">BinaryTree</span>(val<span class="op">,</span> left<span class="op">,</span> right) {</span>
<span id="cb19-2"><a href="#cb19-2" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="kw">function</span> (command) {</span>
<span id="cb19-3"><a href="#cb19-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (command <span class="op">==</span> <span class="st">"val"</span>) { <span class="cf">return</span> val<span class="op">;</span> }</span>
<span id="cb19-4"><a href="#cb19-4" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (command <span class="op">==</span> <span class="st">"left"</span>) { <span class="cf">return</span> left<span class="op">;</span> }</span>
<span id="cb19-5"><a href="#cb19-5" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (command <span class="op">==</span> <span class="st">"right"</span>) { <span class="cf">return</span> right<span class="op">;</span> }</span>
<span id="cb19-6"><a href="#cb19-6" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb19-7"><a href="#cb19-7" aria-hidden="true" tabindex="-1"></a> }<span class="op">;</span></span>
<span id="cb19-8"><a href="#cb19-8" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb19-9"><a href="#cb19-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb19-10"><a href="#cb19-10" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">binaryTreeVal</span>(tree) { <span class="cf">return</span> <span class="fu">tree</span>(<span class="st">"val"</span>)<span class="op">;</span> }</span>
<span id="cb19-11"><a href="#cb19-11" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">binaryTreeLeft</span>(tree) { <span class="cf">return</span> <span class="fu">tree</span>(<span class="st">"left"</span>)<span class="op">;</span> }</span>
<span id="cb19-12"><a href="#cb19-12" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">binaryTreeRight</span>(tree) { <span class="cf">return</span> <span class="fu">tree</span>(<span class="st">"right"</span>)<span class="op">;</span> }</span></code></pre></div>
<p>We define a binary tree as a function that takes a node value, and left and right subtrees as parameters, and returns an anonymous function that takes a command, and returns the corresponding parameter value. The <code class="sourceCode javascript">binaryTreeVal</code>, <code class="sourceCode javascript">binaryTreeLeft</code> and <code class="sourceCode javascript">binaryTreeRight</code> are helper functions that call the returned anonymous function with the appropriate command.</p>
<p>Next, we write a function to generate a <a href="https://en.wikipedia.org/wiki/Binary_tree#perfect" target="_blank" rel="noopener">perfect binary tree</a> given a starting power-of-two number:</p>
<div class="sourceCode" id="cb20" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb20-1"><a href="#cb20-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">generatePowersOfTwoBinaryTree</span>(start) {</span>
<span id="cb20-2"><a href="#cb20-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">function</span> <span class="fu">generateTree</span>(start<span class="op">,</span> interval) {</span>
<span id="cb20-3"><a href="#cb20-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (start <span class="op">==</span> <span class="dv">1</span>) {</span>
<span id="cb20-4"><a href="#cb20-4" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="fu">BinaryTree</span>(<span class="dv">1</span><span class="op">,</span> <span class="kw">null</span><span class="op">,</span> <span class="kw">null</span>)<span class="op">;</span></span>
<span id="cb20-5"><a href="#cb20-5" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb20-6"><a href="#cb20-6" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="fu">BinaryTree</span>(start<span class="op">,</span></span>
<span id="cb20-7"><a href="#cb20-7" aria-hidden="true" tabindex="-1"></a> <span class="fu">generateTree</span>(start <span class="op">-</span> interval<span class="op">/</span><span class="dv">2</span><span class="op">,</span> interval<span class="op">/</span><span class="dv">2</span>)<span class="op">,</span></span>
<span id="cb20-8"><a href="#cb20-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">generateTree</span>(start <span class="op">-</span> interval<span class="op">/</span><span class="dv">2</span><span class="op">,</span> interval<span class="op">/</span><span class="dv">2</span>))<span class="op">;</span></span>
<span id="cb20-9"><a href="#cb20-9" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb20-10"><a href="#cb20-10" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="fu">generateTree</span>(start<span class="op">,</span> start)<span class="op">;</span></span>
<span id="cb20-11"><a href="#cb20-11" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
<p>And, a function to pretty-print a tree node:</p>
<div class="sourceCode" id="cb21" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb21-1"><a href="#cb21-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">printTreeNode</span>(val<span class="op">,</span> depth) {</span>
<span id="cb21-2"><a href="#cb21-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb21-3"><a href="#cb21-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> padding <span class="op">=</span> <span class="st">"┃━"</span><span class="op">;</span></span>
<span id="cb21-4"><a href="#cb21-4" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> depth) {</span>
<span id="cb21-5"><a href="#cb21-5" aria-hidden="true" tabindex="-1"></a> padding <span class="op">=</span> padding <span class="op">+</span> <span class="st">"━━━━━━━━"</span><span class="op">;</span></span>
<span id="cb21-6"><a href="#cb21-6" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb21-7"><a href="#cb21-7" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb21-8"><a href="#cb21-8" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb21-9"><a href="#cb21-9" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(padding <span class="op">+</span> <span class="st">" "</span> <span class="op">+</span> val)<span class="op">;</span></span>
<span id="cb21-10"><a href="#cb21-10" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
<p>Finally, here’s the function that does the breadth-first traversal, and prints the tree:</p>
<div class="sourceCode" id="cb22" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb22-1"><a href="#cb22-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">printBinaryTreeBreadthFirst</span>(tree) {</span>
<span id="cb22-2"><a href="#cb22-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">function</span> <span class="fu">traverseTree</span>(tree<span class="op">,</span> depth) {</span>
<span id="cb22-3"><a href="#cb22-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (tree <span class="op">==</span> <span class="kw">null</span>) { <span class="cf">return</span><span class="op">;</span> }</span>
<span id="cb22-4"><a href="#cb22-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">printTreeNode</span>(<span class="fu">binaryTreeVal</span>(tree)<span class="op">,</span> depth)<span class="op">;</span></span>
<span id="cb22-5"><a href="#cb22-5" aria-hidden="true" tabindex="-1"></a> spawn <span class="fu">traverseTree</span>(<span class="fu">binaryTreeLeft</span>(tree)<span class="op">,</span> depth <span class="op">+</span> <span class="dv">1</span>)<span class="op">;</span></span>
<span id="cb22-6"><a href="#cb22-6" aria-hidden="true" tabindex="-1"></a> spawn <span class="fu">traverseTree</span>(<span class="fu">binaryTreeRight</span>(tree)<span class="op">,</span> depth <span class="op">+</span> <span class="dv">1</span>)<span class="op">;</span></span>
<span id="cb22-7"><a href="#cb22-7" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb22-8"><a href="#cb22-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">traverseTree</span>(tree<span class="op">,</span> <span class="dv">0</span>)<span class="op">;</span></span>
<span id="cb22-9"><a href="#cb22-9" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
<p>We run it like this:</p>
<div class="sourceCode" id="cb23" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb23-1"><a href="#cb23-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> tree <span class="op">=</span> <span class="fu">generatePowersOfTwoBinaryTree</span>(<span class="dv">16</span>)<span class="op">;</span></span>
<span id="cb23-2"><a href="#cb23-2" aria-hidden="true" tabindex="-1"></a><span class="fu">printBinaryTreeBreadthFirst</span>(tree)<span class="op">;</span></span></code></pre></div>
<details>
<summary>
And, we get the following output:
</summary>
<pre class="plain"><code>┃━ 16
┃━━━━━━━━━ 8
┃━━━━━━━━━ 8
┃━━━━━━━━━━━━━━━━━ 4
┃━━━━━━━━━━━━━━━━━ 4
┃━━━━━━━━━━━━━━━━━ 4
┃━━━━━━━━━━━━━━━━━ 4
┃━━━━━━━━━━━━━━━━━━━━━━━━━ 2
┃━━━━━━━━━━━━━━━━━━━━━━━━━ 2
┃━━━━━━━━━━━━━━━━━━━━━━━━━ 2
┃━━━━━━━━━━━━━━━━━━━━━━━━━ 2
┃━━━━━━━━━━━━━━━━━━━━━━━━━ 2
┃━━━━━━━━━━━━━━━━━━━━━━━━━ 2
┃━━━━━━━━━━━━━━━━━━━━━━━━━ 2
┃━━━━━━━━━━━━━━━━━━━━━━━━━ 2
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1
┃━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1</code></pre>
</details>
<p>The trick here is to use the coroutine queue itself for the breadth-first traversal. After printing a tree node, we spawn a coroutine for traversing each child node. The coroutines are scheduled in the order they are spawned, so the traversal is breadth-first, resulting in the above output.</p>
<hr></hr>
<p>In this post, we added support for coroutines to our <span class="fancy">Co</span> interpreter. We learned about the continuation-passing style, and used it to implement coroutines. In the <a href="https://abhinavsarkar.net/posts/implementing-co-4/?mtm_campaign=feed">next part</a>, we’ll add support for channels to our interpreter, and use them for cross-coroutine communication.</p>
<p>The code for complete <span class="fancy">Co</span> interpreter is available <a href="https://abhinavsarkar.net/code/co-interpreter.html?mtm_campaign=feed">here</a>.</p>
<h2 class="notoc" data-track-content data-content-name="acknowledgements" data-content-piece="implementing-co-3" id="acknowledgements">Acknowledgements</h2>
<p>Many thanks to <a href="https://arunraghavan.net/" target="_blank" rel="noopener">Arun Raghavan</a> for reviewing a draft of this article.</p>
<div id="refs" class="references csl-bib-body hanging-indent" role="list">
<div id="ref-Abelson1996-c32" class="csl-entry" role="listitem">
Abelson, Harold, Gerald Jay Sussman, and with Julie Sussman. <span>“The Environment Model of Evaluation.”</span> In <em>Structure and Interpretation of Computer Programs</em>, 2nd Editon. MIT Press/McGraw-Hill, 1996. <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-21.html#%_sec_3.2" target="_blank" rel="noopener">https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-21.html#%_sec_3.2</a>.
</div>
<div id="ref-Bartel2011-ap" class="csl-entry" role="listitem">
Bartel, Joe. <span>“<span>Non-Preemptive</span> Multitasking.”</span> <em>The Computer Journal</em>, no. 30 (May 2011): 37–38, 28. <a href="http://cini.classiccmp.org/pdf/HT68K/HT68K%20TCJ30p37.pdf" target="_blank" rel="noopener">http://cini.classiccmp.org/pdf/HT68K/HT68K%20TCJ30p37.pdf</a>.
</div>
<div id="ref-Knuth1997-rv" class="csl-entry" role="listitem">
Knuth, Donald E. <span>“Coroutines.”</span> In <em>The Art of Computer Programming: Volume 1: Fundamental Algorithms</em>, 3rd ed., 193–200. Addison Wesley, 1997.
</div>
<div id="ref-Reynolds1993-dc" class="csl-entry" role="listitem">
Reynolds, John C. <span>“The Discoveries of Continuations.”</span> <em>LISP and Symbolic Computation</em> 6, no. 3-4 (1993): 233–47. <a href="https://www.cs.ru.nl/~freek/courses/tt-2011/papers/cps/histcont.pdf" target="_blank" rel="noopener">https://www.cs.ru.nl/~freek/courses/tt-2011/papers/cps/histcont.pdf</a>.
</div>
</div>
<section id="footnotes" class="footnotes footnotes-end-of-document" role="doc-endnotes">
<hr></hr>
<ol>
<li id="fn1"><p>This representation is copied from a <a href="https://dmitrykandalov.com/coroutines-as-threads" target="_blank" rel="noopener">series</a> of articles on coroutines by Dmitry Kandalov. The articles are a great introduction to coroutines, and are highly recommended.<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn2"><p>Read the <a href="https://kotlinlang.org/docs/coroutines-guide.html" target="_blank" rel="noopener">Kotlin docs</a> and <a href="https://docs.python.org/3/library/asyncio-task.html" target="_blank" rel="noopener">Python docs</a> to learn more about coroutines in Kotlin and Python respectively.<a href="#fnref2" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn3"><p>Generators are similar to coroutines. The main difference is that generators are typically used to produce a sequence of values, while coroutines are used to implement concurrency. But coroutines (as we have them in this post) can be implemented over generators, and generators can be implemented over coroutines and channels. So the difference is mostly of intent.<a href="#fnref3" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn4"><p>Coroutines as we have them in <span class="fancy">Co</span>, are asymmetric, non-first-class, and stackful.</p>
<p>In contrast, coroutines in</p>
<ul>
<li>Kotlin are asymmetric, non-first-class and stackless,</li>
<li>Python are asymmetric, first-class and stackless,</li>
<li>Lua are asymmetric, first-class and stackful, and</li>
<li>Zig are symmetric, non-first-class and stackless.</li>
</ul>
<p>See the Wikipedia article on coroutines for more details on the <a href="https://en.wikipedia.org/wiki/Coroutine#Definition_and_Types" target="_blank" rel="noopener">types of coroutines</a> and their <a href="https://en.wikipedia.org/wiki/Coroutine#Implementations" target="_blank" rel="noopener">various implementations</a>.<a href="#fnref4" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn5"><p>The core.async library implements something like coroutines in Clojure, but they are not true coroutines. They have <a href="https://www.clojure.org/guides/core_async_go#_unsupported_constructs_and_other_limitations_in_go_blocks" target="_blank" rel="noopener">various limitations</a> like not being able to yield from a functions called from a <code>go</code> block. This is because core.async is implemented as a macro that transforms the code directly inside a <code>go</code> block into a state machine, but not the functions called from the <code>go</code> block.<a href="#fnref5" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn6"><p>See <a href="https://free.cofree.io/2020/01/02/cps/" target="_blank" rel="noopener">this article</a> by
Ziyang Liu and <a href="https://matt.might.net/articles/programming-with-continuations--exceptions-backtracking-search-threads-generators-coroutines/" target="_blank" rel="noopener">this one</a> by Matt Might for detailed explanations of the various use-cases of <abbr title="Continuation-passing style">CPS</abbr>.<a href="#fnref6" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn7"><p>See <a href="https://blog.poisson.chat/posts/2019-10-26-reasonable-continuations.html" target="_blank" rel="noopener">this article</a> by Li-yao XIA for an introduction to the <code class="sourceCode haskell"><span class="dt">Cont</span></code> monad.<a href="#fnref7" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn8"><p><a href="https://en.wikipedia.org/wiki/Scheme_(programming_language)" target="_blank" rel="noopener">Scheme</a> was the first language to introduce <code class="sourceCode scheme"><span class="kw">call/cc</span></code>. Since then <a href="https://en.wikipedia.org/wiki/Continuation#Programming_language_support" target="_blank" rel="noopener">many languages</a> have added support for it.<a href="#fnref8" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn9"><p>If we compare the <a href="#cb9-1">CPS version</a> of the program with the <a href="#cb8-1">direct style version</a>, we can see that it is possible to print the recommendations twice in the <abbr title="Continuation-passing style">CPS</abbr> version by calling the continuation twice. However, this is not possible in the direct style version, since the flow of control is implicit in it.<a href="#fnref9" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn10"><p>We use the min-priority queue from the <a href="https://hackage.haskell.org/package/pqueue" target="_blank" rel="noopener">pqueue</a> library.<a href="#fnref10" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn11"><p>It is essential to call <code>runNextCoroutine</code> after the expression in the <code class="sourceCode"><span class="cf">spawn</span></code> statement is evaluated. The evaluation of the expression may or may not yield. If it does, yielding causes the next coroutine to be <a href="#cb17-1">run</a>.</p>
<p>However, if it does not yield, but instead returns, and we do not call <code>runNextCoroutine</code> after it, the flow of control then goes to the end of the previous call to <code>runNextCoroutine</code> called from a previous yield. This causes the program after the previous yield to start executing, but with the interpreter environment set to that of the expression in the <code class="sourceCode"><span class="cf">spawn</span></code> statement, leading to unexpected behavior.</p>
<p>So, calling <code>runNextCoroutine</code> after the expression evaluation is a must to ensure correct execution.<a href="#fnref11" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn12"><p>The coroutines in <span class="fancy">Co</span> are stackful, which means that the <abbr title="Thread of computation">ToC</abbr> can be yielded from anywhere in the program, including nested function calls, and are resumed from the same point. This is in contrast to stackless coroutine implementations, where the <abbr title="Thread of computation">ToC</abbr> can only be yielded from particular functions that are marked as being able to yield, like generators in Python or <code>async</code> functions in JavaScript. Stackless coroutines are more efficient, but they are also more restrictive.<a href="#fnref12" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
</ol>
</section><p>If you liked this post, please <a href="https://abhinavsarkar.net/posts/implementing-co-3/?mtm_campaign=feed#syndications">leave a comment</a>.</p><img referrerpolicy="no-referrer-when-downgrade" src="https://anna.abhinavsarkar.net/matomo.php?idsite=1&rec=1" style="border:0" alt="" />2023-02-11T00:00:00Z<p>In the <a href="https://abhinavsarkar.net/posts/implementing-co-2/">previous post</a>, we wrote the interpreter for basic features of <span class="fancy">Co</span>, the small language we are building in this series of posts. In this post, we explore and implement what makes <span class="fancy">Co</span> really interesting: support for lightweight concurrency using Coroutines.</p>
https://abhinavsarkar.net/posts/static-site-generator-using-shake/Writing a Static Site Generator Using Shake2022-12-17T00:00:00ZAbhinav Sarkarhttps://abhinavsarkar.net/about/abhinav@abhinavsarkar.net<p>Static site generators (SSGs) are all rage these days as people realize that plain HTML websites are good enough for most cases. <abbr title="Static site generator">SSGs</abbr> take raw data in various formats—often <a href="https://en.wikipedia.org/wiki/Markdown" target="_blank" rel="noopener">Markdown</a>, <a href="https://en.wikipedia.org/wiki/JSON" target="_blank" rel="noopener">JSON</a>, and <a href="https://en.wikipedia.org/wiki/YAML" target="_blank" rel="noopener">YAML</a>—and process them to produce the static websites, which can then be hosted easily on any hosting provider, or on personal <abbr title="Virtual private server">VPSes</abbr>. In this post, we write a bespoke <abbr title="Static site generator">SSG</abbr> using the <a href="https://shakebuild.com/" target="_blank" rel="noopener">Shake</a> build system.</p>
<p>This post was originally published on <a href="https://abhinavsarkar.net/posts/static-site-generator-using-shake/?mtm_campaign=feed">abhinavsarkar.net</a>.</p><!--more-->
<nav id="toc" class="right-toc"><h3>Contents</h3><ol><li><a href="#introduction">Introduction</a></li><li><a href="#build-systems">Build Systems</a></li><li><a href="#static-site-structure">Static Site Structure</a></li><li><a href="#main">Main</a></li><li><a href="#build-targets">Build Targets</a></li><li><a href="#build-rules">Build Rules</a></li><li><a href="#utilities">Utilities</a></li><li><a href="#building-the-blog">Building the Blog</a></li><li><a href="#shake-features">Shake Features</a></li><li><a href="#tips-and-tricks">Tips and Tricks</a></li><li><a href="#conclusion">Conclusion</a></li></ol></nav>
<h2 data-track-content data-content-name="introduction" data-content-piece="static-site-generator-using-shake" id="introduction">Introduction</h2>
<p>In the <a href="http://info.cern.ch/" target="_blank" rel="noopener">beginning</a>, people coded websites by hand, painstakingly writing the HTML tags and CSS styles (and JavaScript code, if they were into <a href="https://en.wikipedia.org/wiki/DHTML" target="_blank" rel="noopener">DHTML</a>). Many <a href="https://en.wikipedia.org/wiki/Weblog" target="_blank" rel="noopener">weblogs</a> were crafted by the hands of passionate individuals, even before the word <em>Blog</em> came into being.</p>
<p>Over time, these websites grew in size and some clever people decided to separate the data for the websites from the presentation and layout. The data moved into databases and <a href="https://en.wikipedia.org/wiki/Common_Gateway_Interface" target="_blank" rel="noopener">CGI</a> scripts were invented to pull the data and create webpages out of them programmatically, on request. Thus began the age of <a href="https://en.wikipedia.org/wiki/Content_management_systems" target="_blank" rel="noopener">Content management systems</a> (CMS) like <a href="https://en.wikipedia.org/wiki/Drupal" target="_blank" rel="noopener">Drupal</a>, and of course, blogging software like <a href="https://en.wikipedia.org/wiki/Wordpress" target="_blank" rel="noopener">Wordpress</a> and <a href="https://en.wikipedia.org/wiki/Blogspot" target="_blank" rel="noopener">Blogspot</a>.</p>
<p>Things eventually came full circle, as people realized that they don’t need the bloated and complicated mess of <abbr title="Content management system">CMSes</abbr> and blogging software, but at the same time appreciated the separation of presentation and data. Thus <a href="https://en.wikipedia.org/wiki/Static_site_generator" target="_blank" rel="noopener">Static site generators</a> were born<a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a>.</p>
<p><abbr title="Static site generator">SSGs</abbr> allow users to write blog articles and pages as plain data in various simple formats like <a href="https://en.wikipedia.org/wiki/Markdown" target="_blank" rel="noopener">Markdown</a> or <a href="https://en.wikipedia.org/wiki/reStructuredText" target="_blank" rel="noopener">reStructuredText</a>, and configurations in <a href="https://en.wikipedia.org/wiki/YAML" target="_blank" rel="noopener">YAML</a>, <a href="https://en.wikipedia.org/wiki/JSON" target="_blank" rel="noopener">JSON</a> or <a href="https://en.wikipedia.org/wiki/TOML" target="_blank" rel="noopener">TOML</a>, and process them to produce static websites in HTML/CSS/JS.</p>
<p>Most <abbr title="Static site generator">SSGs</abbr> allow the user to operate in a default mode where you can follow the conventions of the <abbr title="Static site generator">SSG</abbr>—like writing the blog articles in certain formats, and putting them in certain directories—and the <abbr title="Static site generator">SSG</abbr> takes care of everything else. The user does not need to know any internals.</p>
<p>At the same time, most <abbr title="Static site generator">SSGs</abbr> allow users to customize the output website by creating custom templates, and custom URLs. However, all <abbr title="Static site generator">SSGs</abbr> limit what users can do with them. If you need to do something that goes against the grain of your <abbr title="Static site generator">SSG</abbr>, you are stuck.</p>
<h2 data-track-content data-content-name="build-systems" data-content-piece="static-site-generator-using-shake" id="build-systems">Build Systems</h2>
<p><abbr title="Static site generator">SSGs</abbr> are used to create websites by transforming a set of input files (templates, content, and assets) into a set of output files (HTML, CSS, and JavaScript files). In this sense, <abbr title="Static site generator">SSGs</abbr> can be seen as a type of build system, as they automate the process of building a website by following a set of rules and dependencies.</p>
<p>A build system is a tool for automating the process of building complex projects. Build systems are commonly used in software development to ensure that the correct sequence of steps is followed in order to produce a working version of the software. This typically involves compiling source code, linking libraries, and running tests to ensure that the software is correct. However, build systems can also be used for projects in other domains where a set of input files need to be transformed into a set of output files according to some rules.</p>
<p><a href="https://shakebuild.com/" target="_blank" rel="noopener">Shake</a> is a build system written in the <a href="https://haskell.org" target="_blank" rel="noopener">Haskell</a>. It is flexible and powerful enough for managing the build process of complex software projects like <a href="https://www.haskell.org/ghc/" target="_blank" rel="noopener">GHC</a>, but at the same time, it is simple enough to be used to create an <abbr title="Static site generator">SSG</abbr><a href="#fn2" class="footnote-ref" id="fnref2" role="doc-noteref"><sup>2</sup></a>.</p>
<h3 id="shake">Shake</h3>
<p>In Shake, build targets represent the files or outputs that need to be produced as part of the build process. These could be executable binaries, library files, or any other type of output that is required to complete the build. Build targets are declared in a build script, along with information about their dependencies. For example, if an executable binary depends on a particular library file, the build script would specify this dependency.</p>
<p>Once the build targets and their dependencies have been declared, Shake uses <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#t:Rules" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">Rules</span></code></a> to specify how those targets should be built. A rule typically consists of a pattern that matches one or more targets, along with a set of instructions—called build <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#t:Action" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">Action</span></code></a>s by Shake—for building them. For example, a rule might specify that a certain type of source code file should be compiled using a particular compiler, with a certain set of flags. When Shake encounters a target that matches the pattern in a rule, it executes the instructions in the rule to build it.</p>
<p>By declaring dependencies between targets and defining rules to build them , Shake is able to figure out the correct order in which to build the targets <a href="#fn3" class="footnote-ref" id="fnref3" role="doc-noteref"><sup>3</sup></a>. Shake also provides a number of features to help users customize and optimize their build process, such as support for parallel builds, on-demand rebuilding, and caching of intermediate results.</p>
<p>In this post, we use Shake to build an <abbr title="Static site generator">SSG</abbr> by defining the build targets and rules for building the website. In addition, we use <a href="https://pandoc.org/" target="_blank" rel="noopener">Pandoc</a> to render Markdown content into HTML, and <a href="https://hackage.haskell.org/package/mustache" target="_blank" rel="noopener">Mustache</a> to render HTML templates.</p>
<h2 data-track-content data-content-name="static-site-structure" data-content-piece="static-site-generator-using-shake" id="static-site-structure">Static Site Structure</h2>
<p>The source of our website is arranged like this:</p>
<pre class="plain ascii-art"><code>shake-blog
├── Site.hs
├── about.md
├── contact.md
├── css
│ └── default.css
├── images
│ └── logo.png
├── posts
│ ├── 2022-08-12-welcome.md
│ ├── 2022-10-07-hello-world.md
└── templates
├── archive.html
├── default.html
├── home.html
├── post-list.html
└── post.html</code></pre>
<p><code>Site.hs</code> contains the Haskell code that we are going to write in this post. <code>about.md</code> and <code>contact.md</code> are two static pages. The <code>css</code> and <code>images</code> directories contain assets for the website. The <code>posts</code> directory contains blog posts, names of which start with the post publication dates in <code>YYYY-mm-dd</code> format. Finally, the <code>templates</code> directory contains the Mustache templates for the website.</p>
<p>The blog posts start with YAML metadata sections that contain the title of the post, name of the author (optional) and a list of tags for the post. For example:</p>
<div id="lst:welcome.md" class="listing numberSource markdown">
<div class="sourceCode" id="cb2" data-lang="markdown"><pre class="sourceCode numberSource markdown"><code class="sourceCode markdown"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="co">---</span></span>
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a><span class="an">title:</span><span class="co"> Welcome to my blog</span></span>
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a><span class="an">author:</span><span class="co"> Abhinav Sarkar</span></span>
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a><span class="an">tags:</span></span>
<span id="cb2-5"><a href="#cb2-5" aria-hidden="true" tabindex="-1"></a><span class="co"> - brag</span></span>
<span id="cb2-6"><a href="#cb2-6" aria-hidden="true" tabindex="-1"></a><span class="co"> - note</span></span>
<span id="cb2-7"><a href="#cb2-7" aria-hidden="true" tabindex="-1"></a><span class="co">---</span></span>
<span id="cb2-8"><a href="#cb2-8" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb2-9"><a href="#cb2-9" aria-hidden="true" tabindex="-1"></a>Welcome to my new blog. I wrote the blog generator myself.</span></code></pre></div>
<p><code>posts/2022-08-12-welcome.md</code></p>
</div>
<p>Pages are written in a similar fashion, but have only title in their YAML metadata.</p>
<p>After processing the input above, our <abbr title="Static site generator">SSG</abbr> produces the following file structure:</p>
<pre class="plain ascii-art"><code>_site/
├── about
│ └── index.html
├── archive
│ └── index.html
├── contact
│ └── index.html
├── css
│ └── default.css
├── images
│ └── logo.png
├── index.html
├── posts
│ ├── 2022-08-12-welcome
│ │ └── index.html
│ ├── 2022-10-07-hello-world
│ │ └── index.html
└── tags
├── brag
│ └── index.html
├── note
│ └── index.html
└── programming
└── index.html</code></pre>
<p>The CSS and image assets are copied directly. One <code>index.html</code> file is generated for each page, post, and tag. Additionally, one file is generated for the archive of posts, and one for the home page.</p>
<p>With the input and output described, let’s get started with writing the generator.</p>
<h2 data-track-content data-content-name="main" data-content-piece="static-site-generator-using-shake" id="main">Main</h2>
<p>We are going to write the program in a top-down fashion, starting with the <code>main</code> function. First come the extensions and imports. Other than imports from Shake, Pandoc and Mustache libraries, we also import from <a href="https://hackage.haskell.org/package/aeson" target="_blank" rel="noopener">aeson</a>, <a href="https://hackage.haskell.org/package/text" target="_blank" rel="noopener">text</a>, <a href="https://hackage.haskell.org/package/time" target="_blank" rel="noopener">time</a> and <a href="https://hackage.haskell.org/package/unordered-containers" target="_blank" rel="noopener">unordered-containers</a> libraries<a href="#fn4" class="footnote-ref" id="fnref4" role="doc-noteref"><sup>4</sup></a>.</p>
<div class="sourceCode" id="cb7" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="ot">{-# LANGUAGE ApplicativeDo, DataKinds, DeriveGeneric #-}</span></span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a><span class="ot">{-# LANGUAGE DerivingVia, LambdaCase, TypeApplications #-}</span></span>
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a><span class="kw">module</span> <span class="dt">Main</span> <span class="kw">where</span></span>
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Monad</span> (forM, void)</span>
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Data.Aeson.Types</span> (<span class="dt">Result</span> (..))</span>
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Data.List</span> (nub, sortOn)</span>
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Data.Text</span> (<span class="dt">Text</span>)</span>
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Data.Time</span> (<span class="dt">UTCTime</span>, defaultTimeLocale, formatTime, parseTimeM)</span>
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Deriving.Aeson</span></span>
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Deriving.Aeson.Stock</span> (<span class="dt">PrefixedSnake</span>)</span>
<span id="cb7-13"><a href="#cb7-13" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Development.Shake</span> (<span class="dt">Action</span>, <span class="dt">Rules</span>, (%>), (|%>), (~>))</span>
<span id="cb7-14"><a href="#cb7-14" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Development.Shake.FilePath</span> ((<.>), (</>))</span>
<span id="cb7-15"><a href="#cb7-15" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Text.Pandoc</span> (<span class="dt">Block</span> (<span class="dt">Plain</span>), <span class="dt">Meta</span> (..), <span class="dt">MetaValue</span> (..), <span class="dt">Pandoc</span> (..))</span>
<span id="cb7-16"><a href="#cb7-16" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Data.Aeson.Types</span> <span class="kw">as</span> <span class="dt">A</span></span>
<span id="cb7-17"><a href="#cb7-17" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Data.HashMap.Strict</span> <span class="kw">as</span> <span class="dt">HM</span></span>
<span id="cb7-18"><a href="#cb7-18" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Data.Ord</span> <span class="kw">as</span> <span class="dt">Ord</span></span>
<span id="cb7-19"><a href="#cb7-19" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Data.Text</span> <span class="kw">as</span> <span class="dt">T</span></span>
<span id="cb7-20"><a href="#cb7-20" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Development.Shake</span> <span class="kw">as</span> <span class="dt">Shake</span></span>
<span id="cb7-21"><a href="#cb7-21" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Development.Shake.FilePath</span> <span class="kw">as</span> <span class="dt">Shake</span></span>
<span id="cb7-22"><a href="#cb7-22" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Text.Mustache</span> <span class="kw">as</span> <span class="dt">Mus</span></span>
<span id="cb7-23"><a href="#cb7-23" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Text.Mustache.Compile</span> <span class="kw">as</span> <span class="dt">Mus</span></span>
<span id="cb7-24"><a href="#cb7-24" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Text.Pandoc</span> <span class="kw">as</span> <span class="dt">Pandoc</span></span></code></pre></div>
<p>The <code>main</code> function sets up the top-level Shake build targets, and lets Shake invoke the right one depending on the arguments passed at runtime.</p>
<div class="sourceCode" id="cb8" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="ot">main ::</span> <span class="dt">IO</span> ()</span>
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a>main <span class="ot">=</span> Shake.shakeArgs Shake.shakeOptions <span class="op">$</span> <span class="kw">do</span></span>
<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a> Shake.withTargetDocs <span class="st">"Build the site"</span> <span class="op">$</span></span>
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a> <span class="st">"build"</span> <span class="op">~></span> buildTargets</span>
<span id="cb8-5"><a href="#cb8-5" aria-hidden="true" tabindex="-1"></a> Shake.withTargetDocs <span class="st">"Clean the built site"</span> <span class="op">$</span></span>
<span id="cb8-6"><a href="#cb8-6" aria-hidden="true" tabindex="-1"></a> <span class="st">"clean"</span> <span class="op">~></span> Shake.removeFilesAfter outputDir [<span class="st">"//*"</span>]</span>
<span id="cb8-7"><a href="#cb8-7" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb8-8"><a href="#cb8-8" aria-hidden="true" tabindex="-1"></a> Shake.withoutTargets buildRules</span>
<span id="cb8-9"><a href="#cb8-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb8-10"><a href="#cb8-10" aria-hidden="true" tabindex="-1"></a><span class="ot">outputDir ::</span> <span class="dt">String</span></span>
<span id="cb8-11"><a href="#cb8-11" aria-hidden="true" tabindex="-1"></a>outputDir <span class="ot">=</span> <span class="st">"_site"</span></span></code></pre></div>
<p>There are two top-level build targets:</p>
<ol type="1">
<li><code>build</code>: generates the website.</li>
<li><code>clean</code>: deletes the generated website.</li>
</ol>
<p><code>outputDir</code> is the subdirectory in which the website is generated. Building the <code>clean</code> target deletes all files inside <code>outputDir</code>. The <code>build</code> target runs the <code>buildTargets</code> action that sets up the build targets for generating the site. The <code>buildRules</code> are also included in the Shake setup.</p>
<h2 data-track-content data-content-name="build-targets" data-content-piece="static-site-generator-using-shake" id="build-targets">Build Targets</h2>
<p>The <code>buildTargets</code> function sets up the build targets for the files to be generated by Shake.</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="13-16"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">buildTargets ::</span> <span class="dt">Action</span> ()</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>buildTargets <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> assetPaths <span class="ot"><-</span> Shake.getDirectoryFiles <span class="st">""</span> assetGlobs</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">$</span> <span class="fu">map</span> (outputDir <span class="op"></></span>) assetPaths</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">$</span> <span class="fu">map</span> indexHtmlOutputPath pagePaths</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> postPaths <span class="ot"><-</span> Shake.getDirectoryFiles <span class="st">""</span> postGlobs</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">$</span> <span class="fu">map</span> indexHtmlOutputPath postPaths</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">$</span> <span class="fu">map</span> (outputDir <span class="op"></></span>) [<span class="st">"archive/index.html"</span>, <span class="st">"index.html"</span>]</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> posts <span class="ot"><-</span> forM postPaths readPost</span></span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> Shake.need</span></span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> [ outputDir <span class="op"></></span> <span class="st">"tags"</span> <span class="op"></></span> T.unpack tag <span class="op"></></span> <span class="st">"index.html"</span></span></span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op">|</span> post <span class="ot"><-</span> posts, tag <span class="ot"><-</span> postTags post ]</span></span></code></pre></div>
<p>The <a href="https://hackage.haskell.org/package/shake-0.19.6/docs/Development-Shake.html#v:need" target="_blank" rel="noopener"><code class="sourceCode haskell">Shake.need</code></a> function registers one or more targets with Shake.</p>
<p>For assets, we just want them to be copied to the <code>outputDir</code> at the same path.</p>
<p>Page and post target paths in the <code>outputDir</code> are stripped of their extensions and appended with <code>/index.html</code>. So a post sourced from <code>posts/example.md</code> ends up at <code><outputDir>/posts/example/index.html</code>.</p>
<p>We also register two composite targets for the post archive and the home page<a href="#fn5" class="footnote-ref" id="fnref5" role="doc-noteref"><sup>5</sup></a>.</p>
<p>The paths, globs and helper function are shown below:</p>
<div class="sourceCode" id="cb9" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="ot">assetGlobs ::</span> [<span class="dt">String</span>]</span>
<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a>assetGlobs <span class="ot">=</span> [<span class="st">"css/*.css"</span>, <span class="st">"images/*.png"</span>]</span>
<span id="cb9-3"><a href="#cb9-3" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb9-4"><a href="#cb9-4" aria-hidden="true" tabindex="-1"></a><span class="ot">pagePaths ::</span> [<span class="dt">String</span>]</span>
<span id="cb9-5"><a href="#cb9-5" aria-hidden="true" tabindex="-1"></a>pagePaths <span class="ot">=</span> [<span class="st">"about.md"</span>, <span class="st">"contact.md"</span>]</span>
<span id="cb9-6"><a href="#cb9-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb9-7"><a href="#cb9-7" aria-hidden="true" tabindex="-1"></a><span class="ot">postGlobs ::</span> [<span class="dt">String</span>]</span>
<span id="cb9-8"><a href="#cb9-8" aria-hidden="true" tabindex="-1"></a>postGlobs <span class="ot">=</span> [<span class="st">"posts/*.md"</span>]</span>
<span id="cb9-9"><a href="#cb9-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb9-10"><a href="#cb9-10" aria-hidden="true" tabindex="-1"></a><span class="ot">indexHtmlOutputPath ::</span> <span class="dt">FilePath</span> <span class="ot">-></span> <span class="dt">FilePath</span></span>
<span id="cb9-11"><a href="#cb9-11" aria-hidden="true" tabindex="-1"></a>indexHtmlOutputPath srcPath <span class="ot">=</span></span>
<span id="cb9-12"><a href="#cb9-12" aria-hidden="true" tabindex="-1"></a> outputDir <span class="op"></></span> Shake.dropExtension srcPath <span class="op"></></span> <span class="st">"index.html"</span></span></code></pre></div>
<p>Now Shake knows what we want it to build. But how does it know how to build them? That’s what the build rules are for.</p>
<h2 data-track-content data-content-name="build-rules" data-content-piece="static-site-generator-using-shake" id="build-rules">Build Rules</h2>
<p>We have one build rule function for each build target type:</p>
<div class="sourceCode" id="cb10" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="ot">buildRules ::</span> <span class="dt">Rules</span> ()</span>
<span id="cb10-2"><a href="#cb10-2" aria-hidden="true" tabindex="-1"></a>buildRules <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb10-3"><a href="#cb10-3" aria-hidden="true" tabindex="-1"></a> assets</span>
<span id="cb10-4"><a href="#cb10-4" aria-hidden="true" tabindex="-1"></a> pages</span>
<span id="cb10-5"><a href="#cb10-5" aria-hidden="true" tabindex="-1"></a> posts</span>
<span id="cb10-6"><a href="#cb10-6" aria-hidden="true" tabindex="-1"></a> archive</span>
<span id="cb10-7"><a href="#cb10-7" aria-hidden="true" tabindex="-1"></a> tags</span>
<span id="cb10-8"><a href="#cb10-8" aria-hidden="true" tabindex="-1"></a> home</span></code></pre></div>
<p>Let’s start with the simplest one, the build rule for assets.</p>
<h3 id="assets">Assets</h3>
<p>In Shake, the build rules are written with <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:-124--37--62-" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="op">|%></span></code></a> or <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:-37--62-" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="op">%></span></code></a> operators. The <code class="sourceCode haskell"><span class="op">|%></span></code> operator takes a list of output globs or paths, and a function from target path to build action. When <code class="sourceCode haskell">Shake.need</code> is called with a file that matches a target glob, the corresponding build action is called with the target path.</p>
<div class="sourceCode" id="cb11" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="ot">assets ::</span> <span class="dt">Rules</span> ()</span>
<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a>assets <span class="ot">=</span> <span class="fu">map</span> (outputDir <span class="op"></></span>) assetGlobs <span class="op">|%></span> \target <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb11-3"><a href="#cb11-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> src <span class="ot">=</span> Shake.dropDirectory1 target</span>
<span id="cb11-4"><a href="#cb11-4" aria-hidden="true" tabindex="-1"></a> Shake.copyFileChanged src target</span>
<span id="cb11-5"><a href="#cb11-5" aria-hidden="true" tabindex="-1"></a> Shake.putInfo <span class="op">$</span> <span class="st">"Copied "</span> <span class="op"><></span> target <span class="op"><></span> <span class="st">" from "</span> <span class="op"><></span> src</span></code></pre></div>
<p>In case of assets, we simply get the original source path by dropping the first directory from the target path (that is, <code>outputDir</code>), and copy the source file to the target path if the file has changed<a href="#fn6" class="footnote-ref" id="fnref6" role="doc-noteref"><sup>6</sup></a>.</p>
<h3 id="pages">Pages</h3>
<p>Building pages is a bit more interesting. First, we write a data type to represent a page:</p>
<div class="sourceCode" id="cb12" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Page</span> <span class="ot">=</span> <span class="dt">Page</span> {<span class="ot">pageTitle ::</span> <span class="dt">Text</span>,<span class="ot"> pageContent ::</span> <span class="dt">Text</span>}</span>
<span id="cb12-2"><a href="#cb12-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Generic</span>)</span>
<span id="cb12-3"><a href="#cb12-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">ToJSON</span>) via <span class="dt">PrefixedSnake</span> <span class="st">"page"</span> <span class="dt">Page</span></span></code></pre></div>
<p>A page has a title and some text content. We also make <code class="sourceCode haskell"><span class="dt">Page</span></code> data type JSON serializable so that it can be consumed by the Mustache library for filling templates.</p>
<p>Now, the code that builds pages:</p>
<div class="sourceCode" id="cb13" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="ot">pages ::</span> <span class="dt">Rules</span> ()</span>
<span id="cb13-2"><a href="#cb13-2" aria-hidden="true" tabindex="-1"></a>pages <span class="ot">=</span> <span class="fu">map</span> indexHtmlOutputPath pagePaths <span class="op">|%></span> \target <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb13-3"><a href="#cb13-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> src <span class="ot">=</span> indexHtmlSourcePath target</span>
<span id="cb13-4"><a href="#cb13-4" aria-hidden="true" tabindex="-1"></a> (meta, html) <span class="ot"><-</span> markdownToHtml src</span>
<span id="cb13-5"><a href="#cb13-5" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb13-6"><a href="#cb13-6" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> page <span class="ot">=</span> <span class="dt">Page</span> (meta <span class="op">HM.!</span> <span class="st">"title"</span>) html</span>
<span id="cb13-7"><a href="#cb13-7" aria-hidden="true" tabindex="-1"></a> applyTemplateAndWrite <span class="st">"default.html"</span> page target</span>
<span id="cb13-8"><a href="#cb13-8" aria-hidden="true" tabindex="-1"></a> Shake.putInfo <span class="op">$</span> <span class="st">"Built "</span> <span class="op"><></span> target <span class="op"><></span> <span class="st">" from "</span> <span class="op"><></span> src</span>
<span id="cb13-9"><a href="#cb13-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb13-10"><a href="#cb13-10" aria-hidden="true" tabindex="-1"></a><span class="ot">indexHtmlSourcePath ::</span> <span class="dt">FilePath</span> <span class="ot">-></span> <span class="dt">FilePath</span></span>
<span id="cb13-11"><a href="#cb13-11" aria-hidden="true" tabindex="-1"></a>indexHtmlSourcePath <span class="ot">=</span></span>
<span id="cb13-12"><a href="#cb13-12" aria-hidden="true" tabindex="-1"></a> Shake.dropDirectory1</span>
<span id="cb13-13"><a href="#cb13-13" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> (<span class="op"><.></span> <span class="st">"md"</span>)</span>
<span id="cb13-14"><a href="#cb13-14" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> Shake.dropTrailingPathSeparator</span>
<span id="cb13-15"><a href="#cb13-15" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> Shake.dropFileName</span></code></pre></div>
<p>We get the source path from the target path by passing it through the <code>indexHtmlSourcePath</code> function. We read and render the source file by calling the <code>markdownToHtml</code> function. It returns the page YAML metadata as a <a href="https://hackage.haskell.org/package/aeson/docs/Data-Aeson-Types.html#t:FromJSON" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">FromJSON</span></code></a>-able value (a <a href="https://hackage.haskell.org/package/unordered-containers/docs/Data-HashMap-Strict.html" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">HashMap</span></code></a> in this case), and the page HTML text.</p>
<p>Next, we apply the <code class="sourceCode haskell"><span class="dt">Page</span></code> data to the <code>default.html</code> template, and write it to the target path by calling the <code>applyTemplateAndWrite</code> function. This creates the HTML file for the page.</p>
<p>The <code>default.html</code> Mustache template can be seen below:</p>
<details>
<summary>
<code>templates/default.html</code>
</summary>
<div class="sourceCode" id="cb14" data-lang="mustache"><pre class="sourceCode numberSource mustache"><code class="sourceCode mustache"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="dt"><!DOCTYPE </span>html<span class="dt">></span></span>
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true" tabindex="-1"></a><span class="kw"><html</span><span class="ot"> lang</span><span class="op">=</span><span class="st">"en"</span><span class="kw">></span></span>
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a><span class="kw"><head></span></span>
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a> <span class="kw"><meta</span><span class="ot"> charset</span><span class="op">=</span><span class="st">"UTF-8"</span><span class="kw">></span></span>
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true" tabindex="-1"></a> <span class="kw"><meta</span><span class="ot"> name</span><span class="op">=</span><span class="st">"viewport"</span><span class="ot"> content</span><span class="op">=</span><span class="st">"width=device-width, initial-scale=1.0"</span><span class="kw">></span></span>
<span id="cb14-6"><a href="#cb14-6" aria-hidden="true" tabindex="-1"></a> <span class="kw"><meta</span><span class="ot"> http-equiv</span><span class="op">=</span><span class="st">"X-UA-Compatible"</span><span class="ot"> content</span><span class="op">=</span><span class="st">"ie=edge"</span><span class="kw">></span></span>
<span id="cb14-7"><a href="#cb14-7" aria-hidden="true" tabindex="-1"></a> <span class="kw"><title></span>My Shake Blog — <span class="sc">{{{</span>title<span class="sc">}}}</span><span class="kw"></title></span></span>
<span id="cb14-8"><a href="#cb14-8" aria-hidden="true" tabindex="-1"></a> <span class="kw"><link</span><span class="ot"> rel</span><span class="op">=</span><span class="st">"stylesheet"</span><span class="ot"> type</span><span class="op">=</span><span class="st">"text/css"</span><span class="ot"> href</span><span class="op">=</span><span class="st">"/css/default.css"</span> <span class="kw">/></span></span>
<span id="cb14-9"><a href="#cb14-9" aria-hidden="true" tabindex="-1"></a><span class="kw"></head></span></span>
<span id="cb14-10"><a href="#cb14-10" aria-hidden="true" tabindex="-1"></a><span class="kw"><body></span></span>
<span id="cb14-11"><a href="#cb14-11" aria-hidden="true" tabindex="-1"></a> <span class="kw"><div</span><span class="ot"> id</span><span class="op">=</span><span class="st">"header"</span><span class="kw">></span></span>
<span id="cb14-12"><a href="#cb14-12" aria-hidden="true" tabindex="-1"></a> <span class="kw"><div</span><span class="ot"> id</span><span class="op">=</span><span class="st">"logo"</span><span class="kw">></span></span>
<span id="cb14-13"><a href="#cb14-13" aria-hidden="true" tabindex="-1"></a> <span class="kw"><a</span><span class="ot"> href</span><span class="op">=</span><span class="st">"/"</span><span class="kw">></span>My Shake Blog<span class="kw"></a></span></span>
<span id="cb14-14"><a href="#cb14-14" aria-hidden="true" tabindex="-1"></a> <span class="kw"></div></span></span>
<span id="cb14-15"><a href="#cb14-15" aria-hidden="true" tabindex="-1"></a> <span class="kw"><div</span><span class="ot"> id</span><span class="op">=</span><span class="st">"navigation"</span><span class="kw">></span></span>
<span id="cb14-16"><a href="#cb14-16" aria-hidden="true" tabindex="-1"></a> <span class="kw"><a</span><span class="ot"> href</span><span class="op">=</span><span class="st">"/"</span><span class="kw">></span>Home<span class="kw"></a></span></span>
<span id="cb14-17"><a href="#cb14-17" aria-hidden="true" tabindex="-1"></a> <span class="kw"><a</span><span class="ot"> href</span><span class="op">=</span><span class="st">"/about/"</span><span class="kw">></span>About<span class="kw"></a></span></span>
<span id="cb14-18"><a href="#cb14-18" aria-hidden="true" tabindex="-1"></a> <span class="kw"><a</span><span class="ot"> href</span><span class="op">=</span><span class="st">"/contact/"</span><span class="kw">></span>Contact<span class="kw"></a></span></span>
<span id="cb14-19"><a href="#cb14-19" aria-hidden="true" tabindex="-1"></a> <span class="kw"><a</span><span class="ot"> href</span><span class="op">=</span><span class="st">"/archive/"</span><span class="kw">></span>Archive<span class="kw"></a></span></span>
<span id="cb14-20"><a href="#cb14-20" aria-hidden="true" tabindex="-1"></a> <span class="kw"></div></span></span>
<span id="cb14-21"><a href="#cb14-21" aria-hidden="true" tabindex="-1"></a> <span class="kw"></div></span></span>
<span id="cb14-22"><a href="#cb14-22" aria-hidden="true" tabindex="-1"></a> <span class="kw"><div</span><span class="ot"> id</span><span class="op">=</span><span class="st">"content"</span><span class="kw">></span></span>
<span id="cb14-23"><a href="#cb14-23" aria-hidden="true" tabindex="-1"></a> <span class="kw"><h1></span><span class="sc">{{{</span>title<span class="sc">}}}</span><span class="kw"></h1></span></span>
<span id="cb14-24"><a href="#cb14-24" aria-hidden="true" tabindex="-1"></a> <span class="sc">{{{</span>content<span class="sc">}}}</span></span>
<span id="cb14-25"><a href="#cb14-25" aria-hidden="true" tabindex="-1"></a> <span class="kw"></div></span></span>
<span id="cb14-26"><a href="#cb14-26" aria-hidden="true" tabindex="-1"></a> <span class="kw"><div</span><span class="ot"> id</span><span class="op">=</span><span class="st">"footer"</span><span class="kw">></span></span>
<span id="cb14-27"><a href="#cb14-27" aria-hidden="true" tabindex="-1"></a> Site proudly generated by <span class="kw"><a</span><span class="ot"> href</span><span class="op">=</span><span class="st">"https://shakebuild.com"</span><span class="kw">></span>Shake<span class="kw"></a></span></span>
<span id="cb14-28"><a href="#cb14-28" aria-hidden="true" tabindex="-1"></a> <span class="kw"></div></span></span>
<span id="cb14-29"><a href="#cb14-29" aria-hidden="true" tabindex="-1"></a><span class="kw"></body></span></span>
<span id="cb14-30"><a href="#cb14-30" aria-hidden="true" tabindex="-1"></a><span class="kw"></html></span></span></code></pre></div>
</details>
<h3 id="posts">Posts</h3>
<p>Building posts is similar to building pages. We have a data type for posts:</p>
<div class="sourceCode" id="cb15" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Post</span> <span class="ot">=</span> <span class="dt">Post</span></span>
<span id="cb15-2"><a href="#cb15-2" aria-hidden="true" tabindex="-1"></a> {<span class="ot"> postTitle ::</span> <span class="dt">Text</span>,</span>
<span id="cb15-3"><a href="#cb15-3" aria-hidden="true" tabindex="-1"></a><span class="ot"> postAuthor ::</span> <span class="dt">Maybe</span> <span class="dt">Text</span>,</span>
<span id="cb15-4"><a href="#cb15-4" aria-hidden="true" tabindex="-1"></a><span class="ot"> postTags ::</span> [<span class="dt">Text</span>],</span>
<span id="cb15-5"><a href="#cb15-5" aria-hidden="true" tabindex="-1"></a><span class="ot"> postDate ::</span> <span class="dt">Maybe</span> <span class="dt">Text</span>,</span>
<span id="cb15-6"><a href="#cb15-6" aria-hidden="true" tabindex="-1"></a><span class="ot"> postContent ::</span> <span class="dt">Maybe</span> <span class="dt">Text</span>,</span>
<span id="cb15-7"><a href="#cb15-7" aria-hidden="true" tabindex="-1"></a><span class="ot"> postLink ::</span> <span class="dt">Maybe</span> <span class="dt">Text</span></span>
<span id="cb15-8"><a href="#cb15-8" aria-hidden="true" tabindex="-1"></a> } <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Generic</span>)</span>
<span id="cb15-9"><a href="#cb15-9" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">FromJSON</span>, <span class="dt">ToJSON</span>) via <span class="dt">PrefixedSnake</span> <span class="st">"post"</span> <span class="dt">Post</span></span></code></pre></div>
<p>Other than the title and text content, a post also has a date, a list of tags, an optional author, and a permalink. Some of these data come from the post YAML metadata, and some are derived from the post source path. as we see below:</p>
<div class="sourceCode" id="cb16" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb16-1"><a href="#cb16-1" aria-hidden="true" tabindex="-1"></a><span class="ot">posts ::</span> <span class="dt">Rules</span> ()</span>
<span id="cb16-2"><a href="#cb16-2" aria-hidden="true" tabindex="-1"></a>posts <span class="ot">=</span> <span class="fu">map</span> indexHtmlOutputPath postGlobs <span class="op">|%></span> \target <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb16-3"><a href="#cb16-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> src <span class="ot">=</span> indexHtmlSourcePath target</span>
<span id="cb16-4"><a href="#cb16-4" aria-hidden="true" tabindex="-1"></a> post <span class="ot"><-</span> readPost src</span>
<span id="cb16-5"><a href="#cb16-5" aria-hidden="true" tabindex="-1"></a> postHtml <span class="ot"><-</span> applyTemplate <span class="st">"post.html"</span> post</span>
<span id="cb16-6"><a href="#cb16-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb16-7"><a href="#cb16-7" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> page <span class="ot">=</span> <span class="dt">Page</span> (postTitle post) postHtml</span>
<span id="cb16-8"><a href="#cb16-8" aria-hidden="true" tabindex="-1"></a> applyTemplateAndWrite <span class="st">"default.html"</span> page target</span>
<span id="cb16-9"><a href="#cb16-9" aria-hidden="true" tabindex="-1"></a> Shake.putInfo <span class="op">$</span> <span class="st">"Built "</span> <span class="op"><></span> target <span class="op"><></span> <span class="st">" from "</span> <span class="op"><></span> src</span>
<span id="cb16-10"><a href="#cb16-10" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb16-11"><a href="#cb16-11" aria-hidden="true" tabindex="-1"></a><span class="ot">readPost ::</span> <span class="dt">FilePath</span> <span class="ot">-></span> <span class="dt">Action</span> <span class="dt">Post</span></span>
<span id="cb16-12"><a href="#cb16-12" aria-hidden="true" tabindex="-1"></a>readPost postPath <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb16-13"><a href="#cb16-13" aria-hidden="true" tabindex="-1"></a> date <span class="ot"><-</span> parseTimeM <span class="dt">False</span> defaultTimeLocale <span class="st">"%Y-%-m-%-d"</span></span>
<span id="cb16-14"><a href="#cb16-14" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> <span class="fu">take</span> <span class="dv">10</span></span>
<span id="cb16-15"><a href="#cb16-15" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> Shake.takeBaseName</span>
<span id="cb16-16"><a href="#cb16-16" aria-hidden="true" tabindex="-1"></a> <span class="op">$</span> postPath</span>
<span id="cb16-17"><a href="#cb16-17" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> formattedDate <span class="ot">=</span></span>
<span id="cb16-18"><a href="#cb16-18" aria-hidden="true" tabindex="-1"></a> T.pack <span class="op">$</span> formatTime <span class="op">@</span><span class="dt">UTCTime</span> defaultTimeLocale <span class="st">"%b %e, %Y"</span> date</span>
<span id="cb16-19"><a href="#cb16-19" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb16-20"><a href="#cb16-20" aria-hidden="true" tabindex="-1"></a> (post, html) <span class="ot"><-</span> markdownToHtml postPath</span>
<span id="cb16-21"><a href="#cb16-21" aria-hidden="true" tabindex="-1"></a> Shake.putInfo <span class="op">$</span> <span class="st">"Read "</span> <span class="op"><></span> postPath</span>
<span id="cb16-22"><a href="#cb16-22" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> <span class="op">$</span> post</span>
<span id="cb16-23"><a href="#cb16-23" aria-hidden="true" tabindex="-1"></a> { postDate <span class="ot">=</span> <span class="dt">Just</span> formattedDate,</span>
<span id="cb16-24"><a href="#cb16-24" aria-hidden="true" tabindex="-1"></a> postContent <span class="ot">=</span> <span class="dt">Just</span> html,</span>
<span id="cb16-25"><a href="#cb16-25" aria-hidden="true" tabindex="-1"></a> postLink <span class="ot">=</span> <span class="dt">Just</span> <span class="op">.</span> T.pack <span class="op">$</span> <span class="st">"/"</span> <span class="op"><></span> Shake.dropExtension postPath <span class="op"><></span> <span class="st">"/"</span></span>
<span id="cb16-26"><a href="#cb16-26" aria-hidden="true" tabindex="-1"></a> }</span></code></pre></div>
<p>We call the <code>readPost</code> function, which parses the post date from the post path, and renders the post text using the <code>markdownToHtml</code> function. We then apply the <code class="sourceCode haskell"><span class="dt">Post</span></code> data to the <code>post.html</code> template to create the templated HTML content. Finally, we create the <code class="sourceCode haskell"><span class="dt">Page</span></code> data from the rendered post, apply it to the <code>default.html</code> template, and write the final HTML file to the target path.</p>
<p>The template for the post page can be seen below:</p>
<details>
<summary>
<code>templates/post.html</code>
</summary>
<div class="sourceCode" id="cb17" data-lang="mustache"><pre class="sourceCode numberSource mustache"><code class="sourceCode mustache"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a><span class="kw"><div</span><span class="ot"> class</span><span class="op">=</span><span class="st">"info"</span><span class="kw">></span></span>
<span id="cb17-2"><a href="#cb17-2" aria-hidden="true" tabindex="-1"></a> Posted on <span class="sc">{{{</span>date<span class="sc">}}}</span></span>
<span id="cb17-3"><a href="#cb17-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">{{#author}}</span></span>
<span id="cb17-4"><a href="#cb17-4" aria-hidden="true" tabindex="-1"></a> by <span class="sc">{{{</span>author<span class="sc">}}}</span></span>
<span id="cb17-5"><a href="#cb17-5" aria-hidden="true" tabindex="-1"></a> <span class="fu">{{/author}}</span></span>
<span id="cb17-6"><a href="#cb17-6" aria-hidden="true" tabindex="-1"></a><span class="kw"></div></span></span>
<span id="cb17-7"><a href="#cb17-7" aria-hidden="true" tabindex="-1"></a><span class="kw"><div</span><span class="ot"> class</span><span class="op">=</span><span class="st">"info"</span><span class="kw">></span></span>
<span id="cb17-8"><a href="#cb17-8" aria-hidden="true" tabindex="-1"></a> Tags:</span>
<span id="cb17-9"><a href="#cb17-9" aria-hidden="true" tabindex="-1"></a> <span class="kw"><ul</span><span class="ot"> class</span><span class="op">=</span><span class="st">"tags"</span><span class="kw">></span></span>
<span id="cb17-10"><a href="#cb17-10" aria-hidden="true" tabindex="-1"></a> <span class="fu">{{#tags}}</span></span>
<span id="cb17-11"><a href="#cb17-11" aria-hidden="true" tabindex="-1"></a> <span class="kw"><li><a</span><span class="ot"> href</span><span class="op">=</span><span class="st">"/tags/</span><span class="sc">{{{</span><span class="op">.</span><span class="sc">}}}</span><span class="st">/"</span><span class="kw">></span><span class="sc">{{{</span><span class="op">.</span><span class="sc">}}}</span><span class="kw"></a></li></span></span>
<span id="cb17-12"><a href="#cb17-12" aria-hidden="true" tabindex="-1"></a> <span class="fu">{{/tags}}</span></span>
<span id="cb17-13"><a href="#cb17-13" aria-hidden="true" tabindex="-1"></a> <span class="kw"></ul></span></span>
<span id="cb17-14"><a href="#cb17-14" aria-hidden="true" tabindex="-1"></a><span class="kw"></div></span></span>
<span id="cb17-15"><a href="#cb17-15" aria-hidden="true" tabindex="-1"></a><span class="sc">{{{</span>content<span class="sc">}}}</span></span></code></pre></div>
</details>
<h3 id="archive">Archive</h3>
<p>The archive page is a bit more involved. We read all the posts, and sort them by date. Then we apply the <code>archive.html</code> template, and then the <code>default.html</code> template to create the final HTML file, as shown below:</p>
<div class="sourceCode" id="cb18" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb18-1"><a href="#cb18-1" aria-hidden="true" tabindex="-1"></a><span class="ot">archive ::</span> <span class="dt">Rules</span> ()</span>
<span id="cb18-2"><a href="#cb18-2" aria-hidden="true" tabindex="-1"></a>archive <span class="ot">=</span> outputDir <span class="op"></></span> <span class="st">"archive/index.html"</span> <span class="op">%></span> \target <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb18-3"><a href="#cb18-3" aria-hidden="true" tabindex="-1"></a> postPaths <span class="ot"><-</span> Shake.getDirectoryFiles <span class="st">""</span> postGlobs</span>
<span id="cb18-4"><a href="#cb18-4" aria-hidden="true" tabindex="-1"></a> posts <span class="ot"><-</span> sortOn (<span class="dt">Ord</span><span class="op">.</span><span class="dt">Down</span> <span class="op">.</span> postDate) <span class="op"><$></span> forM postPaths readPost</span>
<span id="cb18-5"><a href="#cb18-5" aria-hidden="true" tabindex="-1"></a> writeArchive (T.pack <span class="st">"Archive"</span>) posts target</span>
<span id="cb18-6"><a href="#cb18-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb18-7"><a href="#cb18-7" aria-hidden="true" tabindex="-1"></a><span class="ot">writeArchive ::</span> <span class="dt">Text</span> <span class="ot">-></span> [<span class="dt">Post</span>] <span class="ot">-></span> <span class="dt">FilePath</span> <span class="ot">-></span> <span class="dt">Action</span> ()</span>
<span id="cb18-8"><a href="#cb18-8" aria-hidden="true" tabindex="-1"></a>writeArchive title posts target <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb18-9"><a href="#cb18-9" aria-hidden="true" tabindex="-1"></a> html <span class="ot"><-</span> applyTemplate <span class="st">"archive.html"</span> <span class="op">$</span> HM.singleton <span class="st">"posts"</span> posts</span>
<span id="cb18-10"><a href="#cb18-10" aria-hidden="true" tabindex="-1"></a> applyTemplateAndWrite <span class="st">"default.html"</span> (<span class="dt">Page</span> title html) target</span>
<span id="cb18-11"><a href="#cb18-11" aria-hidden="true" tabindex="-1"></a> Shake.putInfo <span class="op">$</span> <span class="st">"Built "</span> <span class="op"><></span> target</span></code></pre></div>
<p>The <code>archive.html</code> template transcludes the <code>post-list.html</code> template for reuse with the home page.</p>
<details>
<summary>
<code>templates/archive.html</code>
</summary>
<div class="sourceCode" id="cb19" data-lang="mustache"><pre class="sourceCode numberSource mustache"><code class="sourceCode mustache"><span id="cb19-1"><a href="#cb19-1" aria-hidden="true" tabindex="-1"></a>My posts:</span>
<span id="cb19-2"><a href="#cb19-2" aria-hidden="true" tabindex="-1"></a><span class="va">{{></span><span class="ch"> templates/post-list</span><span class="op">.</span><span class="ch">html </span><span class="va">}}</span></span></code></pre></div>
</details>
<details>
<summary>
<code>templates/post-list.html</code>
</summary>
<div class="sourceCode" id="cb20" data-lang="mustache"><pre class="sourceCode numberSource mustache"><code class="sourceCode mustache"><span id="cb20-1"><a href="#cb20-1" aria-hidden="true" tabindex="-1"></a><span class="kw"><ul></span></span>
<span id="cb20-2"><a href="#cb20-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">{{#posts}}</span></span>
<span id="cb20-3"><a href="#cb20-3" aria-hidden="true" tabindex="-1"></a> <span class="kw"><li></span></span>
<span id="cb20-4"><a href="#cb20-4" aria-hidden="true" tabindex="-1"></a> <span class="kw"><a</span><span class="ot"> href</span><span class="op">=</span><span class="st">"</span><span class="sc">{{{link}}}</span><span class="st">"</span><span class="kw">></span><span class="sc">{{{</span>title<span class="sc">}}}</span><span class="kw"></a></span> - <span class="sc">{{{</span>date<span class="sc">}}}</span></span>
<span id="cb20-5"><a href="#cb20-5" aria-hidden="true" tabindex="-1"></a> <span class="kw"></li></span></span>
<span id="cb20-6"><a href="#cb20-6" aria-hidden="true" tabindex="-1"></a> <span class="fu">{{/posts}}</span></span>
<span id="cb20-7"><a href="#cb20-7" aria-hidden="true" tabindex="-1"></a><span class="kw"></ul></span></span></code></pre></div>
</details>
<h3 id="tags">Tags</h3>
<p>Now, we build a page for each post tag. Step one is to read all the posts, collect the tags, and add build targets for each tag. We do this in the <code>buildTargets</code> function, as shown in the emphasized code below:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-emphasize="13-16"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">buildTargets ::</span> <span class="dt">Action</span> ()</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>buildTargets <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> assetPaths <span class="ot"><-</span> Shake.getDirectoryFiles <span class="st">""</span> assetGlobs</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">$</span> <span class="fu">map</span> (outputDir <span class="op"></></span>) assetPaths</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">$</span> <span class="fu">map</span> indexHtmlOutputPath pagePaths</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> postPaths <span class="ot"><-</span> Shake.getDirectoryFiles <span class="st">""</span> postGlobs</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">$</span> <span class="fu">map</span> indexHtmlOutputPath postPaths</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">$</span> <span class="fu">map</span> (outputDir <span class="op"></></span>) [<span class="st">"archive/index.html"</span>, <span class="st">"index.html"</span>]</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> posts <span class="ot"><-</span> forM postPaths readPost</span></span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> Shake.need</span></span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> [ outputDir <span class="op"></></span> <span class="st">"tags"</span> <span class="op"></></span> T.unpack tag <span class="op"></></span> <span class="st">"index.html"</span></span></span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a><span class="emphasis"> <span class="op">|</span> post <span class="ot"><-</span> posts, tag <span class="ot"><-</span> postTags post ]</span></span></code></pre></div>
<p>Next, we implement the build rule for tags:</p>
<div class="sourceCode" id="cb21" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb21-1"><a href="#cb21-1" aria-hidden="true" tabindex="-1"></a><span class="ot">tags ::</span> <span class="dt">Rules</span> ()</span>
<span id="cb21-2"><a href="#cb21-2" aria-hidden="true" tabindex="-1"></a>tags <span class="ot">=</span> outputDir <span class="op"></></span> <span class="st">"tags/*/index.html"</span> <span class="op">%></span> \target <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb21-3"><a href="#cb21-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> tag <span class="ot">=</span> T.pack <span class="op">$</span> Shake.splitDirectories target <span class="op">!!</span> <span class="dv">2</span></span>
<span id="cb21-4"><a href="#cb21-4" aria-hidden="true" tabindex="-1"></a> postPaths <span class="ot"><-</span> Shake.getDirectoryFiles <span class="st">""</span> postGlobs</span>
<span id="cb21-5"><a href="#cb21-5" aria-hidden="true" tabindex="-1"></a> posts <span class="ot"><-</span> sortOn (<span class="dt">Ord</span><span class="op">.</span><span class="dt">Down</span> <span class="op">.</span> postDate)</span>
<span id="cb21-6"><a href="#cb21-6" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> <span class="fu">filter</span> ((tag <span class="ot">`elem`</span>) <span class="op">.</span> postTags)</span>
<span id="cb21-7"><a href="#cb21-7" aria-hidden="true" tabindex="-1"></a> <span class="op"><$></span> forM postPaths readPost</span>
<span id="cb21-8"><a href="#cb21-8" aria-hidden="true" tabindex="-1"></a> writeArchive (T.pack <span class="st">"Posts tagged "</span> <span class="op"><></span> tag) posts target</span></code></pre></div>
<p>First, we parse the tag from the target path. We then read all the posts, filter them by tag, and render the tag page using the <code>writeArchive</code> function that we use for the archive page.</p>
<h3 id="home">Home</h3>
<p>Finally, we come to the home page. It is quite similar to the archive page, except that we only show the first few posts<a href="#fn7" class="footnote-ref" id="fnref7" role="doc-noteref"><sup>7</sup></a><a href="#fn8" class="footnote-ref" id="fnref8" role="doc-noteref"><sup>8</sup></a>:</p>
<div class="sourceCode" id="cb22" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb22-1"><a href="#cb22-1" aria-hidden="true" tabindex="-1"></a><span class="ot">home ::</span> <span class="dt">Rules</span> ()</span>
<span id="cb22-2"><a href="#cb22-2" aria-hidden="true" tabindex="-1"></a>home <span class="ot">=</span> outputDir <span class="op"></></span> <span class="st">"index.html"</span> <span class="op">%></span> \target <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb22-3"><a href="#cb22-3" aria-hidden="true" tabindex="-1"></a> postPaths <span class="ot"><-</span> Shake.getDirectoryFiles <span class="st">""</span> postGlobs</span>
<span id="cb22-4"><a href="#cb22-4" aria-hidden="true" tabindex="-1"></a> posts <span class="ot"><-</span> <span class="fu">take</span> <span class="dv">3</span></span>
<span id="cb22-5"><a href="#cb22-5" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> sortOn (<span class="dt">Ord</span><span class="op">.</span><span class="dt">Down</span> <span class="op">.</span> postDate)</span>
<span id="cb22-6"><a href="#cb22-6" aria-hidden="true" tabindex="-1"></a> <span class="op"><$></span> forM postPaths readPost</span>
<span id="cb22-7"><a href="#cb22-7" aria-hidden="true" tabindex="-1"></a> html <span class="ot"><-</span> applyTemplate <span class="st">"home.html"</span> <span class="op">$</span> HM.singleton <span class="st">"posts"</span> posts</span>
<span id="cb22-8"><a href="#cb22-8" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb22-9"><a href="#cb22-9" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> page <span class="ot">=</span> <span class="dt">Page</span> (T.pack <span class="st">"Home"</span>) html</span>
<span id="cb22-10"><a href="#cb22-10" aria-hidden="true" tabindex="-1"></a> applyTemplateAndWrite <span class="st">"default.html"</span> page target</span>
<span id="cb22-11"><a href="#cb22-11" aria-hidden="true" tabindex="-1"></a> Shake.putInfo <span class="op">$</span> <span class="st">"Built "</span> <span class="op"><></span> target</span></code></pre></div>
<p>The <code>home.html</code> template also transcludes the <code>post-list.html</code> template:</p>
<details>
<summary>
<code>templates/home.html</code>
</summary>
<div class="sourceCode" id="cb23" data-lang="mustache"><pre class="sourceCode numberSource mustache"><code class="sourceCode mustache"><span id="cb23-1"><a href="#cb23-1" aria-hidden="true" tabindex="-1"></a><span class="kw"><h2></span>Welcome<span class="kw"></h2></span></span>
<span id="cb23-2"><a href="#cb23-2" aria-hidden="true" tabindex="-1"></a><span class="kw"><img</span><span class="ot"> src</span><span class="op">=</span><span class="st">"/images/logo.png"</span><span class="ot"> style</span><span class="op">=</span><span class="st">"float: right; margin: 10px;"</span> <span class="kw">/></span></span>
<span id="cb23-3"><a href="#cb23-3" aria-hidden="true" tabindex="-1"></a><span class="kw"><p></span>Welcome to my blog!<span class="kw"></p></span></span>
<span id="cb23-4"><a href="#cb23-4" aria-hidden="true" tabindex="-1"></a><span class="kw"><p></span>My recent posts here for your reading pleasure:<span class="kw"></p></span></span>
<span id="cb23-5"><a href="#cb23-5" aria-hidden="true" tabindex="-1"></a><span class="kw"><h2></span>Posts<span class="kw"></h2></span></span>
<span id="cb23-6"><a href="#cb23-6" aria-hidden="true" tabindex="-1"></a><span class="va">{{></span><span class="ch"> templates/post-list</span><span class="op">.</span><span class="ch">html </span><span class="va">}}</span></span>
<span id="cb23-7"><a href="#cb23-7" aria-hidden="true" tabindex="-1"></a><span class="kw"><p></span>You can find all posts in the <span class="kw"><a</span><span class="ot"> href</span><span class="op">=</span><span class="st">"/archive/"</span><span class="kw">></span>archives<span class="kw"></a></span>.</span></code></pre></div>
</details>
<p>That’s it for the build rules. We have covered all the targets that we defined in the <code>buildTargets</code> function. Next, we look at the Pandoc and Mustache utilities that we use in the build rules.</p>
<h2 data-track-content data-content-name="utilities" data-content-piece="static-site-generator-using-shake" id="utilities">Utilities</h2>
<p>We use the Pandoc library to render Markdown to HTML. We also use the Mustache library to render the generated HTML with the Mustache templates. We wrap these libraries in a few utility functions, as shown in the next sections.</p>
<h3 id="pandoc">Pandoc</h3>
<p>We wrap Pandoc’s Markdown-to-HTML rendering to make it a Shake build action. We also parse the YAML metadata from the Markdown source, and return it as a <code class="sourceCode haskell"><span class="dt">FromJSON</span></code>-able value<a href="#fn9" class="footnote-ref" id="fnref9" role="doc-noteref"><sup>9</sup></a>.</p>
<div class="sourceCode" id="cb24" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb24-1"><a href="#cb24-1" aria-hidden="true" tabindex="-1"></a><span class="ot">markdownToHtml ::</span> <span class="dt">FromJSON</span> a <span class="ot">=></span> <span class="dt">FilePath</span> <span class="ot">-></span> <span class="dt">Action</span> (a, <span class="dt">Text</span>)</span>
<span id="cb24-2"><a href="#cb24-2" aria-hidden="true" tabindex="-1"></a>markdownToHtml filePath <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb24-3"><a href="#cb24-3" aria-hidden="true" tabindex="-1"></a> content <span class="ot"><-</span> Shake.readFile' filePath</span>
<span id="cb24-4"><a href="#cb24-4" aria-hidden="true" tabindex="-1"></a> Shake.quietly <span class="op">.</span> Shake.traced <span class="st">"Markdown to HTML"</span> <span class="op">$</span> <span class="kw">do</span></span>
<span id="cb24-5"><a href="#cb24-5" aria-hidden="true" tabindex="-1"></a> pandoc<span class="op">@</span>(<span class="dt">Pandoc</span> meta _) <span class="ot"><-</span></span>
<span id="cb24-6"><a href="#cb24-6" aria-hidden="true" tabindex="-1"></a> runPandoc <span class="op">.</span> Pandoc.readMarkdown readerOptions <span class="op">.</span> T.pack <span class="op">$</span> content</span>
<span id="cb24-7"><a href="#cb24-7" aria-hidden="true" tabindex="-1"></a> meta' <span class="ot"><-</span> fromMeta meta</span>
<span id="cb24-8"><a href="#cb24-8" aria-hidden="true" tabindex="-1"></a> html <span class="ot"><-</span> runPandoc <span class="op">.</span> Pandoc.writeHtml5String writerOptions <span class="op">$</span> pandoc</span>
<span id="cb24-9"><a href="#cb24-9" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> (meta', html)</span>
<span id="cb24-10"><a href="#cb24-10" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb24-11"><a href="#cb24-11" aria-hidden="true" tabindex="-1"></a> readerOptions <span class="ot">=</span></span>
<span id="cb24-12"><a href="#cb24-12" aria-hidden="true" tabindex="-1"></a> Pandoc.def {Pandoc.readerExtensions <span class="ot">=</span> Pandoc.pandocExtensions}</span>
<span id="cb24-13"><a href="#cb24-13" aria-hidden="true" tabindex="-1"></a> writerOptions <span class="ot">=</span></span>
<span id="cb24-14"><a href="#cb24-14" aria-hidden="true" tabindex="-1"></a> Pandoc.def {Pandoc.writerExtensions <span class="ot">=</span> Pandoc.pandocExtensions}</span>
<span id="cb24-15"><a href="#cb24-15" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb24-16"><a href="#cb24-16" aria-hidden="true" tabindex="-1"></a> fromMeta (<span class="dt">Meta</span> meta) <span class="ot">=</span></span>
<span id="cb24-17"><a href="#cb24-17" aria-hidden="true" tabindex="-1"></a> A.fromJSON <span class="op">.</span> A.toJSON <span class="op"><$></span> <span class="fu">traverse</span> metaValueToJSON meta <span class="op">>>=</span> \<span class="kw">case</span></span>
<span id="cb24-18"><a href="#cb24-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">Success</span> res <span class="ot">-></span> <span class="fu">pure</span> res</span>
<span id="cb24-19"><a href="#cb24-19" aria-hidden="true" tabindex="-1"></a> <span class="dt">Error</span> err <span class="ot">-></span> <span class="fu">fail</span> <span class="op">$</span> <span class="st">"json conversion error:"</span> <span class="op"><></span> err</span>
<span id="cb24-20"><a href="#cb24-20" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb24-21"><a href="#cb24-21" aria-hidden="true" tabindex="-1"></a> metaValueToJSON <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb24-22"><a href="#cb24-22" aria-hidden="true" tabindex="-1"></a> <span class="dt">MetaMap</span> m <span class="ot">-></span> A.toJSON <span class="op"><$></span> <span class="fu">traverse</span> metaValueToJSON m</span>
<span id="cb24-23"><a href="#cb24-23" aria-hidden="true" tabindex="-1"></a> <span class="dt">MetaList</span> m <span class="ot">-></span> A.toJSONList <span class="op"><$></span> <span class="fu">traverse</span> metaValueToJSON m</span>
<span id="cb24-24"><a href="#cb24-24" aria-hidden="true" tabindex="-1"></a> <span class="dt">MetaBool</span> m <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> A.toJSON m</span>
<span id="cb24-25"><a href="#cb24-25" aria-hidden="true" tabindex="-1"></a> <span class="dt">MetaString</span> m <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> A.toJSON <span class="op">$</span> T.strip m</span>
<span id="cb24-26"><a href="#cb24-26" aria-hidden="true" tabindex="-1"></a> <span class="dt">MetaInlines</span> m <span class="ot">-></span> metaValueToJSON <span class="op">$</span> <span class="dt">MetaBlocks</span> [<span class="dt">Plain</span> m]</span>
<span id="cb24-27"><a href="#cb24-27" aria-hidden="true" tabindex="-1"></a> <span class="dt">MetaBlocks</span> m <span class="ot">-></span></span>
<span id="cb24-28"><a href="#cb24-28" aria-hidden="true" tabindex="-1"></a> <span class="fu">fmap</span> (A.toJSON <span class="op">.</span> T.strip)</span>
<span id="cb24-29"><a href="#cb24-29" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> runPandoc</span>
<span id="cb24-30"><a href="#cb24-30" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> Pandoc.writePlain Pandoc.def</span>
<span id="cb24-31"><a href="#cb24-31" aria-hidden="true" tabindex="-1"></a> <span class="op">$</span> <span class="dt">Pandoc</span> <span class="fu">mempty</span> m</span>
<span id="cb24-32"><a href="#cb24-32" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb24-33"><a href="#cb24-33" aria-hidden="true" tabindex="-1"></a> runPandoc action <span class="ot">=</span></span>
<span id="cb24-34"><a href="#cb24-34" aria-hidden="true" tabindex="-1"></a> Pandoc.runIO (Pandoc.setVerbosity <span class="dt">Pandoc.ERROR</span> <span class="op">>></span> action)</span>
<span id="cb24-35"><a href="#cb24-35" aria-hidden="true" tabindex="-1"></a> <span class="op">>>=</span> <span class="fu">either</span> (<span class="fu">fail</span> <span class="op">.</span> <span class="fu">show</span>) <span class="fu">return</span></span></code></pre></div>
<h3 id="mustache">Mustache</h3>
<p>We wrap Mustache’s template reading and rendering to make them Shake build actions.</p>
<div class="sourceCode" id="cb25" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb25-1"><a href="#cb25-1" aria-hidden="true" tabindex="-1"></a><span class="ot">applyTemplate ::</span> <span class="dt">ToJSON</span> a <span class="ot">=></span> <span class="dt">String</span> <span class="ot">-></span> a <span class="ot">-></span> <span class="dt">Action</span> <span class="dt">Text</span></span>
<span id="cb25-2"><a href="#cb25-2" aria-hidden="true" tabindex="-1"></a>applyTemplate templateName context <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb25-3"><a href="#cb25-3" aria-hidden="true" tabindex="-1"></a> tmpl <span class="ot"><-</span> readTemplate <span class="op">$</span> <span class="st">"templates"</span> <span class="op"></></span> templateName</span>
<span id="cb25-4"><a href="#cb25-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">case</span> Mus.checkedSubstitute tmpl (A.toJSON context) <span class="kw">of</span></span>
<span id="cb25-5"><a href="#cb25-5" aria-hidden="true" tabindex="-1"></a> ([], text) <span class="ot">-></span> <span class="fu">return</span> text</span>
<span id="cb25-6"><a href="#cb25-6" aria-hidden="true" tabindex="-1"></a> (errs, _) <span class="ot">-></span> <span class="fu">fail</span> <span class="op">$</span></span>
<span id="cb25-7"><a href="#cb25-7" aria-hidden="true" tabindex="-1"></a> <span class="st">"Error while substituting template "</span> <span class="op"><></span> templateName</span>
<span id="cb25-8"><a href="#cb25-8" aria-hidden="true" tabindex="-1"></a> <span class="op"><></span> <span class="st">": "</span> <span class="op"><></span> <span class="fu">unlines</span> (<span class="fu">map</span> <span class="fu">show</span> errs)</span>
<span id="cb25-9"><a href="#cb25-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb25-10"><a href="#cb25-10" aria-hidden="true" tabindex="-1"></a><span class="ot">applyTemplateAndWrite ::</span> <span class="dt">ToJSON</span> a <span class="ot">=></span> <span class="dt">String</span> <span class="ot">-></span> a <span class="ot">-></span> <span class="dt">FilePath</span> <span class="ot">-></span> <span class="dt">Action</span> ()</span>
<span id="cb25-11"><a href="#cb25-11" aria-hidden="true" tabindex="-1"></a>applyTemplateAndWrite templateName context outputPath <span class="ot">=</span></span>
<span id="cb25-12"><a href="#cb25-12" aria-hidden="true" tabindex="-1"></a> applyTemplate templateName context</span>
<span id="cb25-13"><a href="#cb25-13" aria-hidden="true" tabindex="-1"></a> <span class="op">>>=</span> Shake.writeFile' outputPath <span class="op">.</span> T.unpack</span>
<span id="cb25-14"><a href="#cb25-14" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb25-15"><a href="#cb25-15" aria-hidden="true" tabindex="-1"></a><span class="ot">readTemplate ::</span> <span class="dt">FilePath</span> <span class="ot">-></span> <span class="dt">Action</span> <span class="dt">Mus.Template</span></span>
<span id="cb25-16"><a href="#cb25-16" aria-hidden="true" tabindex="-1"></a>readTemplate templatePath <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb25-17"><a href="#cb25-17" aria-hidden="true" tabindex="-1"></a> Shake.need [templatePath]</span>
<span id="cb25-18"><a href="#cb25-18" aria-hidden="true" tabindex="-1"></a> eTemplate <span class="ot"><-</span> Shake.quietly</span>
<span id="cb25-19"><a href="#cb25-19" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> Shake.traced <span class="st">"Compile template"</span></span>
<span id="cb25-20"><a href="#cb25-20" aria-hidden="true" tabindex="-1"></a> <span class="op">$</span> Mus.localAutomaticCompile templatePath</span>
<span id="cb25-21"><a href="#cb25-21" aria-hidden="true" tabindex="-1"></a> <span class="kw">case</span> eTemplate <span class="kw">of</span></span>
<span id="cb25-22"><a href="#cb25-22" aria-hidden="true" tabindex="-1"></a> <span class="dt">Right</span> template <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb25-23"><a href="#cb25-23" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">.</span> Mus.getPartials <span class="op">.</span> Mus.ast <span class="op">$</span> template</span>
<span id="cb25-24"><a href="#cb25-24" aria-hidden="true" tabindex="-1"></a> Shake.putInfo <span class="op">$</span> <span class="st">"Read "</span> <span class="op"><></span> templatePath</span>
<span id="cb25-25"><a href="#cb25-25" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> template</span>
<span id="cb25-26"><a href="#cb25-26" aria-hidden="true" tabindex="-1"></a> <span class="dt">Left</span> err <span class="ot">-></span> <span class="fu">fail</span> <span class="op">$</span> <span class="fu">show</span> err</span></code></pre></div>
<p>The <code>readTemplate</code> function specially takes care of marking the template (and its transcluded templates) as dependencies of pages that use them. By doing this, Shake rebuilds the pages if any of the templates change.</p>
<h2 data-track-content data-content-name="building-the-blog" data-content-piece="static-site-generator-using-shake" id="building-the-blog">Building the Blog</h2>
<p>We are now ready to run the build:</p>
<details>
<summary>
Build log
</summary>
<pre class="plain"><code>$ ./Site.hs clean
Build completed in 0.02s
$ ./Site.hs build
Copied _site/images/logo.png from images/logo.png
Copied _site/css/default.css from css/default.css
Read templates/default.html
Built _site/contact/index.html from contact.md
Read templates/default.html
Built _site/about/index.html from about.md
Read posts/2022-10-07-hello-world.md
Read templates/post.html
Read templates/default.html
Built _site/posts/2022-10-07-hello-world/index.html from posts/2022-10-07-hello-world.md
Read posts/2022-08-12-welcome.md
Read templates/post.html
Read templates/default.html
Built _site/posts/2022-08-12-welcome/index.html from posts/2022-08-12-welcome.md
Read posts/2022-08-12-welcome.md
Read posts/2022-10-07-hello-world.md
Read templates/home.html
Read templates/default.html
Built _site/index.html
Read posts/2022-08-12-welcome.md
Read posts/2022-10-07-hello-world.md
Read templates/archive.html
Read templates/default.html
Built _site/archive/index.html
Read posts/2022-08-12-welcome.md
Read posts/2022-10-07-hello-world.md
Read posts/2022-08-12-welcome.md
Read posts/2022-10-07-hello-world.md
Read templates/archive.html
Read templates/default.html
Built _site/tags/programming/index.html
Read posts/2022-08-12-welcome.md
Read posts/2022-10-07-hello-world.md
Read templates/archive.html
Read templates/default.html
Built _site/tags/note/index.html
Read posts/2022-08-12-welcome.md
Read posts/2022-10-07-hello-world.md
Read templates/archive.html
Read templates/default.html
Built _site/tags/brag/index.html
Build completed in 0.10s</code></pre>
</details>
<p>The logs show that Shake built all the targets that we define in the <code>buildTargets</code> function<a href="#fn10" class="footnote-ref" id="fnref10" role="doc-noteref"><sup>10</sup></a><a href="#fn11" class="footnote-ref" id="fnref11" role="doc-noteref"><sup>11</sup></a>.</p>
<p>Next, we look into some helpful Shake specific features.</p>
<h2 data-track-content data-content-name="shake-features" data-content-piece="static-site-generator-using-shake" id="shake-features">Shake Features</h2>
<p>Being a generic build system, Shake has some unique features that are not found in most other <abbr title="Static site generator">SSGs</abbr>. In this section, we look at some of these features.</p>
<h3 id="caching">Caching</h3>
<p>As we see in the build log above, the posts and templates are read multiple times. This is because Shake does not cache the dependencies of build rules by default. However, we can add caching by using the <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:newCacheIO" target="_blank" rel="noopener"><code>newCacheIO</code></a> function<a href="#fn12" class="footnote-ref" id="fnref12" role="doc-noteref"><sup>12</sup></a><a href="#fn13" class="footnote-ref" id="fnref13" role="doc-noteref"><sup>13</sup></a>. Once we add caching, the build log show that the posts and templates are read only once:</p>
<details>
<summary>
Build log
</summary>
<pre class="plain"><code>Copied _site/images/logo.png from images/logo.png
Copied _site/css/default.css from css/default.css
Read templates/default.html
Built _site/contact/index.html from contact.md
Built _site/about/index.html from about.md
Read posts/2022-08-12-welcome.md
Read templates/post.html
Built _site/posts/2022-08-12-welcome/index.html from posts/2022-08-12-welcome.md
Read posts/2022-10-07-hello-world.md
Built _site/posts/2022-10-07-hello-world/index.html from posts/2022-10-07-hello-world.md
Read templates/home.html
Built _site/index.html
Read templates/archive.html
Built _site/archive/index.html
Built _site/tags/programming/index.html
Built _site/tags/note/index.html
Built _site/tags/brag/index.html
Build completed in 0.03s</code></pre>
</details>
<h3 id="parallelism">Parallelism</h3>
<p>Shake can run build actions in parallel. We can enable parallelism by using the <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:shakeThreads" target="_blank" rel="noopener"><code>shakeThreads</code></a> configuration option, or by using the <code>--jobs</code> command line option. Enabling parallel builds can reduce build times significantly.</p>
<p>Shake tries to automatically detect which build actions can be run in parallel. However, we can specify it explicitly as well. We explore this in the <a href="#tips-and-tricks">Tips and Tricks</a> section.</p>
<h3 id="fine-grain-dependency-management">Fine-grain Dependency Management</h3>
<p>Using the <code>Shake.need</code> function, we can explicitly specify the dependencies of a build target. For example, we can use it to mark the <code>Site.hs</code> file as a dependency of all targets. This way, Shake rebuilds the site if the build script changes. We have already seen how we can use it to mark the templates as dependencies of pages that use them.</p>
<h3 id="traces-and-reports">Traces and Reports</h3>
<p>Shake can be instructed to generate build traces and reports. These can be used to understand/debug/improve the builds. We can enable these features by using the <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:shakeReport" target="_blank" rel="noopener"><code>shakeReport</code></a> configuration option, or by using the <code>--report</code> command line option.</p>
<p>The report generated by Shake shows time taken by each build rule, their dependency graph, and the command plot traced by the <code>Shake.traced</code> function. For example, here is the command plot for a build of the website you are reading right now<a href="#fn14" class="footnote-ref" id="fnref14" role="doc-noteref"><sup>14</sup></a>:</p>
<details>
<summary>
Command plot of a build of this website
</summary>
<figure>
<a href="https://abhinavsarkar.net/images/static-site-generator-using-shake/command-plot.png" class="img-link"><img src class="lazyload" data-src="/images/static-site-generator-using-shake/command-plot.png" alt="Command plot of a build of this website"></img>
<noscript><img src="/images/static-site-generator-using-shake/command-plot.png" alt="Command plot of a build of this website"></img></noscript></a>
<figcaption>Command plot of a build of this website</figcaption>
</figure>
</details>
<p>The traces can be viewed using a trace viewer like <a href="https://ui.perfetto.dev/" target="_blank" rel="noopener">Perfetto</a>. For example, here is a trace of a build of this website:</p>
<details>
<summary>
Trace of a build of this website
</summary>
<figure>
<a href="https://abhinavsarkar.net/images/static-site-generator-using-shake/trace.png" class="img-link"><img src class="lazyload" data-src="/images/static-site-generator-using-shake/trace.png" alt="Trace of a build of this website"></img>
<noscript><img src="/images/static-site-generator-using-shake/trace.png" alt="Trace of a build of this website"></img></noscript></a>
<figcaption>Trace of a build of this website</figcaption>
</figure>
</details>
<h3 id="errors">Errors</h3>
<p>Shake provides detailed error messages when builds fail. For example, here is the error message when a build fails due to a missing template:</p>
<pre class="plain"><code>Error when running Shake build system:
at want, called at src/Development/Shake/Internal/Args.hs:83:67 in shake-0.19.7-IRPInZXX5QOAqz04qHWdHp:Development.Shake.Internal.Args
* Depends on: build
at need, called at Site.hs:54:3 in main:Main
* Depends on: _site/posts/2022-10-07-hello-world/index.html
* Depends on: templates/post.html
at error, called at src/Development/Shake/Internal/Rules/File.hs:179:58 in shake-0.19.7-IRPInZXX5QOAqz04qHWdHp:Development.Shake.Internal.Rules.File
* Raised the exception:
Error, file does not exist and no rule available:
templates/post.html</code></pre>
<p>To learn more about Shake, read the Shake <a href="https://shakebuild.com/manual" target="_blank" rel="noopener">manual</a> and the <a href="https://shakebuild.com/faq" target="_blank" rel="noopener">FAQ</a>.</p>
<h2 data-track-content data-content-name="tips-and-tricks" data-content-piece="static-site-generator-using-shake" id="tips-and-tricks">Tips and Tricks</h2>
<p>Let’s look at some tips and tricks that can be used to improve the build.</p>
<h3 id="explicit-parallelism">Explicit Parallelism</h3>
<p>Shake is a monadic build system. That means, while the build actions are executing for a build target, they can add new dependencies for the target. These dependencies can depend on the result of previous build actions. So, Shake cannot know all the dependencies of a build target before the build actions for it are executed. This makes it difficult for Shake to automatically detect which build actions can be run in parallel.</p>
<p>However, we can explicitly specify it by using the <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:parallel" target="_blank" rel="noopener"><code>parallel</code></a>, and <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:forP" target="_blank" rel="noopener"><code>forP</code></a>, and <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:par" target="_blank" rel="noopener"><code>par</code></a> functions<a href="#fn15" class="footnote-ref" id="fnref15" role="doc-noteref"><sup>15</sup></a>. Additionally, Shake also builds all builds targets specified in a single <code>Shake.need</code> call in parallel. Here is how we can improve the parallelism of our <abbr title="Static site generator">SSG</abbr> using these functions:</p>
<div class="sourceCode" id="cb29" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb29-1"><a href="#cb29-1" aria-hidden="true" tabindex="-1"></a><span class="ot">buildTargetsParallel ::</span> <span class="dt">Action</span> ()</span>
<span id="cb29-2"><a href="#cb29-2" aria-hidden="true" tabindex="-1"></a>buildTargetsParallel <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb29-3"><a href="#cb29-3" aria-hidden="true" tabindex="-1"></a> (assetPaths, postPaths) <span class="ot"><-</span></span>
<span id="cb29-4"><a href="#cb29-4" aria-hidden="true" tabindex="-1"></a> Shake.getDirectoryFiles <span class="st">""</span> assetGlobs</span>
<span id="cb29-5"><a href="#cb29-5" aria-hidden="true" tabindex="-1"></a> <span class="ot">`Shake.par`</span> Shake.getDirectoryFiles <span class="st">""</span> postGlobs</span>
<span id="cb29-6"><a href="#cb29-6" aria-hidden="true" tabindex="-1"></a> posts <span class="ot"><-</span> Shake.forP postPaths readPost</span>
<span id="cb29-7"><a href="#cb29-7" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb29-8"><a href="#cb29-8" aria-hidden="true" tabindex="-1"></a> void <span class="op">$</span> Shake.parallel [</span>
<span id="cb29-9"><a href="#cb29-9" aria-hidden="true" tabindex="-1"></a> Shake.need <span class="op">$</span></span>
<span id="cb29-10"><a href="#cb29-10" aria-hidden="true" tabindex="-1"></a> <span class="fu">map</span> (outputDir <span class="op"></></span>)</span>
<span id="cb29-11"><a href="#cb29-11" aria-hidden="true" tabindex="-1"></a> (assetPaths <span class="op"><></span> [<span class="st">"archive/index.html"</span>, <span class="st">"index.html"</span>]</span>
<span id="cb29-12"><a href="#cb29-12" aria-hidden="true" tabindex="-1"></a> <span class="op"><></span> [<span class="st">"tags"</span> <span class="op"></></span> T.unpack tag <span class="op"></></span> <span class="st">"index.html"</span></span>
<span id="cb29-13"><a href="#cb29-13" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> post <span class="ot"><-</span> posts, tag <span class="ot"><-</span> postTags post])</span>
<span id="cb29-14"><a href="#cb29-14" aria-hidden="true" tabindex="-1"></a> , Shake.need <span class="op">$</span> <span class="fu">map</span> indexHtmlOutputPath (pagePaths <span class="op"><></span> postPaths)</span>
<span id="cb29-15"><a href="#cb29-15" aria-hidden="true" tabindex="-1"></a> ]</span></code></pre></div>
<h3 id="faster-builds">Faster Builds</h3>
<p>There are different modes in which we can run our builds depending on the complexity of our generator, and our preferences:</p>
<ol type="1">
<li>Run the build script without compiling it using the <code>runhaskell</code> command.</li>
<li>Compile the build script using <code>ghc</code> or <code>cabal</code> every time we have to run the build, and then run the build using the compiled executable.</li>
<li>Compile the build script using <code>ghc</code> or <code>cabal</code> once, and then run the build using the compiled executable.</li>
</ol>
<p>Mode 1 is good enough for small scripts. However, it is slow for large scripts because it runs the script using an interpreter, which is slower than running a compiled executable.</p>
<p>Mode 2 and 3 speed up the build by compiling the build script. However, they have different tradeoffs: mode 2 is good if we change the build script often, but, it is useless work if the build script stays the same. Mode 3 is good if the build script does not change often. But if we do change it often, we’ll have to remember to recompile it.</p>
<p>If we go with compiling the script, we can use the tips in <a href="https://www.parsonsmatt.org/2019/11/27/keeping_compilation_fast.html" target="_blank" rel="noopener">this article</a> to speed up the compilation. Additionally, hand-writing the JSON instances for data types instead of deriving them also gives a noticeable speedup. We may also want to switch on/off optimizations by passing the <code>-O2</code>/<code>-O0</code> flag to <code>ghc</code> or <code>cabal</code> to speed up the compilation. We may also enable parallel compilation by passing the <code>-j</code> flag.</p>
<p>If we decide to go with mode 2, that is, to compile the build script every time we run the build, we may want to use <a href="https://cabal.readthedocs.io/en/3.6/cabal-project.html#cfg-field-executable-dynamic" target="_blank" rel="noopener">dynamic linking</a> to reduce linking time.</p>
<p>When running the build using a compiled executable, Shake recommends switching on multithreading but switching off idle and parallel garbage collection. Additionally, we may also want to tune the allocation area sizes for the garbage collector.</p>
<p>Putting all this together, we may want to use the following flags to compile the generator in mode 2:</p>
<pre class="plain"><code>-O0 -dynamic -j -threaded -rtsopts "-with-rtsopts=-I0 -qg -N -A32m -n8m"</code></pre>
<p>and these flags for mode 3:</p>
<pre class="plain"><code>-O2 -j -threaded -rtsopts "-with-rtsopts=-I0 -qg -N -A32m -n8m"</code></pre>
<p>However, these flags are suggestions only. We should experiment with them to find the best combination for our build.</p>
<h3 id="watch-and-serve">Watch and Serve</h3>
<p>We can add support for automatically rebuilding the site when the Markdown files or assets change using the <a href="https://hackage.haskell.org/package/fsnotify" target="_blank" rel="noopener">fsnotify</a> package. We can add support for automatic rebuilding for the Haskell source as well using <a href="http://eradman.com/entrproject/" target="_blank" rel="noopener"><code>entr</code></a> to rerun the script, or using <a href="https://github.com/ndmitchell/ghcid" target="_blank" rel="noopener"><code>ghcid</code></a> to re-interpret the script on every change.</p>
<p>We can also add support for serving the site using the <a href="https://hackage.haskell.org/package/warp" target="_blank" rel="noopener">warp</a> and <a href="https://hackage.haskell.org/package/wai-app-static" target="_blank" rel="noopener">wai-app-static</a> packages<a href="#fn16" class="footnote-ref" id="fnref16" role="doc-noteref"><sup>16</sup></a><a href="#fn17" class="footnote-ref" id="fnref17" role="doc-noteref"><sup>17</sup></a>. We can add live reloading on the browser side using the <a href="https://livejs.com/" target="_blank" rel="noopener">livejs</a> JavaScript library.</p>
<p>Together, these features give us a hot-reloading development environment with fast feedback loop for our <abbr title="Static site generator">SSG</abbr>.</p>
<h2 data-track-content data-content-name="conclusion" data-content-piece="static-site-generator-using-shake" id="conclusion">Conclusion</h2>
<p>In this article, we looked at how we can use Shake to build a static site generator. We also looked at Shake specific features, and some tips and tricks that can be used to improve the build. Shake offers flexibility that is unparalleled by other <abbr title="Static site generator">SSGs</abbr>, but at the cost of writing your own build script. However, if you do want to write your own <abbr title="Static site generator">SSG</abbr>, Shake is a great choice as the foundation for it.</p>
<h2 class="notoc" data-track-content data-content-name="acknowledgements" data-content-piece="static-site-generator-using-shake" id="acknowledgements">Acknowledgements</h2>
<p>Many thanks to <a href="https://arunraghavan.net/" target="_blank" rel="noopener">Arun Raghavan</a> and <a href="https://deobald.ca/" target="_blank" rel="noopener">Steven Deobald</a> for reviewing a draft of this article.</p>
<section id="footnotes" class="footnotes footnotes-end-of-document" role="doc-endnotes">
<hr></hr>
<ol>
<li id="fn1"><p><a href="https://jekyllrb.com/" target="_blank" rel="noopener">Jekyll</a> was the first modern <abbr title="Static site generator">SSG</abbr> released in 2008. Since then, there has been a <a href="https://staticsitegenerators.net/" target="_blank" rel="noopener">proliferation</a> of <abbr title="Static site generator">SSGs</abbr>.<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn2"><p>There are already a few <abbr title="Static site generator">SSGs</abbr> that use Shake as their build system. See <a href="https://github.com/ChrisPenner/slick#" target="_blank" rel="noopener">Slick</a> and <a href="https://github.com/srid/rib" target="_blank" rel="noopener">Rib</a>.<a href="#fnref2" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn3"><p>Shake is a monadic and suspending build system. Being monadic here means that while build actions are executing, they can add new dependencies for build targets, and those dependencies can depend on the results of previous build actions. Being suspending means that when a build action requires a dependency that is not yet built, Shake suspends the build action and builds the dependency first. Together, these features make Shake flexible and powerful. Read the detailed and accessible paper <a href="https://www.microsoft.com/en-us/research/uploads/prod/2020/04/build-systems-jfp.pdf" target="_blank" rel="noopener">Build systems à la carte: Theory and practice</a> for a comparison of Shake with other build systems.<a href="#fnref3" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn4"><p>To run the generator directly without writing a separate file for dependency management, we can prepend one of these three <a href="https://en.wikipedia.org/wiki/Shebang_(Unix)" target="_blank" rel="noopener">Shebang</a> snippets to <code>Site.hs</code>.</p>
<div id="lst:nix-shell-shebang" class="listing numberSource bash">
<div class="sourceCode" id="cb4" data-lang="bash"><pre class="sourceCode numberSource bash"><code class="sourceCode bash"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="co">#! /usr/bin/env nix-shell</span></span>
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a><span class="co">#! nix-shell -p "haskellPackages.ghcWithPackages (p: [p.mustache p.pandoc p.shake p.deriving-aeson])"</span></span>
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a><span class="co">#! nix-shell -i runhaskell</span></span></code></pre></div>
<p><a href="http://iam.travishartwell.net/2015/06/17/nix-shell-shebang/" target="_blank" rel="noopener">Nix shell</a> shebang snippet</p>
</div>
<div id="lst:cabal-shebang" class="listing numberSource bash">
<div class="sourceCode" id="cb5" data-lang="bash"><pre class="sourceCode numberSource bash"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="co">#! /usr/bin/env cabal</span></span>
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a><span class="ex">{-</span> cabal:</span>
<span id="cb5-3"><a href="#cb5-3" aria-hidden="true" tabindex="-1"></a><span class="ex">build-depends:</span> base, aeson, deriving-aeson, mustache, pandoc, shake, text, time, unordered-containers</span>
<span id="cb5-4"><a href="#cb5-4" aria-hidden="true" tabindex="-1"></a><span class="ex">-}</span></span></code></pre></div>
<p><a href="https://cabal.readthedocs.io/en/latest/cabal-commands.html#cabal-run" target="_blank" rel="noopener">Cabal</a> shebang snippet</p>
</div>
<div id="lst:stack-shebang" class="listing numberSource bash">
<div class="sourceCode" id="cb6" data-lang="bash"><pre class="sourceCode numberSource bash"><code class="sourceCode bash"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="co">#! /usr/bin/env stack</span></span>
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a><span class="ex">{-</span> stack script</span>
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a> <span class="ex">--resolver</span> lts-19.28</span>
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a> <span class="ex">--package</span> <span class="st">"base aeson deriving-aeson mustache pandoc shake text time unordered-containers"</span></span>
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a><span class="ex">-}</span></span></code></pre></div>
<p><a href="https://docs.haskellstack.org/en/latest/scripts/" target="_blank" rel="noopener">Stack</a> shebang snippet</p>
</div>
<p>We need to have the corresponding toolchain (Nix, Cabal or Stack) installed to run the generator. The snippets take care of downloading and/or building the dependencies, and running the generator.<a href="#fnref4" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn5"><p>Since tag page generation is a bit more involved, we have faded out the related code for now. We come back to it a later section.<a href="#fnref5" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn6"><p>We use the <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:putInfo" target="_blank" rel="noopener"><code class="sourceCode haskell">Shake.putInfo</code></a> function to print a message to the console. There also exist <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:putWarn" target="_blank" rel="noopener"><code class="sourceCode haskell">putWarn</code></a> and <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:putError" target="_blank" rel="noopener"><code class="sourceCode haskell">putError</code></a> functions for printing warnings and errors respectively.<a href="#fnref6" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn7"><p>If you are familiar with other templating languages like <a href="https://github.com/Shopify/liquid" target="_blank" rel="noopener">Liquid</a>, and are wondering why we are limiting the post count in the Haskell code, and not in the Mustache template, it is because Mustache is a logic-less template engine. It does not have any control flow constructs except check for null values. Hence, we have to do the limiting in the Haskell code.<a href="#fnref7" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn8"><p>If the code accumulates a lot of config options like post count on home page, we can move them to an external JSON/YAML/TOML config file, and read them at the start of the build script. We can wrap the <code class="sourceCode haskell"><span class="dt">Rules</span></code> monad in a <a href="https://hackage.haskell.org/package/transformers/docs/Control-Monad-Trans-Reader.html#t:ReaderT" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">ReaderT</span></code></a> monad transformer to make the config options available to all build rules.<a href="#fnref8" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn9"><p>We use the <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake.html#v:traced" target="_blank" rel="noopener"><code>Shake.traced</code></a> function to trace the operations in build actions. It logs the operations to the console, and also records them in traces and reports. See the <a href="#traces-and-reports">Traces and Reports</a> section for more details.<a href="#fnref9" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn10"><p>The full code for the SSG is available <a href="https://abhinavsarkar.net/code/shake-blog.html?mtm_campaign=feed">here</a>.<a href="#fnref10" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn11"><p>When we run the build for the first time, it takes some time to download and/or build the dependencies. Subsequent builds are much faster.<a href="#fnref11" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn12"><p>You can find the code for the <abbr title="Static site generator">SSG</abbr> with caching <a href="https://abhinavsarkar.net/code/shake-blog-with-caching.html?mtm_campaign=feed">here</a>.<a href="#fnref12" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn13"><p>If we use the experimental forward build feature, it’s easier to cache the output of build actions using one of the <code>cache*</code> functions in the <a href="https://hackage.haskell.org/package/shake-0.19.7/docs/Development-Shake-Forward.html" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">Development.Shake.Forward</span></code></a> module. However, forward builds require <a href="https://github.com/jacereda/fsatrace" target="_blank" rel="noopener"><code>fsatrace</code></a> to be installed on the system, and it doesn’t work on macOS with <a href="https://support.apple.com/en-us/HT204899" target="_blank" rel="noopener">System Integrity Protection</a> enabled.<a href="#fnref13" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn14"><p>That’s right, <a href="https://abhinavsarkar.net?mtm_campaign=feed" target="_blank" rel="noopener">abhinavsarkar.net</a> is also built using Shake! See <a href="https://abhinavsarkar.net/about/?mtm_campaign=feed#about-the-website">About the Website</a> section for more details. It used to be built using <a href="https://jaspervdj.be/hakyll/" target="_blank" rel="noopener">Hakyll</a>, but I switched to Shake after getting frustrated with the opaqueness of Hakyll’s build system, and those pesky <a href="https://hackage.haskell.org/package/hakyll-4.15.1.1/docs/Hakyll-Web-Template-Context.html" target="_blank" rel="noopener">contexts</a>.<a href="#fnref14" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn15"><p>Shake also supports the <a href="https://downloads.haskell.org/ghc/latest/docs/users_guide/exts/applicative_do.html" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">ApplicativeDo</span></code></a> extension, enabling which causes the compiler to automatically detect the build actions that can be run in parallel. However, it may not detect all cases. Regardless, it is better to enable it to improve the parallelism of the build.<a href="#fnref15" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn16"><p>Don’t forget to add a signal handler to stop the watcher and server threads when the build script is interrupted.<a href="#fnref16" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn17"><p>See how Rib does watch and serve <a href="https://github.com/srid/rib/tree/master/rib-core/src/Rib" target="_blank" rel="noopener">here</a>.<a href="#fnref17" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
</ol>
</section><p>If you liked this post, please <a href="https://abhinavsarkar.net/posts/static-site-generator-using-shake/?mtm_campaign=feed#syndications">leave a comment</a>.</p><img referrerpolicy="no-referrer-when-downgrade" src="https://anna.abhinavsarkar.net/matomo.php?idsite=1&rec=1" style="border:0" alt="" />2022-12-17T00:00:00Z<p>Static site generators (SSGs) are all rage these days as people realize that plain HTML websites are good enough for most cases. <abbr title="Static site generator">SSGs</abbr> take raw data in various formats—often <a href="https://en.wikipedia.org/wiki/Markdown" target="_blank" rel="noopener">Markdown</a>, <a href="https://en.wikipedia.org/wiki/JSON" target="_blank" rel="noopener">JSON</a>, and <a href="https://en.wikipedia.org/wiki/YAML" target="_blank" rel="noopener">YAML</a>—and process them to produce the static websites, which can then be hosted easily on any hosting provider, or on personal <abbr title="Virtual private server">VPSes</abbr>. In this post, we write a bespoke <abbr title="Static site generator">SSG</abbr> using the <a href="https://shakebuild.com/" target="_blank" rel="noopener">Shake</a> build system.</p>
https://abhinavsarkar.net/posts/implementing-co-2/Implementing Co, a Small Language With Coroutines #2: The Interpreter2021-09-21T00:00:00ZAbhinav Sarkarhttps://abhinavsarkar.net/about/abhinav@abhinavsarkar.net<p>In the <a href="https://abhinavsarkar.net/posts/implementing-co-1/?mtm_campaign=feed">previous post</a>, we wrote the parser for <span class="fancy">Co</span>, the small language we are building in this series of posts. The previous post was all about the syntax of <span class="fancy">Co</span>. In this post we dive into the semantics of <span class="fancy">Co</span>, and write an interpreter for its basic features.</p>
<p>This post was originally published on <a href="https://abhinavsarkar.net/posts/implementing-co-2/?mtm_campaign=feed">abhinavsarkar.net</a>.</p><!--more-->
<p>This is the second post in a series of posts:</p>
<ol type="1">
<li><a href="https://abhinavsarkar.net/posts/implementing-co-1/?mtm_campaign=feed">Implementing Co #1: The Parser</a></li>
<li>Implementing Co #2: The Interpreter</li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-3/?mtm_campaign=feed">Implementing Co #3: Adding Coroutines</a></li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-4/?mtm_campaign=feed">Implementing Co #4: Adding Channels</a></li>
<li>Implementing Co #5: Adding Sleep</li>
</ol>
<nav id="toc" class="right-toc"><h3>Contents</h3><ol><li><a href="#previously-on">Previously, on …</a></li><li><a href="#running-a-program">Running a Program</a></li><li><a href="#runtime-values">Runtime Values</a></li><li><a href="#environment-model-of-evaluation">Environment Model of Evaluation</a></li><li><a href="#scopes">Scopes</a></li><li><a href="#closures">Closures</a></li><li><a href="#early-returns">Early Returns</a></li><li><a href="#the-interpreter">The Interpreter</a></li><li><a href="#manipulating-environments">Manipulating Environments</a></li><li><a href="#evaluating-expressions">Evaluating Expressions</a></li><li><a href="#executing-statements">Executing Statements</a></li><li><a href="#evaluating-function-calls">Evaluating Function Calls</a></li><li><a href="#interpreting-a-program">Interpreting a Program</a></li></ol></nav>
<h2 data-track-content data-content-name="previously-on" data-content-piece="implementing-co-2" id="previously-on">Previously, on …</h2>
<p>Here’s a quick recap. The basic features of <span class="fancy">Co</span> that we are aiming to implement in this post are:</p>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Dynamic_typing" target="_blank" rel="noopener">Dynamic</a> and <a href="https://en.wikipedia.org/wiki/Strong_typing" target="_blank" rel="noopener">strong</a> typing.</li>
<li>Null, boolean, string and integer literals, and values.</li>
<li>Addition, subtraction, multiplication and integer division arithmetic operations.</li>
<li>String concatenation operation.</li>
<li>Equality and inequality checks on booleans, strings and numbers.</li>
<li>Less-than and greater-than comparison operations on numbers.</li>
<li>Variable declarations, usage and assignments.</li>
<li><code class="sourceCode javascript"><span class="cf">if</span></code> and <code class="sourceCode javascript"><span class="cf">while</span></code> statements.</li>
<li>Function declarations and calls, with support for recursion.</li>
<li>First class functions and anonymous functions.</li>
<li>Mutable closures.</li>
</ul>
<div class="note">
<p>Note that some parts of code snippets in this post have been faded away. These are the part which add support for coroutines and channels. You can safely ignore these parts for now. We’ll go over them in the next post.</p>
</div>
<p>We represent the <span class="fancy">Co</span> <a href="https://en.wikipedia.org/wiki/Abstract_Syntax_Tree" target="_blank" rel="noopener"><em>Abstract Syntax Tree</em></a> (AST) as a pair of Haskell <a href="https://en.wikipedia.org/wiki/Algebraic_data_type" target="_blank" rel="noopener">Algebraic Data Types</a> (ADTs), one for <a href="https://en.wikipedia.org/wiki/Expression_(computer_science)" target="_blank" rel="noopener"><em>Expressions</em></a>:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="10-10"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Expr</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">LNull</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LBool</span> <span class="dt">Bool</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LStr</span> <span class="dt">String</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LNum</span> <span class="dt">Integer</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Variable</span> <span class="dt">Identifier</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Binary</span> <span class="dt">BinOp</span> <span class="dt">Expr</span> <span class="dt">Expr</span></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Call</span> <span class="dt">Expr</span> [<span class="dt">Expr</span>]</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Lambda</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>]</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op">|</span> <span class="dt">Receive</span> <span class="dt">Expr</span></span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Eq</span>)</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Identifier</span> <span class="ot">=</span> <span class="dt">String</span></span></code></pre></div>
<p>And another for <a href="https://en.wikipedia.org/wiki/Statement_(computer_science)" target="_blank" rel="noopener"><em>Statements</em></a>:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="9-11"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Stmt</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">ExprStmt</span> <span class="dt">Expr</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">VarStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">AssignStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">IfStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">WhileStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">FunctionStmt</span> <span class="dt">Identifier</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>]</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">ReturnStmt</span> (<span class="dt">Maybe</span> <span class="dt">Expr</span>)</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op">|</span> <span class="dt">YieldStmt</span></span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op">|</span> <span class="dt">SpawnStmt</span> <span class="dt">Expr</span></span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op">|</span> <span class="dt">SendStmt</span> <span class="dt">Expr</span> <span class="dt">Expr</span></span></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Eq</span>)</span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Program</span> <span class="ot">=</span> [<span class="dt">Stmt</span>]</span></code></pre></div>
<p>Also, <code>program</code> is the parser for <span class="fancy">Co</span> programs. To parse code, run the <code>program</code> parser with the <code>runParser</code> function like this:</p>
<div class="sourceCode" id="cb1" data-lang="ghci"><pre class="sourceCode lhs numberSource"><code class="sourceCode literatehaskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> runParser program <span class="st">"var x = 1 + s;"</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>Right [VarStmt "x" (Binary Plus (LNum 1) (Variable "s"))]</span></code></pre></div>
<p>Now, off to the new stuff.</p>
<h2 data-track-content data-content-name="running-a-program" data-content-piece="implementing-co-2" id="running-a-program">Running a Program</h2>
<p>There are many ways to run a program. If the program is written in <a href="https://en.wikipedia.org/wiki/Machine_Code" target="_blank" rel="noopener"><em>Machine Code</em></a>, you can run it directly on the matching <a href="https://en.wikipedia.org/wiki/CPU" target="_blank" rel="noopener">CPU</a>. But machine code is too <a href="https://en.wikipedia.org/wiki/Low-level_programming_language" target="_blank" rel="noopener">low-level</a>, and writing programs in it is very tedious and error-prone. Thus, programmers prefer to write code in <a href="https://en.wikipedia.org/wiki/high-level_programming_languages" target="_blank" rel="noopener">high-level programming languages</a>, and turn it into machine code to be able to run it<sup><a href="#ref-Abelson1996-c4" class="citation" title="Abelson, Sussman, and with Julie Sussman, “Metalinguistic Abstraction.”
">@1</a></sup>. Here’s where different ways of running code come in:</p>
<ul>
<li>We can run the high-level code through a <a href="https://en.wikipedia.org/wiki/Compiler" target="_blank" rel="noopener"><em>Compiler</em></a> to turn it into machine code to be able to run it directly. Example: compiling <a href="https://en.wikipedia.org/wiki/C++" target="_blank" rel="noopener">C++</a> using <a href="https://en.wikipedia.org/wiki/GNU_Compiler_Collection" target="_blank" rel="noopener">GCC</a>.</li>
<li>We can run the code through a compiler which turns it into a relatively lower-level programming language code, and then run that lower-level program through another compiler to turn it into machine code. Example: compiling <a href="https://haskell.org" target="_blank" rel="noopener">Haskell</a> into <a href="https://llvm.org/" target="_blank" rel="noopener">LLVM IR</a> using <a href="https://www.haskell.org/ghc/" target="_blank" rel="noopener">GHC</a>, which can then be run through the LLVM toolchain to generate machine code.</li>
<li>We can run the code through a <a href="https://en.wikipedia.org/wiki/Transpiler" target="_blank" rel="noopener"><em>Transpiler</em></a> (also called <em>Source-to-source compiler</em>) to turn it into code in a programming language that is of similar level, and then run the resultant code with that language’s toolchain. Example: transpiling <a href="https://www.purescript.org/" target="_blank" rel="noopener">Purescript</a> into <a href="https://en.wikipedia.org/wiki/JavaScript" target="_blank" rel="noopener">JavaScript</a>, and running it with <a href="https://nodejs.org/" target="_blank" rel="noopener">node.js</a>.</li>
<li>We can compile the source code to <a href="https://en.wikipedia.org/wiki/Bytecode" target="_blank" rel="noopener"><em>Bytecode</em></a> and run the bytecode on a <a href="https://en.wikipedia.org/wiki/Virtual_Machine" target="_blank" rel="noopener"><em>Virtual Machine</em></a>. Example: <a href="https://en.wikipedia.org/wiki/Java_virtual_machine" target="_blank" rel="noopener">Java virtual machine</a> running <a href="https://en.wikipedia.org/wiki/Java" target="_blank" rel="noopener">Java</a> bytecode compiled from <a href="https://clojure.org/" target="_blank" rel="noopener">Clojure</a> source code by the Clojure compiler.</li>
<li>We can parse the code to an AST, and immediately execute the AST using an <a href="https://en.wikipedia.org/wiki/Interpreter_(computing)#Abstract_syntax_tree_interpreters" target="_blank" rel="noopener"><em>AST Interpreter</em></a>. Example: <a href="https://www.php.net/" target="_blank" rel="noopener">PHP</a> version 3, <a href="https://www.gnu.org/software/bash/" target="_blank" rel="noopener">Bash</a>. <a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a></li>
<li>We can also mix-and-match parts of the above options to create hybrids, like <a href="https://en.wikipedia.org/wiki/Just-in-time_compilation" target="_blank" rel="noopener"><em>Just-in-time compilation</em></a> to machine code within a virtual machine.</li>
</ul>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 641 497'%3E%3C/svg%3E" class="lazyload ascii-art nolink" style="--image-aspect-ratio: 1.289738430583501" data-src="/images/implementing-co-2/source-to-machine.svg" alt="Many ways to run a program"></img>
<noscript><img src="/images/implementing-co-2/source-to-machine.svg" class="ascii-art nolink" alt="Many ways to run a program"></img></noscript>
<figcaption>Many ways to run a program</figcaption>
</figure>
<p>For running <span class="fancy">Co</span> programs, we will implement an AST-walking interpreter. The interpreter implemented in this post will support only the <a href="#previously-on">basic features</a> of <span class="fancy">Co</span>. In the next post, we’ll extend it to support coroutines and channels.</p>
<div class="note">
<p>The complete code for the interpreter is <a href="https://abhinavsarkar.net/code/co-interpreter.html?mtm_campaign=feed">here</a>. You can load it in GHCi using <a href="https://en.wikipedia.org/wiki/Low-level_programming_language" target="_blank" rel="noopener">stack</a> (by running <code>stack co-interpreter.hs</code>), and follow along while reading this article.</p>
</div>
<h2 data-track-content data-content-name="runtime-values" data-content-piece="implementing-co-2" id="runtime-values">Runtime Values</h2>
<p>An AST-walking interpreter takes an AST as its input, and recursively walks down the AST nodes, from top to bottom. While doing this, it evaluates expressions to runtime values, and executes the statements to do their effects.</p>
<p>The runtime values are things that can be passed around in the code during the program run time. Often called “first-class”, these values can be assigned to variables, passed as function arguments, and returned from functions. If <span class="fancy">Co</span> were to support data structures like lists and maps, these values could be stored in them as well. The <code class="sourceCode haskell"><span class="dt">Value</span></code> ADT below represents these values:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="8-8"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Value</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">Null</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Boolean</span> <span class="dt">Bool</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Str</span> <span class="dt">String</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Num</span> <span class="dt">Integer</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Function</span> <span class="dt">Identifier</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>] <span class="dt">Env</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">BuiltinFunction</span> <span class="dt">Identifier</span> <span class="dt">Int</span> ([<span class="dt">Expr</span>] <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span>)</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op">|</span> <span class="dt">Chan</span> <span class="dt">Channel</span></span></span></code></pre></div>
<p>Other than the usual values like <code class="sourceCode javascript"><span class="kw">null</span></code>, booleans, strings, and numbers, we also have functions as first-class runtime values in <span class="fancy">Co</span>. We have a constructor <code class="sourceCode haskell"><span class="dt">Function</span></code> for the functions that programmers define in their <span class="fancy">Co</span> code, and another constructor <code class="sourceCode haskell"><span class="dt">BuiltinFunction</span></code> for built-in functions like <code>print</code><a href="#fn2" class="footnote-ref" id="fnref2" role="doc-noteref"><sup>2</sup></a>.</p>
<p>We also write instances to show and check equality for these values:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="9-9"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">instance</span> <span class="dt">Show</span> <span class="dt">Value</span> <span class="kw">where</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">show</span> <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">Null</span> <span class="ot">-></span> <span class="st">"null"</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">Boolean</span> b <span class="ot">-></span> <span class="fu">show</span> b</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">Str</span> s <span class="ot">-></span> s</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">Num</span> n <span class="ot">-></span> <span class="fu">show</span> n</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">Function</span> name _ _ _ <span class="ot">-></span> <span class="st">"function "</span> <span class="op"><></span> name</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">BuiltinFunction</span> name _ _ <span class="ot">-></span> <span class="st">"function "</span> <span class="op"><></span> name</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">Chan</span> <span class="dt">Channel</span> {} <span class="ot">-></span> <span class="st">"Channel"</span></span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a><span class="kw">instance</span> <span class="dt">Eq</span> <span class="dt">Value</span> <span class="kw">where</span></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">Null</span> <span class="op">==</span> <span class="dt">Null</span> <span class="ot">=</span> <span class="dt">True</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">Boolean</span> b1 <span class="op">==</span> <span class="dt">Boolean</span> b2 <span class="ot">=</span> b1 <span class="op">==</span> b2</span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">Str</span> s1 <span class="op">==</span> <span class="dt">Str</span> s2 <span class="ot">=</span> s1 <span class="op">==</span> s2</span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">Num</span> n1 <span class="op">==</span> <span class="dt">Num</span> n2 <span class="ot">=</span> n1 <span class="op">==</span> n2</span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a> _ <span class="op">==</span> _ <span class="ot">=</span> <span class="dt">False</span></span></code></pre></div>
<p>Note that only <code class="sourceCode javascript"><span class="kw">null</span></code>, booleans, strings and numbers can be checked for equality in <span class="fancy">Co</span>. Also, only values of same type can be equals. A string can never be equal to a number<a href="#fn3" class="footnote-ref" id="fnref3" role="doc-noteref"><sup>3</sup></a>.</p>
<p>So, how do we go about turning the expressions to values, and executing statements? Before learning that, we must take a detour into some theory of programming languages.</p>
<div class="note">
<p>Readers familiar with the concepts of environments, scopes, closures and early returns can skip the next sections, and jump directly to the <a href="#the-interpreter">implementation</a>.</p>
</div>
<h2 data-track-content data-content-name="environment-model-of-evaluation" data-content-piece="implementing-co-2" id="environment-model-of-evaluation">Environment Model of Evaluation</h2>
<p>Let’s say we have this little <span class="fancy">Co</span> program to run:</p>
<div class="sourceCode" id="cb2" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> a <span class="op">=</span> <span class="dv">2</span><span class="op">;</span></span>
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">twice</span>(x) { <span class="cf">return</span> x <span class="op">+</span> x<span class="op">;</span> }</span>
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="fu">twice</span>(a))<span class="op">;</span></span></code></pre></div>
<p>We need to evaluate <code class="sourceCode javascript"><span class="fu">twice</span>(a)</code> to a value to print it. One way to do that is to substitute variables for their values, quite literally. <code>twice</code> is a variable, value of which is a function. And <code>a</code> is another variable, with value <code class="sourceCode javascript"><span class="dv">2</span></code>. We can do repeated substitution to arrive at a resultant value like this:</p>
<div class="sourceCode" id="cb3" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="fu">twice</span>(a))<span class="op">;</span></span>
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a><span class="kw">=></span> <span class="fu">print</span>(<span class="fu">twice</span>(<span class="dv">2</span>))<span class="op">;</span></span>
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a><span class="kw">=></span> <span class="fu">print</span>(<span class="dv">2</span> <span class="op">+</span> <span class="dv">2</span>)<span class="op">;</span></span>
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a><span class="kw">=></span> <span class="fu">print</span>(<span class="dv">4</span>)<span class="op">;</span></span></code></pre></div>
<p>This is called the <a href="https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-10.html#%_sec_1.1.5" target="_blank" rel="noopener"><em>Substitution model of evaluation</em></a><sup><a href="#ref-Abelson1996-c115" class="citation" title="Abelson, Sussman, and with Julie Sussman, “The Substitution Model for
Procedure Application.”
">@5</a></sup>. This works for the example we have above, and for a large set of programs<a href="#fn4" class="footnote-ref" id="fnref4" role="doc-noteref"><sup>4</sup></a>. However, it breaks down as soon as we add mutability to the mix:</p>
<div class="sourceCode" id="cb4" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> a <span class="op">=</span> <span class="dv">2</span><span class="op">;</span></span>
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">incA</span>() {</span>
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> b <span class="op">=</span> a <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> b<span class="op">;</span></span>
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="fu">incA</span>())<span class="op">;</span></span>
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a>a <span class="op">=</span> <span class="dv">3</span><span class="op">;</span></span>
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="fu">incA</span>())<span class="op">;</span></span></code></pre></div>
<p>Running this with the <span class="fancy">Co</span> interpreter results in the output:</p>
<pre class="plain"><code>3
4</code></pre>
<p>We can’t use the substitution model here because we can’t consider variables like <code>a</code> to be substitutable with single values anymore. Now, we must think of them more as places in which the values are stored. Also, the stored values may change over the lifetime of the program execution. We call this place where the variable values are stored, the <em>Environment</em>, and this understanding of program execution is called the <a href="https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-21.html#%_sec_3.2" target="_blank" rel="noopener"><em>Environment Model of Evaluation</em></a><sup><a href="#ref-Abelson1996-c32" class="citation" title="Abelson, Sussman, and with Julie Sussman, “The Environment Model of
Evaluation.”
">@7</a></sup>.</p>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 497 65'%3E%3C/svg%3E" class="lazyload ascii-art w-80pct nolink" style="--image-aspect-ratio: 7.6461538461538465" data-src="/images/implementing-co-2/env-model.svg" alt="Value of a variable may change over time"></img>
<noscript><img src="/images/implementing-co-2/env-model.svg" class="ascii-art w-80pct nolink" alt="Value of a variable may change over time"></img></noscript>
<figcaption>Value of a variable may change over time</figcaption>
</figure>
<p>A pair of a variable’s name and its value at any particular time is called a <em>Binding</em>. An <em>Environment</em> is a collection of zero-or-more bindings. To fully understand environments, first we have to learn about scopes.</p>
<h2 data-track-content data-content-name="scopes" data-content-piece="implementing-co-2" id="scopes">Scopes</h2>
<p>Let’s consider the <code>twice</code> function again:</p>
<div class="sourceCode" id="cb6" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">twice</span>(x) { <span class="cf">return</span> x <span class="op">+</span> x<span class="op">;</span> }</span>
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="fu">twice</span>(<span class="dv">1</span>))<span class="op">;</span></span>
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="fu">twice</span>(<span class="dv">2</span>))<span class="op">;</span></span></code></pre></div>
<p>Calling <code>twice</code> with different arguments prints different results. The function seems to forget the value of its parameter <code>x</code> after each call. This may feel very natural to programmers, but how does it really work? The answer is <a href="https://en.wikipedia.org/wiki/Scope_(computer_science)" target="_blank" rel="noopener"><em>Scopes</em></a>.</p>
<p>A scope is a region of the program lifetime during which a variable name-to-value binding is in effect. When the program execution enters a scope, the variables in that scope become defined and available to the executing code<a href="#fn5" class="footnote-ref" id="fnref5" role="doc-noteref"><sup>5</sup></a>. When the program execution exits the scope, the variables become undefined and inaccessible (also known as “going out of scope”).</p>
<p><a href="https://en.wikipedia.org/wiki/Lexical_scoping" target="_blank" rel="noopener"><em>Lexical scoping</em></a> is a specific style of scoping where the structure of the program itself shows where a scope begins and ends<sup><a href="#ref-Abelson1996-c118" class="citation" title="Abelson, Sussman, and with Julie Sussman, “Procedures as Black-Box
Abstractions.”
">@9</a></sup>. Like most modern languages, <span class="fancy">Co</span> is lexically scoped. A function in <span class="fancy">Co</span> starts a new scope which extends over the entire function body, and the scope ends when the function ends<a href="#fn6" class="footnote-ref" id="fnref6" role="doc-noteref"><sup>6</sup></a>. Functions are the only way of creating new scopes in <span class="fancy">Co</span><a href="#fn7" class="footnote-ref" id="fnref7" role="doc-noteref"><sup>7</sup></a>.</p>
<p>That’s how repeated invocation of functions don’t remember the values of their parameters across the calls. Every time a new call is started, a new scope is created with the parameter names bound to the value of the arguments of the call. And when the call returns, this new scope is destroyed.</p>
<p>Scopes can be enclosed within other scopes. In <span class="fancy">Co</span>, this can be done by defining a function inside the body of another function. All programs have at least one scope, which is the program’s top-level scope, often called the global scope.</p>
<p>Scopes are intimately related to the environment. In fact, the structure of the environment is how scopes are implemented<sup><a href="#ref-Abelson1996-c556" class="citation" title="Abelson, Sussman, and with Julie Sussman, “Lexical Addressing.”
">@12</a></sup>.</p>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 377 193'%3E%3C/svg%3E" class="lazyload ascii-art w-70pct nolink" style="--image-aspect-ratio: 1.9533678756476685" data-src="/images/implementing-co-2/twice-scopes.svg" alt="Scopes are implemented by the environment"></img>
<noscript><img src="/images/implementing-co-2/twice-scopes.svg" class="ascii-art w-70pct nolink" alt="Scopes are implemented by the environment"></img></noscript>
<figcaption>Scopes are implemented by the environment</figcaption>
</figure>
<p>An environment can be thought of as a stack of frames, with one frame per enclosed scope<sup><a href="#ref-Abelson1996-c32" class="citation" title="Abelson, Sussman, and with Julie Sussman, “The Environment Model of
Evaluation.”
">@13</a></sup>. A frame contains zero-or-more bindings. The bindings in enclosed scopes (frames higher in the environment stack) hide the bindings (called <a href="https://en.wikipedia.org/wiki/Variable_shadowing" target="_blank" rel="noopener">shadowing</a>) in enclosing scopes (frames lower in the environment stack). Program’s global scope is represented by the lowermost frame in the stack.</p>
<p>The above diagram shows the frames of the two calls to the <code>twice</code> function. The scope of the <code>twice</code> function is enclosed in the global scope. To find the value of a variable inside the function, the interpret first looks into the topmost frame that represents the scope of the <code>twice</code> function. If the binding is not found, then the interpreter goes down the stack of frames, and looks into the frame for the global scope.</p>
<p>What happens when a function body tries to access variables not defined in the function’s scope? We get <em>Closures</em>.</p>
<h2 data-track-content data-content-name="closures" data-content-piece="implementing-co-2" id="closures">Closures</h2>
<p>If a function body refers to variables not defined in the function’s scope, such variables are called <a href="https://en.wikipedia.org/wiki/Free_variables_and_bound_variables" target="_blank" rel="noopener"><em>Free Variables</em></a><sup><a href="#ref-Abelson1996-c118" class="citation" title="Abelson, Sussman, and with Julie Sussman, “Procedures as Black-Box
Abstractions.”
">@14</a></sup>. In lexically scoped languages, the value of a free variable is determined from the scope in which the function is defined. A function along with the references to all its free variables, is called a <em><a href="https://en.wikipedia.org/wiki/Closure_(computer_programming)" target="_blank" rel="noopener">Closure</a></em><a href="#fn8" class="footnote-ref" id="fnref8" role="doc-noteref"><sup>8</sup></a>.</p>
<p>Closures are prevalent in programming languages with first-class functions. <span class="fancy">Co</span>—with its support for first-class functions—also supports closures. Closures in <span class="fancy">Co</span> are mutable, meaning the values of the free variables of a function can change over time, and the changes are reflected in the behavior of the function<a href="#fn9" class="footnote-ref" id="fnref9" role="doc-noteref"><sup>9</sup></a>.</p>
<p>We already saw an example of closures earlier:</p>
<div class="sourceCode" id="cb7" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> a <span class="op">=</span> <span class="dv">2</span><span class="op">;</span></span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">incA</span>() {</span>
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> b <span class="op">=</span> a <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> b<span class="op">;</span></span>
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="fu">incA</span>())<span class="op">;</span></span>
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a>a <span class="op">=</span> <span class="dv">3</span><span class="op">;</span></span>
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="fu">incA</span>())<span class="op">;</span></span></code></pre></div>
<p>This is how the frames exist over time for the two invocations of the <code>incA</code> function:</p>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 449 209'%3E%3C/svg%3E" class="lazyload ascii-art w-80pct nolink" style="--image-aspect-ratio: 2.148325358851675" data-src="/images/implementing-co-2/incA-scopes.svg" alt="a is a free variable of the function incA"></img>
<noscript><img src="/images/implementing-co-2/incA-scopes.svg" class="ascii-art w-80pct nolink" alt="a is a free variable of the function incA"></img></noscript>
<figcaption><code>a</code> is a free variable of the function <code>incA</code></figcaption>
</figure>
<p>Here, <code>a</code> is a free variable of the function <code>incA</code>. Its value is not present in the scope of <code>incA</code>, but is obtained from the global scope. When its value in the global scope changes later, the value returned by <code>incA</code> changes as well. In other words, <code>incA</code> and <code>a</code> together form a closure.</p>
<p>The following example demonstrates a closure with a mutable free variable and enclosed scopes:</p>
<div class="sourceCode" id="cb8" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">makeCounter</span>(name) {</span>
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> count <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="kw">function</span> () {</span>
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a> count <span class="op">=</span> count <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb8-5"><a href="#cb8-5" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" = "</span> <span class="op">+</span> count)<span class="op">;</span></span>
<span id="cb8-6"><a href="#cb8-6" aria-hidden="true" tabindex="-1"></a> }<span class="op">;</span></span>
<span id="cb8-7"><a href="#cb8-7" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb8-8"><a href="#cb8-8" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb8-9"><a href="#cb8-9" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> countA <span class="op">=</span> <span class="fu">makeCounter</span>(<span class="st">"a"</span>)<span class="op">;</span></span>
<span id="cb8-10"><a href="#cb8-10" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> countB <span class="op">=</span> <span class="fu">makeCounter</span>(<span class="st">"b"</span>)<span class="op">;</span></span>
<span id="cb8-11"><a href="#cb8-11" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb8-12"><a href="#cb8-12" aria-hidden="true" tabindex="-1"></a><span class="fu">countA</span>()<span class="op">;</span></span>
<span id="cb8-13"><a href="#cb8-13" aria-hidden="true" tabindex="-1"></a><span class="fu">countA</span>()<span class="op">;</span></span>
<span id="cb8-14"><a href="#cb8-14" aria-hidden="true" tabindex="-1"></a><span class="fu">countB</span>()<span class="op">;</span></span>
<span id="cb8-15"><a href="#cb8-15" aria-hidden="true" tabindex="-1"></a><span class="fu">countA</span>()<span class="op">;</span></span></code></pre></div>
<p>Here, both <code>name</code> and <code>count</code> are free variables referred in the returned function. While <code>name</code> is only read, <code>count</code> is changed in the body of the function.</p>
<p>Running the above code prints:</p>
<pre class="plain"><code>a = 1
a = 2
b = 1
a = 3</code></pre>
<p>Note that the two functions <code>countA</code> and <code>countB</code> refer to two different instances of the <code>count</code> variable, and are not affected by each other. In other words, <code>countA</code> and <code>countB</code> are two different closures for the same function.</p>
<p>Now for one last thing to learn about before we jump to the implementation: early returns.</p>
<h2 data-track-content data-content-name="early-returns" data-content-piece="implementing-co-2" id="early-returns">Early Returns</h2>
<p>Statement oriented programming languages often allow returning from a function before the entire function is done executing. This is called an <a href="https://en.wikipedia.org/wiki/Return_statement#Multiple_return_statements" target="_blank" rel="noopener"><em>Early return</em></a>. We saw an example of this in the fibonacci function in the previous post:</p>
<div class="sourceCode" id="cb10" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">fib</span>(n) {</span>
<span id="cb10-2"><a href="#cb10-2" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (n <span class="op"><</span> <span class="dv">2</span>) {</span>
<span id="cb10-3"><a href="#cb10-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> n<span class="op">;</span></span>
<span id="cb10-4"><a href="#cb10-4" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb10-5"><a href="#cb10-5" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="fu">fib</span>(n <span class="op">-</span> <span class="dv">2</span>)</span>
<span id="cb10-6"><a href="#cb10-6" aria-hidden="true" tabindex="-1"></a> <span class="op">+</span> <span class="fu">fib</span>(n <span class="op">-</span> <span class="dv">1</span>)<span class="op">;</span></span>
<span id="cb10-7"><a href="#cb10-7" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
<p>In the above code, when the input <code>n</code> is less than 2, the code returns early from the function at the line 3.</p>
<p>Expression oriented programming languages, like Haskell, have no early returns. Every function is an expression in Haskell, and has to be evaluated entirely<a href="#fn10" class="footnote-ref" id="fnref10" role="doc-noteref"><sup>10</sup></a> to get back a value. Since our AST-walking interpreter itself is written in Haskell, we need to figure out how to support early returns in the <span class="fancy">Co</span> code being interpreted. The interpreter should be able to stop evaluating at an AST node representing a <code class="sourceCode javascript"><span class="cf">return</span></code> statement, and jump to the node representing the function’s caller.</p>
<p>One way to implement this is <a href="https://en.wikipedia.org/wiki/Exception_(computer_science)" target="_blank" rel="noopener"><em>Exceptions</em></a>. Exceptions let us abort the execution of code at any point of execution, and resume from some other point in the lower in the function call stack. Although Haskell <a href="https://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Exception.html" target="_blank" rel="noopener">supports</a> exceptions as we know them from languages like Java and Python, it also supports exceptions as values using the <em><a href="https://hackage.haskell.org/package/mtl-2.2.2/docs/Control-Monad-Except.html#t:MonadError" target="_blank" rel="noopener">Error monad</a></em>. That’s what we will leverage for implementing early returns in our interpreter.</p>
<p>Finally, we are really to start implementing the interpreter.</p>
<h2 data-track-content data-content-name="the-interpreter" data-content-piece="implementing-co-2" id="the-interpreter">The Interpreter</h2>
<p>The interpreter is implemented as a Haskell <code class="sourceCode haskell"><span class="kw">newtype</span></code> over a stack of monad using the monad transformers and typeclasses from the <a href="https://hackage.haskell.org/package/mtl" target="_blank" rel="noopener">mtl</a> library:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="4-5,6:41-6:42,16:27-16:28,17-17"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">newtype</span> <span class="dt">Interpreter</span> a <span class="ot">=</span> <span class="dt">Interpreter</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> {<span class="ot"> runInterpreter ::</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">ExceptT</span> <span class="dt">Exception</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> (<span class="dt">ContT</span></span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> (<span class="dt">Either</span> <span class="dt">Exception</span> ())</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> (<span class="dt">StateT</span> <span class="dt">InterpreterState</span> <span class="dt">IO</span>)<span class="deemphasis">)</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> a</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> ( <span class="dt">Functor</span>,</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">Applicative</span>,</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">Monad</span>,</span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">MonadIO</span>,</span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">MonadBase</span> <span class="dt">IO</span>,</span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">MonadState</span> <span class="dt">InterpreterState</span>,</span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">MonadError</span> <span class="dt">Exception</span><span class="deemphasis">,</span></span>
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">MonadCont</span></span></span>
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a> )</span></code></pre></div>
<p>From bottom to top, the monad stack is comprised of:</p>
<ol type="1">
<li>the <a href="https://hackage.haskell.org/package/base-4.14.0.0/docs/System-IO.html#t:IO" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">IO</span></code></a> monad to be able to print to the console,</li>
<li>the <a href="https://hackage.haskell.org/package/mtl-2.2.2/docs/Control-Monad-State-Strict.html#t:State" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">State</span></code></a> monad transformer to track the state of the interpreter, and</li>
<li>the <a href="https://hackage.haskell.org/package/mtl-2.2.2/docs/Control-Monad-Except.html#t:Except" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">Except</span></code></a> monad transformer to propagate exceptions while interpreting the code.</li>
</ol>
<p>We model the environment as <a href="https://hackage.haskell.org/package/containers/docs/Data-Map-Strict.html#t:Map" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">Map</span></code></a> of variable names to <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html#t:IORef" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">IORef</span></code></a>s of values:</p>
<div class="sourceCode" id="cb11" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Env</span> <span class="ot">=</span> <span class="dt">Map.Map</span> <span class="dt">Identifier</span> (<span class="dt">IORef</span> <span class="dt">Value</span>)</span></code></pre></div>
<p>The immutable nature of <code class="sourceCode haskell"><span class="dt">Map</span></code> and the mutable nature of <code class="sourceCode haskell"><span class="dt">IORef</span></code> allow us to correctly model scopes, frames and closures in <span class="fancy">Co</span>, as we see in the later sections of this post.</p>
<p>The interpreter state contains the environment used for interpretation. The state changes as variables come in and go out of scopes.</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="2:17-2:18,3-3,7:56-7:68"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">InterpreterState</span> <span class="ot">=</span> <span class="dt">InterpreterState</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> {<span class="ot"> isEnv ::</span> <span class="dt">Env</span><span class="deemphasis">,</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"><span class="ot"> isCoroutines ::</span> <span class="dt">Queue</span> (<span class="dt">Coroutine</span> ())</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="ot">initInterpreterState ::</span> <span class="dt">IO</span> <span class="dt">InterpreterState</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a>initInterpreterState <span class="ot">=</span> <span class="dt">InterpreterState</span> <span class="op"><$></span> builtinEnv <span class="deemphasis"><span class="op"><*></span> newQueue</span></span></code></pre></div>
<p>Initial interpreter state contains the built-in environment with bindings for the built-in functions like <code>print</code>. In particular, <code>print</code> is implemented by the <code>executePrint</code> function, which we see in a later section. Note that, <a href="https://en.wikipedia.org/wiki/arity" target="_blank" rel="noopener">arity</a> of built-in functions is also encapsulated in them.</p>
<p><a id="print-func"></a></p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="4-10"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">builtinEnv ::</span> <span class="dt">IO</span> <span class="dt">Env</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>builtinEnv <span class="ot">=</span> Map.fromList <span class="op"><$></span> <span class="fu">traverse</span> (<span class="fu">traverse</span> newIORef) [</span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> (<span class="st">"print"</span>, <span class="dt">BuiltinFunction</span> <span class="st">"print"</span> <span class="dv">1</span> executePrint)</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> , (<span class="st">"newChannel"</span>,</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">BuiltinFunction</span> <span class="st">"newChannel"</span> <span class="dv">0</span> <span class="op">$</span> <span class="fu">fmap</span> <span class="dt">Chan</span> <span class="op">.</span> <span class="fu">const</span> (newChannel <span class="dv">0</span>))</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> , (<span class="st">"newBufferedChannel"</span>,</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">BuiltinFunction</span> <span class="st">"newBufferedChannel"</span> <span class="dv">1</span> executeNewBufferedChannel)</span></span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> , (<span class="st">"sleep"</span>, <span class="dt">BuiltinFunction</span> <span class="st">"sleep"</span> <span class="dv">1</span> executeSleep)</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> , (<span class="st">"getCurrentMillis"</span>,</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">BuiltinFunction</span> <span class="st">"getCurrentMillis"</span> <span class="dv">0</span> executeGetCurrentMillis)</span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> ]</span></code></pre></div>
<p>When trying to interpret wrong code like <code class="sourceCode javascript"><span class="dv">1</span> <span class="op">+</span> <span class="kw">true</span></code>, the interpreter throws runtime errors. We roll these errors along with early returns into an ADT for exceptions:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="4-4"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Exception</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">Return</span> <span class="dt">Value</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">RuntimeError</span> <span class="dt">String</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op">|</span> <span class="dt">CoroutineQueueEmpty</span></span></span></code></pre></div>
<p>That’s it for defining the types for the interpreter. Next, we implement the functions to interpret <span class="fancy">Co</span> programs, starting with functions to work with environments.</p>
<h2 data-track-content data-content-name="manipulating-environments" data-content-piece="implementing-co-2" id="manipulating-environments">Manipulating Environments</h2>
<p>In <span class="fancy">Co</span>, variables must be initialized when being defined. Additionally, only the already defined variables can be referenced or assigned.</p>
<p>To define a new variable, we create a new <code class="sourceCode haskell"><span class="dt">IORef</span></code> with the variable’s value, insert it in the current environment map with the variable name as the key, and replace the interpreter state with the new environment map.</p>
<div class="sourceCode" id="cb12" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="ot">defineVar ::</span> <span class="dt">Identifier</span> <span class="ot">-></span> <span class="dt">Value</span> <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb12-2"><a href="#cb12-2" aria-hidden="true" tabindex="-1"></a>defineVar name value <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb12-3"><a href="#cb12-3" aria-hidden="true" tabindex="-1"></a> env <span class="ot"><-</span> State.gets isEnv</span>
<span id="cb12-4"><a href="#cb12-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">if</span> Map.member name env</span>
<span id="cb12-5"><a href="#cb12-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">then</span> throw <span class="op">$</span> <span class="st">"Variable already defined: "</span> <span class="op"><></span> name</span>
<span id="cb12-6"><a href="#cb12-6" aria-hidden="true" tabindex="-1"></a> <span class="kw">else</span> <span class="kw">do</span></span>
<span id="cb12-7"><a href="#cb12-7" aria-hidden="true" tabindex="-1"></a> valueRef <span class="ot"><-</span> newIORef value</span>
<span id="cb12-8"><a href="#cb12-8" aria-hidden="true" tabindex="-1"></a> setEnv <span class="op">$</span> Map.insert name valueRef env</span>
<span id="cb12-9"><a href="#cb12-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb12-10"><a href="#cb12-10" aria-hidden="true" tabindex="-1"></a><span class="ot">setEnv ::</span> <span class="dt">Env</span> <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb12-11"><a href="#cb12-11" aria-hidden="true" tabindex="-1"></a>setEnv env <span class="ot">=</span> State.modify' <span class="op">$</span> \is <span class="ot">-></span> is {isEnv <span class="ot">=</span> env}</span></code></pre></div>
<p>Note that trying to redefine an already defined variable throws a runtime error.</p>
<p>We also create a helper function <code>setEnv</code> that we reuse in later sections.</p>
<p>To lookup and assign a variable, we get the current environment, lookup the <code class="sourceCode haskell"><span class="dt">IORef</span></code> in the map by the variable’s name, and then read the <code class="sourceCode haskell"><span class="dt">IORef</span></code> for lookup, or write the new value to it for assignment.</p>
<div class="sourceCode" id="cb13" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="ot">lookupVar ::</span> <span class="dt">Identifier</span> <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span></span>
<span id="cb13-2"><a href="#cb13-2" aria-hidden="true" tabindex="-1"></a>lookupVar name <span class="ot">=</span></span>
<span id="cb13-3"><a href="#cb13-3" aria-hidden="true" tabindex="-1"></a> State.gets isEnv <span class="op">>>=</span> findValueRef name <span class="op">>>=</span> readIORef</span>
<span id="cb13-4"><a href="#cb13-4" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb13-5"><a href="#cb13-5" aria-hidden="true" tabindex="-1"></a><span class="ot">assignVar ::</span> <span class="dt">Identifier</span> <span class="ot">-></span> <span class="dt">Value</span> <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb13-6"><a href="#cb13-6" aria-hidden="true" tabindex="-1"></a>assignVar name value <span class="ot">=</span></span>
<span id="cb13-7"><a href="#cb13-7" aria-hidden="true" tabindex="-1"></a> State.gets isEnv <span class="op">>>=</span> findValueRef name <span class="op">>>=</span> <span class="fu">flip</span> writeIORef value</span></code></pre></div>
<p>We use the helper function <code>findValueRef</code> to lookup a variable name in the environment map. It throws a runtime error if the variable is not already defined.</p>
<div class="sourceCode" id="cb14" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="ot">findValueRef ::</span> <span class="dt">Identifier</span> <span class="ot">-></span> <span class="dt">Env</span> <span class="ot">-></span> <span class="dt">Interpreter</span> (<span class="dt">IORef</span> <span class="dt">Value</span>)</span>
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true" tabindex="-1"></a>findValueRef name env <span class="ot">=</span></span>
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">case</span> Map.lookup name env <span class="kw">of</span></span>
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">Just</span> ref <span class="ot">-></span> <span class="fu">return</span> ref</span>
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">Nothing</span> <span class="ot">-></span> throw <span class="op">$</span> <span class="st">"Unknown variable: "</span> <span class="op"><></span> name</span>
<span id="cb14-6"><a href="#cb14-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb14-7"><a href="#cb14-7" aria-hidden="true" tabindex="-1"></a><span class="ot">throw ::</span> <span class="dt">String</span> <span class="ot">-></span> <span class="dt">Interpreter</span> a</span>
<span id="cb14-8"><a href="#cb14-8" aria-hidden="true" tabindex="-1"></a>throw <span class="ot">=</span> throwError <span class="op">.</span> <span class="dt">RuntimeError</span></span></code></pre></div>
<p>These functions are enough for us to implement the evaluation of expressions and execution of statements.</p>
<h2 data-track-content data-content-name="evaluating-expressions" data-content-piece="implementing-co-2" id="evaluating-expressions">Evaluating Expressions</h2>
<p><span class="fancy">Co</span> expressions are represented by the <a href="#cb1-1"><code class="sourceCode haskell"><span class="dt">Expr</span></code></a> ADT. The <code>evaluate</code> function below shows how they are evaluated to runtime values.</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="11-13"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">evaluate ::</span> <span class="dt">Expr</span> <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>evaluate <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">LNull</span> <span class="ot">-></span> <span class="fu">pure</span> <span class="dt">Null</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">LBool</span> bool <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Boolean</span> bool</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">LStr</span> str <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Str</span> str</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">LNum</span> num <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Num</span> num</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">Variable</span> v <span class="ot">-></span> lookupVar v</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">Lambda</span> params body <span class="ot">-></span> <span class="dt">Function</span> <span class="st">"<lambda>"</span> params body <span class="op"><$></span> State.gets isEnv</span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> binary<span class="op">@</span><span class="dt">Binary</span> {} <span class="ot">-></span> evaluateBinaryOp binary</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> call<span class="op">@</span><span class="dt">Call</span> {} <span class="ot">-></span> evaluateFuncCall call</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">Receive</span> expr <span class="ot">-></span> evaluate expr <span class="op">>>=</span> \<span class="kw">case</span></span></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">Chan</span> channel <span class="ot">-></span> channelReceive channel</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> val <span class="ot">-></span> throw <span class="op">$</span> <span class="st">"Cannot receive from a non-channel: "</span> <span class="op"><></span> <span class="fu">show</span> val</span></span></code></pre></div>
<p>Literals <code class="sourceCode javascript"><span class="kw">null</span></code>, booleans, strings, and numbers evaluate to themselves. Variables are looked up from the environment using the <code>lookupVar</code> function we wrote earlier. Anonymous functions are evaluated to function values that capture the current environment from the interpreter state. We learn more about function definitions and calls in the next section. Binary operations and function call expressions are handled by helper functions explained below.</p>
<div class="sourceCode" id="cb15" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="ot">evaluateBinaryOp ::</span> <span class="dt">Expr</span> <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span></span>
<span id="cb15-2"><a href="#cb15-2" aria-hidden="true" tabindex="-1"></a>evaluateBinaryOp <span class="op">~</span>(<span class="dt">Binary</span> op leftE rightE) <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb15-3"><a href="#cb15-3" aria-hidden="true" tabindex="-1"></a> left <span class="ot"><-</span> evaluate leftE</span>
<span id="cb15-4"><a href="#cb15-4" aria-hidden="true" tabindex="-1"></a> right <span class="ot"><-</span> evaluate rightE</span>
<span id="cb15-5"><a href="#cb15-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">let</span> errMsg msg <span class="ot">=</span> msg <span class="op"><></span> <span class="st">": "</span> <span class="op"><></span> <span class="fu">show</span> left <span class="op"><></span> <span class="st">" and "</span> <span class="op"><></span> <span class="fu">show</span> right</span>
<span id="cb15-6"><a href="#cb15-6" aria-hidden="true" tabindex="-1"></a> <span class="kw">case</span> (op, left, right) <span class="kw">of</span></span>
<span id="cb15-7"><a href="#cb15-7" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Plus</span>, <span class="dt">Num</span> n1, <span class="dt">Num</span> n2) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Num</span> <span class="op">$</span> n1 <span class="op">+</span> n2</span>
<span id="cb15-8"><a href="#cb15-8" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Plus</span>, <span class="dt">Str</span> s1, <span class="dt">Str</span> s2) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Str</span> <span class="op">$</span> s1 <span class="op"><></span> s2</span>
<span id="cb15-9"><a href="#cb15-9" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Plus</span>, <span class="dt">Str</span> s1, _) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Str</span> <span class="op">$</span> s1 <span class="op"><></span> <span class="fu">show</span> right</span>
<span id="cb15-10"><a href="#cb15-10" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Plus</span>, _, <span class="dt">Str</span> s2) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Str</span> <span class="op">$</span> <span class="fu">show</span> left <span class="op"><></span> s2</span>
<span id="cb15-11"><a href="#cb15-11" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Plus</span>, _, _) <span class="ot">-></span> throw <span class="op">$</span> errMsg <span class="st">"Cannot add or append"</span></span>
<span id="cb15-12"><a href="#cb15-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb15-13"><a href="#cb15-13" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Minus</span>, <span class="dt">Num</span> n1, <span class="dt">Num</span> n2) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Num</span> <span class="op">$</span> n1 <span class="op">-</span> n2</span>
<span id="cb15-14"><a href="#cb15-14" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Minus</span>, _, _) <span class="ot">-></span> throw <span class="op">$</span> errMsg <span class="st">"Cannot subtract non-numbers"</span></span>
<span id="cb15-15"><a href="#cb15-15" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb15-16"><a href="#cb15-16" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Slash</span>, <span class="dt">Num</span> n1, <span class="dt">Num</span> n2) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Num</span> <span class="op">$</span> n1 <span class="ot">`div`</span> n2</span>
<span id="cb15-17"><a href="#cb15-17" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Slash</span>, _, _) <span class="ot">-></span> throw <span class="op">$</span> errMsg <span class="st">"Cannot divide non-numbers"</span></span>
<span id="cb15-18"><a href="#cb15-18" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb15-19"><a href="#cb15-19" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Star</span>, <span class="dt">Num</span> n1, <span class="dt">Num</span> n2) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Num</span> <span class="op">$</span> n1 <span class="op">*</span> n2</span>
<span id="cb15-20"><a href="#cb15-20" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Star</span>, _, _) <span class="ot">-></span> throw <span class="op">$</span> errMsg <span class="st">"Cannot multiply non-numbers"</span></span>
<span id="cb15-21"><a href="#cb15-21" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb15-22"><a href="#cb15-22" aria-hidden="true" tabindex="-1"></a> (<span class="dt">LessThan</span>, <span class="dt">Num</span> n1, <span class="dt">Num</span> n2) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Boolean</span> <span class="op">$</span> n1 <span class="op"><</span> n2</span>
<span id="cb15-23"><a href="#cb15-23" aria-hidden="true" tabindex="-1"></a> (<span class="dt">LessThan</span>, _, _) <span class="ot">-></span> throw <span class="op">$</span> errMsg <span class="st">"Cannot compare non-numbers"</span></span>
<span id="cb15-24"><a href="#cb15-24" aria-hidden="true" tabindex="-1"></a> (<span class="dt">GreaterThan</span>, <span class="dt">Num</span> n1, <span class="dt">Num</span> n2) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Boolean</span> <span class="op">$</span> n1 <span class="op">></span> n2</span>
<span id="cb15-25"><a href="#cb15-25" aria-hidden="true" tabindex="-1"></a> (<span class="dt">GreaterThan</span>, _, _) <span class="ot">-></span> throw <span class="op">$</span> errMsg <span class="st">"Cannot compare non-numbers"</span></span>
<span id="cb15-26"><a href="#cb15-26" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb15-27"><a href="#cb15-27" aria-hidden="true" tabindex="-1"></a> (<span class="dt">Equals</span>, _, _) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Boolean</span> <span class="op">$</span> left <span class="op">==</span> right</span>
<span id="cb15-28"><a href="#cb15-28" aria-hidden="true" tabindex="-1"></a> (<span class="dt">NotEquals</span>, _, _) <span class="ot">-></span> <span class="fu">pure</span> <span class="op">$</span> <span class="dt">Boolean</span> <span class="op">$</span> left <span class="op">/=</span> right</span></code></pre></div>
<p>To evaluate a binary operation, first we recursively evaluate its left and right operands by calling <code>evaluate</code> on them. Then, depending on the operation and types of the operands, we do different things.</p>
<ul>
<li>The <code class="sourceCode javascript"><span class="op">+</span></code> operation can be used to either add two numbers, or to concat two operands when one or both of them are strings. In all other cases, it throws runtime errors.</li>
<li>The <code class="sourceCode javascript"><span class="op">-</span></code>, <code class="sourceCode haskell"><span class="op">/</span></code>, <code class="sourceCode haskell"><span class="op">*</span></code>, <code class="sourceCode javascript"><span class="op">></span></code>, and <code class="sourceCode javascript"><span class="op"><</span></code> operations can be invoked only on numbers. Other cases throw runtime errors.</li>
<li>The <code class="sourceCode javascript"><span class="op">==</span></code> and <code class="sourceCode javascript"><span class="op">!=</span></code> operations run corresponding Haskell operations on their operands.</li>
</ul>
<p>That’s all for evaluating binary operations. Next, let’s look at how to execute statements. We come back to evaluating function calls after that.</p>
<h2 data-track-content data-content-name="executing-statements" data-content-piece="implementing-co-2" id="executing-statements">Executing Statements</h2>
<p><span class="fancy">Co</span> statements are represented by the <a href="#cb2-1"><code class="sourceCode haskell"><span class="dt">Stmt</span></code></a> ADT. The <code>execute</code> function below uses a case expression to execute the various types of statements in different ways:</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="21-27"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">execute ::</span> <span class="dt">Stmt</span> <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>execute <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">ExprStmt</span> expr <span class="ot">-></span> void <span class="op">$</span> evaluate expr</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">VarStmt</span> name expr <span class="ot">-></span> evaluate expr <span class="op">>>=</span> defineVar name</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">AssignStmt</span> name expr <span class="ot">-></span> evaluate expr <span class="op">>>=</span> assignVar name</span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">IfStmt</span> expr body <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> cond <span class="ot"><-</span> evaluate expr</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> when (isTruthy cond) <span class="op">$</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> traverse_ execute body</span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> while<span class="op">@</span>(<span class="dt">WhileStmt</span> expr body) <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> cond <span class="ot"><-</span> evaluate expr</span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> when (isTruthy cond) <span class="op">$</span> <span class="kw">do</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> traverse_ execute body</span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a> execute while</span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">ReturnStmt</span> mExpr <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a> mRet <span class="ot"><-</span> <span class="fu">traverse</span> evaluate mExpr</span>
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a> throwError <span class="op">.</span> <span class="dt">Return</span> <span class="op">.</span> fromMaybe <span class="dt">Null</span> <span class="op">$</span> mRet</span>
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">FunctionStmt</span> name params body <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a> env <span class="ot"><-</span> State.gets isEnv</span>
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a> defineVar name <span class="op">$</span> <span class="dt">Function</span> name params body env</span>
<span id="cb1-21"><a href="#cb1-21" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">YieldStmt</span> <span class="ot">-></span> yield</span></span>
<span id="cb1-22"><a href="#cb1-22" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">SpawnStmt</span> expr <span class="ot">-></span> spawn expr</span></span>
<span id="cb1-23"><a href="#cb1-23" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">SendStmt</span> expr chan <span class="ot">-></span> evaluate chan <span class="op">>>=</span> \<span class="kw">case</span></span></span>
<span id="cb1-24"><a href="#cb1-24" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">Chan</span> channel <span class="ot">-></span> <span class="kw">do</span></span></span>
<span id="cb1-25"><a href="#cb1-25" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> val <span class="ot"><-</span> evaluate expr</span></span>
<span id="cb1-26"><a href="#cb1-26" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> channelSend val channel</span></span>
<span id="cb1-27"><a href="#cb1-27" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> v <span class="ot">-></span> throw <span class="op">$</span> <span class="st">"Cannot send to a non-channel: "</span> <span class="op"><></span> <span class="fu">show</span> v</span></span>
<span id="cb1-28"><a href="#cb1-28" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb1-29"><a href="#cb1-29" aria-hidden="true" tabindex="-1"></a> isTruthy <span class="ot">=</span> \<span class="kw">case</span></span>
<span id="cb1-30"><a href="#cb1-30" aria-hidden="true" tabindex="-1"></a> <span class="dt">Null</span> <span class="ot">-></span> <span class="dt">False</span></span>
<span id="cb1-31"><a href="#cb1-31" aria-hidden="true" tabindex="-1"></a> <span class="dt">Boolean</span> b <span class="ot">-></span> b</span>
<span id="cb1-32"><a href="#cb1-32" aria-hidden="true" tabindex="-1"></a> _ <span class="ot">-></span> <span class="dt">True</span></span></code></pre></div>
<p>Expressions in expression statements are evaluated by calling <code>evaluate</code> on them, and the resultant values are discarded.</p>
<p>For variable definition and assignment statements, first we evaluate the value expressions, and then define or assign variables with the given variable names and the resultant values.</p>
<p>For <code class="sourceCode javascript"><span class="cf">if</span></code> statements, first we evaluate their conditions, and if conditions yield truthy<a href="#fn11" class="footnote-ref" id="fnref11" role="doc-noteref"><sup>11</sup></a> values, we recursively execute the statement bodies. <code class="sourceCode javascript"><span class="cf">while</span></code> statements are executed in a similar fashion, except we recursively execute the <code class="sourceCode javascript"><span class="cf">while</span></code> statements again after executing their bodies.</p>
<p>For <code class="sourceCode javascript"><span class="cf">return</span></code> statements, we evaluate their optional return value expressions, and then throw the resultant values as exceptions wrapped with the <code class="sourceCode haskell"><span class="dt">Return</span></code> constructor.</p>
<p>Execution of function statements is more interesting. First thing that we do is to capture the current environment from the interpreter state. Then we define a new variable<a href="#fn12" class="footnote-ref" id="fnref12" role="doc-noteref"><sup>12</sup></a> with the function’s name and a runtime function value that encapsulates the function’s name, parameter names, and body statements, as well as, the captured environment. This is how closures record the values of functions’ free variables from their definition contexts.</p>
<p>In the next section, we see how the captured environments and returns as exceptions are used to evaluate function calls.</p>
<h2 data-track-content data-content-name="evaluating-function-calls" data-content-piece="implementing-co-2" id="evaluating-function-calls">Evaluating Function Calls</h2>
<p>The capability of defining and calling functions is the cornerstone of abstraction in programming languages. In <span class="fancy">Co</span>, functions are first-class, and are also the means of implementing scopes and closures. Named functions support recursion<a href="#fn13" class="footnote-ref" id="fnref13" role="doc-noteref"><sup>13</sup></a> as well. Hence, this section is the most important and involved one.</p>
<p>We start by evaluating the callee expression of the function call.</p>
<div class="sourceCode" id="cb16" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb16-1"><a href="#cb16-1" aria-hidden="true" tabindex="-1"></a><span class="ot">evaluateFuncCall ::</span> <span class="dt">Expr</span> <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span></span>
<span id="cb16-2"><a href="#cb16-2" aria-hidden="true" tabindex="-1"></a>evaluateFuncCall <span class="op">~</span>(<span class="dt">Call</span> callee argEs) <span class="ot">=</span></span>
<span id="cb16-3"><a href="#cb16-3" aria-hidden="true" tabindex="-1"></a> evaluate callee <span class="op">>>=</span> \<span class="kw">case</span></span>
<span id="cb16-4"><a href="#cb16-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">BuiltinFunction</span> name arity func <span class="ot">-></span> <span class="kw">do</span></span>
<span id="cb16-5"><a href="#cb16-5" aria-hidden="true" tabindex="-1"></a> checkArgCount name argEs arity</span>
<span id="cb16-6"><a href="#cb16-6" aria-hidden="true" tabindex="-1"></a> func argEs</span>
<span id="cb16-7"><a href="#cb16-7" aria-hidden="true" tabindex="-1"></a> func<span class="op">@</span><span class="dt">Function</span> {} <span class="ot">-></span> evaluateFuncCall' func argEs</span>
<span id="cb16-8"><a href="#cb16-8" aria-hidden="true" tabindex="-1"></a> val <span class="ot">-></span> throw <span class="op">$</span> <span class="st">"Cannot call a non-function: "</span> <span class="op"><></span> <span class="fu">show</span> callee <span class="op"><></span> <span class="st">" is "</span> <span class="op"><></span> <span class="fu">show</span> val</span>
<span id="cb16-9"><a href="#cb16-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb16-10"><a href="#cb16-10" aria-hidden="true" tabindex="-1"></a><span class="ot">checkArgCount ::</span> <span class="dt">Identifier</span> <span class="ot">-></span> [<span class="dt">Expr</span>] <span class="ot">-></span> <span class="dt">Int</span> <span class="ot">-></span> <span class="dt">Interpreter</span> ()</span>
<span id="cb16-11"><a href="#cb16-11" aria-hidden="true" tabindex="-1"></a>checkArgCount funcName argEs arity <span class="ot">=</span></span>
<span id="cb16-12"><a href="#cb16-12" aria-hidden="true" tabindex="-1"></a> when (<span class="fu">length</span> argEs <span class="op">/=</span> arity) <span class="op">$</span></span>
<span id="cb16-13"><a href="#cb16-13" aria-hidden="true" tabindex="-1"></a> throw <span class="op">$</span> funcName <span class="op"><></span> <span class="st">" call expected "</span> <span class="op"><></span> <span class="fu">show</span> arity</span>
<span id="cb16-14"><a href="#cb16-14" aria-hidden="true" tabindex="-1"></a> <span class="op"><></span> <span class="st">" argument(s) but received "</span> <span class="op"><></span> <span class="fu">show</span> (<span class="fu">length</span> argEs)</span>
<span id="cb16-15"><a href="#cb16-15" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb16-16"><a href="#cb16-16" aria-hidden="true" tabindex="-1"></a><span class="ot">executePrint ::</span> [<span class="dt">Expr</span>] <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span></span>
<span id="cb16-17"><a href="#cb16-17" aria-hidden="true" tabindex="-1"></a>executePrint argEs <span class="ot">=</span></span>
<span id="cb16-18"><a href="#cb16-18" aria-hidden="true" tabindex="-1"></a> evaluate (<span class="fu">head</span> argEs) <span class="op">>>=</span> liftIO <span class="op">.</span> <span class="fu">print</span> <span class="op">>></span> <span class="fu">return</span> <span class="dt">Null</span></span></code></pre></div>
<p>If the resultant value is not a function, we throw a runtime error.</p>
<p>If we get a built-in function, we check that the count of arguments is same as the arity of the function by invoking <code>checkArgCount</code>, failing which we throw a runtime error. Then, we invoke the corresponding implementation function. For <code>print</code>, it is the <code>executePrint</code> function, in which we evaluate the argument and print it using Haskell’s <a href="https://hackage.haskell.org/package/base-4.12.0.0/docs/Prelude.html#v:print" target="_blank" rel="noopener"><code>print</code></a> function.</p>
<p>If we get a user-defined function, we evaluate the function call with the helper function <code>evaluateFuncCall'</code>. But before diving into it, let’s take a look at how the world looks from inside a function.</p>
<div class="sourceCode" id="cb17" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">makeGreeter</span>(greeting) {</span>
<span id="cb17-2"><a href="#cb17-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">function</span> <span class="fu">greeter</span>(name) {</span>
<span id="cb17-3"><a href="#cb17-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> say <span class="op">=</span> greeting <span class="op">+</span> <span class="st">" "</span> <span class="op">+</span> name<span class="op">;</span></span>
<span id="cb17-4"><a href="#cb17-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(say)<span class="op">;</span></span>
<span id="cb17-5"><a href="#cb17-5" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb17-6"><a href="#cb17-6" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> greeter<span class="op">;</span></span>
<span id="cb17-7"><a href="#cb17-7" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb17-8"><a href="#cb17-8" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb17-9"><a href="#cb17-9" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> hello <span class="op">=</span> <span class="fu">makeGreeter</span>(<span class="st">"hello"</span>)<span class="op">;</span></span>
<span id="cb17-10"><a href="#cb17-10" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> namaste <span class="op">=</span> <span class="fu">makeGreeter</span>(<span class="st">"namaste"</span>)<span class="op">;</span></span>
<span id="cb17-11"><a href="#cb17-11" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb17-12"><a href="#cb17-12" aria-hidden="true" tabindex="-1"></a><span class="fu">hello</span>(<span class="st">"Arthur"</span>)<span class="op">;</span></span>
<span id="cb17-13"><a href="#cb17-13" aria-hidden="true" tabindex="-1"></a><span class="fu">namaste</span>(<span class="st">"Ford"</span>)<span class="op">;</span></span></code></pre></div>
<p>In the above <span class="fancy">Co</span> code, the function <code>greeter</code> has a free variable <code>greeting</code>, a bound parameter <code>name</code>, and a local variable <code>say</code>. Upon executing the code with the interpreter, we get the following output:</p>
<pre class="plain"><code>hello Arthur
namaste Ford</code></pre>
<p>The output makes sense when we understand the variables <code>hello</code> and <code>namaste</code> are closures over the function <code>greeter</code>. The environment seen from inside <code>greeter</code> when it is being executed is a mix of the scope (and hence, the environment) it is defined in, and the scope it is called in.</p>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 465 241'%3E%3C/svg%3E" class="lazyload ascii-art w-70pct nolink" style="--image-aspect-ratio: 1.929460580912863" data-src="/images/implementing-co-2/function-eval.svg" alt="Function environment is a mix of its caller and definition environments"></img>
<noscript><img src="/images/implementing-co-2/function-eval.svg" class="ascii-art w-70pct nolink" alt="Function environment is a mix of its caller and definition environments"></img></noscript>
<figcaption>Function environment is a mix of its caller and definition environments</figcaption>
</figure>
<p>More specifically, the free variables come from the definition scope, and the parameters come from the caller scope. Local variables can be derived from any combinations of free variables and parameters. With this understanding, let’s see how we evaluate function calls:</p>
<div class="sourceCode" id="cb19" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb19-1"><a href="#cb19-1" aria-hidden="true" tabindex="-1"></a><span class="ot">evaluateFuncCall' ::</span> <span class="dt">Value</span> <span class="ot">-></span> [<span class="dt">Expr</span>] <span class="ot">-></span> <span class="dt">Interpreter</span> <span class="dt">Value</span></span>
<span id="cb19-2"><a href="#cb19-2" aria-hidden="true" tabindex="-1"></a>evaluateFuncCall'</span>
<span id="cb19-3"><a href="#cb19-3" aria-hidden="true" tabindex="-1"></a> <span class="op">~</span>func<span class="op">@</span>(<span class="dt">Function</span> funcName params body funcDefEnv) argEs <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb19-4"><a href="#cb19-4" aria-hidden="true" tabindex="-1"></a> checkArgCount funcName argEs (<span class="fu">length</span> params)</span>
<span id="cb19-5"><a href="#cb19-5" aria-hidden="true" tabindex="-1"></a> funcCallEnv <span class="ot"><-</span> State.gets isEnv</span>
<span id="cb19-6"><a href="#cb19-6" aria-hidden="true" tabindex="-1"></a> setupFuncEnv</span>
<span id="cb19-7"><a href="#cb19-7" aria-hidden="true" tabindex="-1"></a> retVal <span class="ot"><-</span> executeBody funcCallEnv</span>
<span id="cb19-8"><a href="#cb19-8" aria-hidden="true" tabindex="-1"></a> setEnv funcCallEnv</span>
<span id="cb19-9"><a href="#cb19-9" aria-hidden="true" tabindex="-1"></a> <span class="fu">return</span> retVal</span>
<span id="cb19-10"><a href="#cb19-10" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb19-11"><a href="#cb19-11" aria-hidden="true" tabindex="-1"></a> setupFuncEnv <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb19-12"><a href="#cb19-12" aria-hidden="true" tabindex="-1"></a> args <span class="ot"><-</span> <span class="fu">traverse</span> evaluate argEs</span>
<span id="cb19-13"><a href="#cb19-13" aria-hidden="true" tabindex="-1"></a> env <span class="ot"><-</span> overrideVar funcDefEnv funcName func</span>
<span id="cb19-14"><a href="#cb19-14" aria-hidden="true" tabindex="-1"></a> env' <span class="ot"><-</span> foldM (<span class="fu">uncurry</span> <span class="op">.</span> overrideVar) env <span class="op">$</span> <span class="fu">zip</span> params args</span>
<span id="cb19-15"><a href="#cb19-15" aria-hidden="true" tabindex="-1"></a> setEnv env'</span>
<span id="cb19-16"><a href="#cb19-16" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb19-17"><a href="#cb19-17" aria-hidden="true" tabindex="-1"></a> overrideVar env name value <span class="ot">=</span></span>
<span id="cb19-18"><a href="#cb19-18" aria-hidden="true" tabindex="-1"></a> Map.insert name <span class="op"><$></span> newIORef value <span class="op"><*></span> <span class="fu">pure</span> env</span>
<span id="cb19-19"><a href="#cb19-19" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb19-20"><a href="#cb19-20" aria-hidden="true" tabindex="-1"></a> executeBody funcCallEnv <span class="ot">=</span></span>
<span id="cb19-21"><a href="#cb19-21" aria-hidden="true" tabindex="-1"></a> (traverse_ execute body <span class="op">>></span> <span class="fu">return</span> <span class="dt">Null</span>) <span class="ot">`catchError`</span> \<span class="kw">case</span></span>
<span id="cb19-22"><a href="#cb19-22" aria-hidden="true" tabindex="-1"></a> <span class="dt">Return</span> val <span class="ot">-></span> <span class="fu">return</span> val</span>
<span id="cb19-23"><a href="#cb19-23" aria-hidden="true" tabindex="-1"></a> err <span class="ot">-></span> setEnv funcCallEnv <span class="op">>></span> throwError err</span></code></pre></div>
<p>Let’s go over the above code, step by step:</p>
<ol type="1">
<li><code>evaluateFuncCall'</code> is called with the function to evaluate. We get access to the function’s name, its parameter names, body statements, and the environment it is defined in. We also get the argument expressions for the function call. (Line 2–3)</li>
<li>First, we check that the count of arguments is same as the count of the function parameter by invoking <code>checkArgCount</code>, failing which we throw a runtime error. (Line 4)</li>
<li>Then, we capture the current environment from the interpreter state. This is the function’s caller’s environment. (Line 5)</li>
<li>Next, we set up the environment in which the function will be executed (line 6). In <code>setupFuncEnv</code>:
<ol type="a">
<li>We evaluate the argument expressions in the current (caller’s) environment<a href="#fn14" class="footnote-ref" id="fnref14" role="doc-noteref"><sup>14</sup></a>. (Line 12)</li>
<li>We bind the callee function itself to its name in its own environment. This lets our function to recursively call itself. (Line 13)</li>
<li>We bind the argument values to their parameter names in the function’s environment. This lets the function body access the arguments being called with. (Line 14)</li>
<li>We set the current environment in the interpreter state to the functions’s environment. (Line 15)</li>
</ol></li>
<li>With the function environment set up, we execute the function body in <code>executeBody</code> (line 7):
<ol type="a">
<li>We execute each statement in the body, and return <code class="sourceCode javascript"><span class="kw">null</span></code> in case there was no explicit <code class="sourceCode javascript"><span class="cf">return</span></code> in the function. (Line 21)</li>
<li>If the body contains a <code class="sourceCode javascript"><span class="cf">return</span></code> statement, or if its execution throws a runtime error, we handle the exception in the <code>catchError</code> case statement.
<ol type="i">
<li>For <code class="sourceCode javascript"><span class="cf">return</span></code>, we pass along the return value. (Line 22)</li>
<li>For a runtime error, first we set the current environment back to the caller’s environment that we captured in step 3, and then we throw the error. The error is eventually handled in the <code>interpret</code> function described in the next section. (Line 23)</li>
</ol></li>
<li>We capture the value returned from executing the body. (Line 7)</li>
</ol></li>
<li>We set the current environment back to the caller’s environment that we captured in step 3. (Line 8)</li>
<li>We return the captured return value from <code>evaluateFuncCall'</code>. The function call is now complete. (Line 9)</li>
</ol>
<p>Curious readers may wonder, why do we need to use State monad, <a href="https://hackage.haskell.org/package/containers/docs/Data-Map-Strict.html#t:Map" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">Map</span></code></a>s, and <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html#t:IORef" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">IORef</span></code></a>s together, when all of them do similar work of storing and mutating variables? Because, together they let us implement function calls, scopes and closures, as described below:</p>
<ol type="1">
<li>State monad lets us swap the current environment for a function’s definition environment when a function call is made, and to restore the calling environment after the call is complete.</li>
<li>Immutable maps are perfect for implementing scopes. Adding variables in an immutable map returns a modified map without changing the original map. This lets us shadow variables defined in outer scopes when entering inner scopes, while also being able to easily restore the shadowed variables by just restoring the original map after the inner scopes end<a href="#fn15" class="footnote-ref" id="fnref15" role="doc-noteref"><sup>15</sup></a>. There is no need to use a stack of mutable maps, which is how environments are generally implemented in interpreters which do not use immutable maps.</li>
<li>Lastly, putting <code class="sourceCode haskell"><span class="dt">IORef</span></code>s as values of immutable maps lets us implement mutable closures. All closures of same function share the same references to the <code class="sourceCode haskell"><span class="dt">IORef</span></code>s. This allows variable mutations made from one closure to be visible to all others. If we had used just immutable maps, changes made to variable values would not propagate between closures because of immutability.</li>
</ol>
<p>So that’s how function calls—the most crucial part of the interpreter—work. That completes the guts of our interpreter for the <a href="#previously-on">basic features</a> of <span class="fancy">Co</span>. In the next and last section, we put everything together.</p>
<h2 data-track-content data-content-name="interpreting-a-program" data-content-piece="implementing-co-2" id="interpreting-a-program">Interpreting a Program</h2>
<p>We are down to the last step. We interpret a program returned from the parser written in the <a href="https://abhinavsarkar.net/posts/implementing-co-1/?mtm_campaign=feed">previous post</a> to run it.</p>
<div id="cb1" class="sourceCode" data-lang="haskell" data-deemphasize="5-5,8:33-8:54,8:7-8:8,12-12"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ot">interpret ::</span> <span class="dt">Program</span> <span class="ot">-></span> <span class="dt">IO</span> (<span class="dt">Either</span> <span class="dt">String</span> ())</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a>interpret program <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> state <span class="ot"><-</span> initInterpreterState</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> retVal <span class="ot"><-</span> <span class="fu">flip</span> evalStateT state</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="op">.</span> <span class="fu">flip</span> runContT <span class="fu">return</span></span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> runExceptT</span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span> runInterpreter</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="op">$</span> <span class="deemphasis">(</span>traverse_ execute program<span class="deemphasis"> <span class="op">>></span> awaitTermination)</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="kw">case</span> retVal <span class="kw">of</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">Left</span> (<span class="dt">RuntimeError</span> err) <span class="ot">-></span> <span class="fu">return</span> <span class="op">$</span> <span class="dt">Left</span> err</span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">Left</span> (<span class="dt">Return</span> _) <span class="ot">-></span> <span class="fu">return</span> <span class="op">$</span> <span class="dt">Left</span> <span class="st">"Cannot return from outside functions"</span></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a><span class="deemphasis"> <span class="dt">Left</span> <span class="dt">CoroutineQueueEmpty</span> <span class="ot">-></span> <span class="fu">return</span> <span class="op">$</span> <span class="dt">Right</span> ()</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">Right</span> _ <span class="ot">-></span> <span class="fu">return</span> <span class="op">$</span> <span class="dt">Right</span> ()</span></code></pre></div>
<p>We run the list of statements in the program by running the <code>execute</code> function on them. Then we run the monad transformer stack, layer by layer, to get the return value. Finally, we case match on the return value to catch errors, and we are done.</p>
<p>We package the parser and the interpreter together to create the <code>runFile</code> function that takes a file path, reads and parses the file, and then interprets the AST:</p>
<div class="sourceCode" id="cb20" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb20-1"><a href="#cb20-1" aria-hidden="true" tabindex="-1"></a><span class="ot">runFile ::</span> <span class="dt">FilePath</span> <span class="ot">-></span> <span class="dt">IO</span> ()</span>
<span id="cb20-2"><a href="#cb20-2" aria-hidden="true" tabindex="-1"></a>runFile file <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb20-3"><a href="#cb20-3" aria-hidden="true" tabindex="-1"></a> code <span class="ot"><-</span> <span class="fu">readFile</span> file</span>
<span id="cb20-4"><a href="#cb20-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">case</span> runParser program code <span class="kw">of</span></span>
<span id="cb20-5"><a href="#cb20-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">Left</span> err <span class="ot">-></span> hPutStrLn stderr err</span>
<span id="cb20-6"><a href="#cb20-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">Right</span> program <span class="ot">-></span> interpret program <span class="op">>>=</span> \<span class="kw">case</span></span>
<span id="cb20-7"><a href="#cb20-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">Left</span> err <span class="ot">-></span> hPutStrLn stderr <span class="op">$</span> <span class="st">"ERROR: "</span> <span class="op"><></span> err</span>
<span id="cb20-8"><a href="#cb20-8" aria-hidden="true" tabindex="-1"></a> _ <span class="ot">-></span> <span class="fu">return</span> ()</span></code></pre></div>
<p>Finally, we can run the interpreter on the <span class="fancy">Co</span> files:</p>
<div class="sourceCode" id="cb21" data-lang="ghci"><pre class="sourceCode lhs numberSource"><code class="sourceCode literatehaskell"><span id="cb21-1"><a href="#cb21-1" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> runFile <span class="st">"fib.co"</span></span>
<span id="cb21-2"><a href="#cb21-2" aria-hidden="true" tabindex="-1"></a>0</span>
<span id="cb21-3"><a href="#cb21-3" aria-hidden="true" tabindex="-1"></a>1</span>
<span id="cb21-4"><a href="#cb21-4" aria-hidden="true" tabindex="-1"></a>1</span>
<span id="cb21-5"><a href="#cb21-5" aria-hidden="true" tabindex="-1"></a>2</span>
<span id="cb21-6"><a href="#cb21-6" aria-hidden="true" tabindex="-1"></a>3</span>
<span id="cb21-7"><a href="#cb21-7" aria-hidden="true" tabindex="-1"></a>5</span>
<span id="cb21-8"><a href="#cb21-8" aria-hidden="true" tabindex="-1"></a>0</span>
<span id="cb21-9"><a href="#cb21-9" aria-hidden="true" tabindex="-1"></a>1</span>
<span id="cb21-10"><a href="#cb21-10" aria-hidden="true" tabindex="-1"></a>1</span>
<span id="cb21-11"><a href="#cb21-11" aria-hidden="true" tabindex="-1"></a>2</span>
<span id="cb21-12"><a href="#cb21-12" aria-hidden="true" tabindex="-1"></a>3</span>
<span id="cb21-13"><a href="#cb21-13" aria-hidden="true" tabindex="-1"></a>5</span></code></pre></div>
<hr></hr>
<p>That’s all for now. We implemented the interpreter for the <a href="#previously-on">basic features</a> for <span class="fancy">Co</span>, and learned about how function calls, scopes and closures work. In the <a href="https://abhinavsarkar.net/posts/implementing-co-3/?mtm_campaign=feed">next part</a>, we’ll extend our interpreter to add support for coroutines and channels in <span class="fancy">Co</span>.</p>
<p>The full code for the interpreter can be seen <a href="https://abhinavsarkar.net/code/co-interpreter.html?mtm_campaign=feed">here</a>.</p>
<div id="refs" class="references csl-bib-body hanging-indent" role="list">
<div id="ref-Abelson1996-c556" class="csl-entry" role="listitem">
Abelson, Harold, Gerald Jay Sussman, and with Julie Sussman. <span>“Lexical Addressing.”</span> In <em>Structure and Interpretation of Computer Programs</em>, 2nd Editon. MIT Press/McGraw-Hill, 1996. <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-35.html#%_sec_5.5.6" target="_blank" rel="noopener">https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-35.html#%_sec_5.5.6</a>.
</div>
<div id="ref-Abelson1996-c4" class="csl-entry" role="listitem">
———. <span>“Metalinguistic Abstraction.”</span> In <em>Structure and Interpretation of Computer Programs</em>, 2nd Editon. MIT Press/McGraw-Hill, 1996. <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-25.html#%_chap_4" target="_blank" rel="noopener">https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-25.html#%_chap_4</a>.
</div>
<div id="ref-Abelson1996-c421" class="csl-entry" role="listitem">
———. <span>“Normal Order and Applicative Order.”</span> In <em>Structure and Interpretation of Computer Programs</em>, 2nd Editon. MIT Press/McGraw-Hill, 1996. <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-27.html#%_sec_4.2.1" target="_blank" rel="noopener">https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-27.html#%_sec_4.2.1</a>.
</div>
<div id="ref-Abelson1996-c118" class="csl-entry" role="listitem">
———. <span>“Procedures as Black-Box Abstractions.”</span> In <em>Structure and Interpretation of Computer Programs</em>, 2nd Editon. MIT Press/McGraw-Hill, 1996. <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-10.html#%_sec_1.1.8" target="_blank" rel="noopener">https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-10.html#%_sec_1.1.8</a>.
</div>
<div id="ref-Abelson1996-c313" class="csl-entry" role="listitem">
———. <span>“The Costs of Introducing Assignment.”</span> In <em>Structure and Interpretation of Computer Programs</em>, 2nd Editon. MIT Press/McGraw-Hill, 1996. <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-20.html#%_sec_3.1.3" target="_blank" rel="noopener">https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-20.html#%_sec_3.1.3</a>.
</div>
<div id="ref-Abelson1996-c32" class="csl-entry" role="listitem">
———. <span>“The Environment Model of Evaluation.”</span> In <em>Structure and Interpretation of Computer Programs</em>, 2nd Editon. MIT Press/McGraw-Hill, 1996. <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-21.html#%_sec_3.2" target="_blank" rel="noopener">https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-21.html#%_sec_3.2</a>.
</div>
<div id="ref-Abelson1996-c115" class="csl-entry" role="listitem">
———. <span>“The Substitution Model for Procedure Application.”</span> In <em>Structure and Interpretation of Computer Programs</em>, 2nd Editon. MIT Press/McGraw-Hill, 1996. <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-10.html#%_sec_1.1.5" target="_blank" rel="noopener">https://mitp-content-server.mit.edu/books/content/sectbyfn/books_pres_0/6515/sicp.zip/full-text/book/book-Z-H-10.html#%_sec_1.1.5</a>.
</div>
</div>
<section id="footnotes" class="footnotes footnotes-end-of-document" role="doc-endnotes">
<hr></hr>
<ol>
<li id="fn1"><p>It’s hard to find examples of real-world programming languages that are run with AST interpreters. This is because AST interpreters are too slow for real-world usage. However, they are the easiest to understand and implement, and hence are widely using in teaching programming languages theory.<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn2"><p>Since the user-defined and built-in functions are first-class, they can be assigned to variables, and passed as arguments to other functions. Thus, <span class="fancy">Co</span> supports <a href="https://en.wikipedia.org/wiki/higher-order_functions" target="_blank" rel="noopener">higher-order functions</a> as well.<a href="#fnref2" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn3"><p>This is called <a href="https://en.wikipedia.org/wiki/Strong_typing" target="_blank" rel="noopener">Strong typing</a> in programming languages parlance. JavaScript, on the other hand, is a weakly typed language. In JavaScript, <code class="sourceCode javascript"><span class="dv">1</span> <span class="op">==</span> <span class="st">'1'</span></code> evaluates to <code class="sourceCode javascript"><span class="kw">true</span></code>, whereas in <span class="fancy">Co</span>, it evaluates to <code class="sourceCode javascript"><span class="kw">false</span></code>.<a href="#fnref3" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn4"><p>The property of being able to substitute expressions for their corresponding values without changing the meaning of the program is called <a href="https://en.wikipedia.org/wiki/Referential_transparency" target="_blank" rel="noopener"><em>Referential transparency</em></a><sup><a href="#ref-Abelson1996-c313" class="citation" title="(Abelson, Sussman, and with Julie Sussman, “The Costs of Introducing
Assignment”)
">@6</a></sup>. Pure functions—like <code>twice</code> here—that do not have any side-effects are referentially transparent.<a href="#fnref4" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn5"><p>I’m being a little hand-wavy here because most programmers have at least an intuitive understanding of scopes. Read literature for accurate details.<a href="#fnref5" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn6"><p>This is in contrast to <a href="https://en.wikipedia.org/wiki/Dynamic_scoping" target="_blank" rel="noopener"><em>Dynamic scoping</em></a> where the a variable’s scope is essentially global, and is defined by function’s execution context instead of definition context, as in lexical scoping.<a href="#fnref6" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn7"><p><a href="https://en.wikipedia.org/wiki/Block_(programming)" target="_blank" rel="noopener"><em>Blocks</em></a> are another widely used structure to support lexical scoping. <span class="fancy">Co</span> doesn’t have blocks in the interest of simplicity of implementation.<a href="#fnref7" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn8"><p>The function is said to close its free variables over its closure. Hence, the name <em>Closure</em>.<a href="#fnref8" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn9"><p>Some programming languages like Java support a limited version of closures, which requires values of the free variables of functions to not change over time.<a href="#fnref9" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn10"><p>Well, not entirely, because Haskell is a lazily evaluated language.<a href="#fnref10" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn11"><p>In <span class="fancy">Co</span>, only <code class="sourceCode javascript"><span class="kw">null</span></code> and <code class="sourceCode javascript"><span class="kw">false</span></code> evaluate to false. All other values evaluate to true. This is implemented by the <code>isTruthy</code> function.<a href="#fnref11" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn12"><p>Functions are just variables in <span class="fancy">Co</span>. That is to say, functions definitions and variable definitions share the same namespace. This is how it works in many programming languages like JavaScript and Python. But some languages like <a href="https://en.wikipedia.org/wiki/Common_Lisp" target="_blank" rel="noopener">Common Lisp</a> have <a href="https://en.wikipedia.org/wiki/Common_Lisp#The_function_namespace" target="_blank" rel="noopener">separate namespaces</a> for functions and variables.<a href="#fnref12" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn13"><p><span class="fancy">Co</span> does not support mutual recursion though. This is because a function in <span class="fancy">Co</span> only sees the bindings done before its own definition. This can be fixed by either adding a special syntax for mutually recursive functions, or by hoisting all the bindings in a scope to the top of the scope, like <a href="https://developer.mozilla.org/en-US/docs/Glossary/Hoisting" target="_blank" rel="noopener">how JavaScript does</a>.</p>
<p>Anonymous functions do not support recursion at all, because they do not have names to refer to themselves in their bodies.<a href="#fnref13" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn14"><p>Evaluating function arguments before the function body is called the <a href="https://en.wikipedia.org/wiki/Evaluation_strategy#Eager_evaluation" target="_blank" rel="noopener"><em>Strict evaluation strategy</em></a>. Most of the modern programming languages work this way, for example, Java, Python, JavaScript, Ruby etc. This is in contrast to <a href="https://en.wikipedia.org/wiki/Evaluation_strategy#Non-strict_evaluation" target="_blank" rel="noopener"><em>Non-strict evaluation</em></a> in programming languages like Haskell, where the arguments to functions are evaluated only when their values are needed in the function bodies<sup><a href="#ref-Abelson1996-c421" class="citation" title="(Abelson, Sussman, and with Julie Sussman, “Normal Order and Applicative
Order”)
">@21</a></sup>.<a href="#fnref14" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn15"><p>This is what the <code>overrideVar</code> function does in the code above.<a href="#fnref15" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
</ol>
</section><p>If you liked this post, please <a href="https://abhinavsarkar.net/posts/implementing-co-2/?mtm_campaign=feed#syndications">leave a comment</a>.</p><img referrerpolicy="no-referrer-when-downgrade" src="https://anna.abhinavsarkar.net/matomo.php?idsite=1&rec=1" style="border:0" alt="" />2021-09-21T00:00:00Z<p>In the <a href="https://abhinavsarkar.net/posts/implementing-co-1/">previous post</a>, we wrote the parser for <span class="fancy">Co</span>, the small language we are building in this series of posts. The previous post was all about the syntax of <span class="fancy">Co</span>. In this post we dive into the semantics of <span class="fancy">Co</span>, and write an interpreter for its basic features.</p>
https://abhinavsarkar.net/posts/implementing-co-1/Implementing Co, a Small Language With Coroutines #1: The Parser2021-04-24T00:00:00ZAbhinav Sarkarhttps://abhinavsarkar.net/about/abhinav@abhinavsarkar.net<p>Many major programming languages these days support some lightweight concurrency primitives. The most recent popular ones are <a href="https://en.wikipedia.org/wiki/Go_(programming_language)#Concurrency:_goroutines_and_channels" target="_blank" rel="noopener">Goroutines</a> in <a href="https://golang.org/" target="_blank" rel="noopener">Go</a>, <a href="https://kotlinlang.org/docs/coroutines-basics.html" target="_blank" rel="noopener">Coroutines</a> in <a href="https://kotlinlang.org/" target="_blank" rel="noopener">Kotlin</a> and <a href="https://rust-lang.github.io/async-book/01_getting_started/02_why_async.html" target="_blank" rel="noopener">Async</a> in <a href="https://www.rust-lang.org/" target="_blank" rel="noopener">Rust</a>. Let’s explore some of these concepts in detail by implementing a programming language with support for coroutines and Go-style channels.</p>
<p>This post was originally published on <a href="https://abhinavsarkar.net/posts/implementing-co-1/?mtm_campaign=feed">abhinavsarkar.net</a>.</p><!--more-->
<p>This is the first post in a series of posts:</p>
<ol type="1">
<li>Implementing Co #1: The Parser</li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-2/?mtm_campaign=feed">Implementing Co #2: The Interpreter</a></li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-3/?mtm_campaign=feed">Implementing Co #3: Adding Coroutines</a></li>
<li><a href="https://abhinavsarkar.net/posts/implementing-co-4/?mtm_campaign=feed">Implementing Co #4: Adding Channels</a></li>
<li>Implementing Co #5: Adding Sleep</li>
</ol>
<nav id="toc" class="right-toc"><h3>Contents</h3><ol><li><a href="#lightweight-concurrency">Lightweight Concurrency</a></li><li><a href="#introducing-co">Introducing <span class="fancy">Co</span></a></li><li><a href="#the-co-interpreter">The <span class="fancy">Co</span> Interpreter</a></li><li><a href="#the-co-ast">The <span class="fancy">Co</span> AST</a><ol><li><a href="#expressions">Expressions</a></li><li><a href="#statements">Statements</a></li></ol></li><li><a href="#parsing">Parsing</a><ol><li><a href="#parsing-expressions">Parsing Expressions</a></li><li><a href="#parsing-statements">Parsing Statements</a></li></ol></li></ol></nav>
<h2 data-track-content data-content-name="lightweight-concurrency" data-content-piece="implementing-co-1" id="lightweight-concurrency">Lightweight Concurrency</h2>
<p><a href="https://en.wikipedia.org/wiki/Light-weight_process" target="_blank" rel="noopener">Lightweight concurrency</a> has been a popular topic among programmers and programming language designers alike in recent times. Many languages created in the last decade have support for them either natively or using libraries. Some example are:</p>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Go_(programming_language)#Concurrency:_goroutines_and_channels" target="_blank" rel="noopener">Goroutines</a> in <a href="https://golang.org/" target="_blank" rel="noopener">Go</a>,</li>
<li><a href="https://kotlinlang.org/docs/coroutines-basics.html" target="_blank" rel="noopener">Coroutines</a> in <a href="https://kotlinlang.org/" target="_blank" rel="noopener">Kotlin</a>,</li>
<li><a href="https://rust-lang.github.io/async-book/01_getting_started/02_why_async.html" target="_blank" rel="noopener">Async</a> in <a href="https://www.rust-lang.org/" target="_blank" rel="noopener">Rust</a>, and</li>
<li><a href="https://clojure.github.io/core.async/" target="_blank" rel="noopener">core.async</a> in <a href="https://clojure.org/" target="_blank" rel="noopener">Clojure</a>.</li>
</ul>
<p>These examples differ in their implementation details but all of them enable programmers to run millions of tasks concurrently. This capability of being able to do multiple tasks at the same time is called <a href="https://en.wikipedia.org/wiki/Computer_multitasking" target="_blank" rel="noopener"><em>Multitasking</em></a>.</p>
<p>Multitasking can be of two types:</p>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Pre-emptive_multitasking" target="_blank" rel="noopener">Pre-emptive multitasking</a> in which the tasks can be <a href="https://en.wikipedia.org/wiki/Preemption_(computing)" target="_blank" rel="noopener">preempted</a> so that other tasks can be run.</li>
<li><a href="https://en.wikipedia.org/wiki/Cooperative_multitasking" target="_blank" rel="noopener">Cooperative multitasking</a><sup><a href="#ref-Bartel2011-ap" class="citation" title="Bartel, “Non-Preemptive Multitasking.”
">@1</a></sup> in which the tasks voluntarily yield control to other tasks to be run.</li>
</ul>
<p><a href="https://en.wikipedia.org/wiki/Coroutines" target="_blank" rel="noopener">Coroutines</a><sup><a href="#ref-Knuth1997-rv" class="citation" title="Knuth, “Coroutines.”
">@2</a></sup> are computations that support cooperative multitasking<a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a>. Unlike ordinary <a href="https://en.wikipedia.org/wiki/Subroutines" target="_blank" rel="noopener"><em>Subroutines</em></a> that execute from start to end and do not hold any state between invocations, coroutines can exit in the middle by calling other coroutines and may later resume at the same point. They also hold state between invocations. They do so by <a href="https://en.wikipedia.org/wiki/Yield_(multithreading)" target="_blank" rel="noopener"><em>yielding</em></a> the control of the current running thread so that some other coroutine can be run on the same thread<a href="#fn2" class="footnote-ref" id="fnref2" role="doc-noteref"><sup>2</sup></a>.</p>
<p>Coroutine implementations often come with support for <a href="https://en.wikipedia.org/wiki/Channel_(programming)" target="_blank" rel="noopener"><em>Channels</em></a> for inter-coroutine communication. One coroutine can send a message over a channel, and another coroutine can receive the message from the same channel. Coroutines and channels together are an implementation of <a href="https://en.wikipedia.org/wiki/Communicating_Sequential_Processes" target="_blank" rel="noopener"><em>Communicating Sequential Processes</em></a> (CSP)<sup><a href="#ref-Hoare1986-ih" class="citation" title="Hoare, Communicating Sequential Processes.
">@5</a></sup>, a formal language for describing patterns of interaction in concurrent systems.</p>
<p>In this series of posts, we implement <span class="fancy">Co</span>, a small dynamically typed <a href="https://en.wikipedia.org/wiki/imperative_programming" target="_blank" rel="noopener">imperative programming</a> language with support for coroutines and channels. <a href="https://haskell.org" target="_blank" rel="noopener">Haskell</a> is our choice of language to write an interpreter for <span class="fancy">Co</span>.</p>
<h2 data-track-content data-content-name="introducing-co" data-content-piece="implementing-co-1" id="introducing-co">Introducing <span class="fancy">Co</span></h2>
<p><span class="fancy">Co</span> has these <span id="basic-features">basic features</span> that are found in many programming languages:</p>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Dynamic_typing" target="_blank" rel="noopener">Dynamic</a> and <a href="https://en.wikipedia.org/wiki/Strong_and_weak_typing" target="_blank" rel="noopener">strong</a> typing.</li>
<li>Null, boolean, string and integer literals, and values.</li>
<li>Addition, subtraction, multiplication and integer division arithmetic operations.</li>
<li>String concatenation operation.</li>
<li>Equality and inequality checks on booleans, strings and numbers.</li>
<li>Less-than and greater-than comparison operations on numbers.</li>
<li>Variable declarations, usage and assignments.</li>
<li><code class="sourceCode javascript"><span class="cf">if</span></code> and <code class="sourceCode javascript"><span class="cf">while</span></code> statements.</li>
<li>Function declarations and calls, with support for recursion.</li>
<li>First class functions and anonymous functions.</li>
<li>Mutable closures.</li>
</ul>
<p>It also has these special features:</p>
<ul>
<li><code class="sourceCode javascript"><span class="kw">yield</span></code> statement to yield the current thread of computation (ToC).</li>
<li><code class="sourceCode"><span class="cf">spawn</span></code> statement to start a new <abbr title="Thread of computation">ToC</abbr>.</li>
<li>First class channels with operators to send and receive values over them.</li>
<li><code class="sourceCode"><span class="cf">sleep</span></code> function to sleep the current <abbr title="Thread of computation">ToC</abbr> for a given number of milliseconds.</li>
</ul>
<p>Let’s see some example code in <span class="fancy">Co</span> for illustration:</p>
<div class="sourceCode" id="cb1" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Fibonacci numbers</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="co">// using a while loop</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> a <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> b <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> j <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> temp <span class="op">=</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="cf">while</span> (j <span class="op"><</span> <span class="dv">6</span>) {</span>
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(a)<span class="op">;</span></span>
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> temp <span class="op">=</span> a<span class="op">;</span></span>
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> a <span class="op">=</span> b<span class="op">;</span></span>
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> b <span class="op">=</span> temp <span class="op">+</span> b<span class="op">;</span></span>
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> j <span class="op">=</span> j <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a><span class="co">// Fibonacci numbers</span></span>
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a><span class="co">// using recursive function call</span></span>
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">fib</span>(n) {</span>
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (n <span class="op"><</span> <span class="dv">2</span>) {</span>
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> n<span class="op">;</span></span>
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb1-21"><a href="#cb1-21" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="fu">fib</span>(n <span class="op">-</span> <span class="dv">2</span>)</span>
<span id="cb1-22"><a href="#cb1-22" aria-hidden="true" tabindex="-1"></a> <span class="op">+</span> <span class="fu">fib</span>(n <span class="op">-</span> <span class="dv">1</span>)<span class="op">;</span></span>
<span id="cb1-23"><a href="#cb1-23" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb1-24"><a href="#cb1-24" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb1-25"><a href="#cb1-25" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> i <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb1-26"><a href="#cb1-26" aria-hidden="true" tabindex="-1"></a><span class="cf">while</span> (i <span class="op"><</span> <span class="dv">6</span>) {</span>
<span id="cb1-27"><a href="#cb1-27" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="fu">fib</span>(i))<span class="op">;</span></span>
<span id="cb1-28"><a href="#cb1-28" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb1-29"><a href="#cb1-29" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
<p>As you may notice, <span class="fancy">Co</span>’s syntax is heavily inspired by <a href="https://en.wikipedia.org/wiki/JavaScript" target="_blank" rel="noopener">JavaScript</a>. The code example above computes and prints<a href="#fn3" class="footnote-ref" id="fnref3" role="doc-noteref"><sup>3</sup></a> the first six <a href="https://en.wikipedia.org/wiki/Fibonacci_numbers" target="_blank" rel="noopener">Fibonacci numbers</a> in two different ways, and demonstrates a number of features of <span class="fancy">Co</span>, including variable declarations and assignments, <code class="sourceCode javascript"><span class="cf">while</span></code> loops, <code class="sourceCode javascript"><span class="cf">if</span></code> conditions, and function declarations and calls along with recursion.</p>
<p>We can save the code in a file and run it with the <span class="fancy">Co</span> interpreter<a href="#fn4" class="footnote-ref" id="fnref4" role="doc-noteref"><sup>4</sup></a> to print this (correct) output:</p>
<pre class="plain"><code>0
1
1
2
3
5
0
1
1
2
3
5</code></pre>
<p>The next example shows the usage of coroutines in <span class="fancy">Co</span>:</p>
<div class="sourceCode" id="cb3" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">printNums</span>(start<span class="op">,</span> end) {</span>
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> start<span class="op">;</span></span>
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> end <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(i)<span class="op">;</span></span>
<span id="cb3-5"><a href="#cb3-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">yield</span><span class="op">;</span></span>
<span id="cb3-6"><a href="#cb3-6" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb3-7"><a href="#cb3-7" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb3-8"><a href="#cb3-8" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb3-9"><a href="#cb3-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb3-10"><a href="#cb3-10" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)<span class="op">;</span></span>
<span id="cb3-11"><a href="#cb3-11" aria-hidden="true" tabindex="-1"></a><span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)<span class="op">;</span></span></code></pre></div>
<p>Running this code with the interpreter prints this output:</p>
<pre class="plain"><code>11
1
12
2
13
3
14
4
15
16</code></pre>
<p>The <code>printNum</code> function prints numbers between the <code>start</code> and <code>end</code> arguments, but yields the <abbr title="Thread of computation">ToC</abbr> after each print. Notice how the prints are interleaved. This is because the two calls to the function <code>printNums</code> run concurrently in two separate coroutines.</p>
<p>The next example show the usage of channels in <span class="fancy">Co</span>:</p>
<div class="sourceCode" id="cb5" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> chan <span class="op">=</span> <span class="fu">newChannel</span>()<span class="op">;</span></span>
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-3"><a href="#cb5-3" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">player</span>(name) {</span>
<span id="cb5-4"><a href="#cb5-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> n <span class="op">=</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb5-5"><a href="#cb5-5" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (<span class="kw">true</span>) {</span>
<span id="cb5-6"><a href="#cb5-6" aria-hidden="true" tabindex="-1"></a> n <span class="op">=</span> <span class="op"><-</span> chan<span class="op">;</span></span>
<span id="cb5-7"><a href="#cb5-7" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (n <span class="op">==</span> <span class="st">"done"</span>) {</span>
<span id="cb5-8"><a href="#cb5-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" done"</span>)<span class="op">;</span></span>
<span id="cb5-9"><a href="#cb5-9" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span><span class="op">;</span></span>
<span id="cb5-10"><a href="#cb5-10" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb5-11"><a href="#cb5-11" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-12"><a href="#cb5-12" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" "</span> <span class="op">+</span> n)<span class="op">;</span></span>
<span id="cb5-13"><a href="#cb5-13" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (n <span class="op">==</span> <span class="dv">0</span>) {</span>
<span id="cb5-14"><a href="#cb5-14" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" done"</span>)<span class="op">;</span></span>
<span id="cb5-15"><a href="#cb5-15" aria-hidden="true" tabindex="-1"></a> <span class="st">"done"</span> <span class="op">-></span> chan<span class="op">;</span></span>
<span id="cb5-16"><a href="#cb5-16" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span><span class="op">;</span></span>
<span id="cb5-17"><a href="#cb5-17" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb5-18"><a href="#cb5-18" aria-hidden="true" tabindex="-1"></a> n <span class="op">-</span> <span class="dv">1</span> <span class="op">-></span> chan<span class="op">;</span></span>
<span id="cb5-19"><a href="#cb5-19" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb5-20"><a href="#cb5-20" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb5-21"><a href="#cb5-21" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb5-22"><a href="#cb5-22" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">player</span>(<span class="st">"ping"</span>)<span class="op">;</span></span>
<span id="cb5-23"><a href="#cb5-23" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">player</span>(<span class="st">"pong"</span>)<span class="op">;</span></span>
<span id="cb5-24"><a href="#cb5-24" aria-hidden="true" tabindex="-1"></a><span class="dv">10</span> <span class="op">-></span> chan<span class="op">;</span></span></code></pre></div>
<p>This is the popular <a href="https://web.archive.org/web/20210406064833/http://erlang.org/doc/getting_started/conc_prog.html" target="_blank" rel="noopener">Ping-pong</a> <a href="https://kotlinlang.org/docs/channels.html#channels-are-fair" target="_blank" rel="noopener">benchmark</a> for communication between <abbr title="Threads of computation">ToCs</abbr>. Here we use channels and coroutines for the same. Running this code prints this output:</p>
<pre class="plain"><code>ping 10
pong 9
ping 8
pong 7
ping 6
pong 5
ping 4
pong 3
ping 2
pong 1
ping 0
ping done
pong done</code></pre>
<p>Lastly, here is <a href="https://www.geeksforgeeks.org/sleep-sort-king-laziness-sorting-sleeping/" target="_blank" rel="noopener">Sleep sort</a> in <span class="fancy">Co</span>:</p>
<div class="sourceCode" id="cb7" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">sleepSort</span>(a<span class="op">,</span> b<span class="op">,</span> c<span class="op">,</span> d<span class="op">,</span> e) {</span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">function</span> <span class="fu">printNum</span>(num) {</span>
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">sleep</span>(num)<span class="op">;</span></span>
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(num)<span class="op">;</span></span>
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a> spawn <span class="fu">printNum</span>(a)<span class="op">;</span></span>
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a> spawn <span class="fu">printNum</span>(b)<span class="op">;</span></span>
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a> spawn <span class="fu">printNum</span>(c)<span class="op">;</span></span>
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a> spawn <span class="fu">printNum</span>(d)<span class="op">;</span></span>
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a> spawn <span class="fu">printNum</span>(e)<span class="op">;</span></span>
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb7-13"><a href="#cb7-13" aria-hidden="true" tabindex="-1"></a><span class="fu">sleepSort</span>(<span class="dv">5</span><span class="op">,</span> <span class="dv">4</span><span class="op">,</span> <span class="dv">3</span><span class="op">,</span> <span class="dv">2</span><span class="op">,</span> <span class="dv">1</span>)<span class="op">;</span></span></code></pre></div>
<p>Running this code prints this output:</p>
<pre class="plain"><code>1
2
3
4
5</code></pre>
<p>We’ll revisit these examples later while building our interpreter, and understand them as well as the code that implements them.</p>
<h2 data-track-content data-content-name="the-co-interpreter" data-content-piece="implementing-co-1" id="the-co-interpreter">The <span class="fancy">Co</span> Interpreter</h2>
<p>The <span class="fancy">Co</span> interpreter works in two stages:</p>
<ol type="1">
<li>Parsing: a parser converts <span class="fancy">Co</span> source code to <a href="https://en.wikipedia.org/wiki/Abstract_Syntax_Tree" target="_blank" rel="noopener"><em>Abstract Syntax Tree</em></a> (AST).</li>
<li>Interpretation: a <a href="https://en.wikipedia.org/wiki/Interpreter_(computing)#Abstract_syntax_tree_interpreters" target="_blank" rel="noopener">tree-walking interpreter</a> walks the AST, executes the instructions and produces the output.</li>
</ol>
<figure>
<img src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 521 65'%3E%3C/svg%3E" class="lazyload ascii-art w-80pct nolink" style="--image-aspect-ratio: 8.015384615384615" data-src="/images/implementing-co-1/stages.svg" alt="Stages of the Co interpreter"></img>
<noscript><img src="/images/implementing-co-1/stages.svg" class="ascii-art w-80pct nolink" alt="Stages of the Co interpreter"></img></noscript>
<figcaption>Stages of the <span class="fancy">Co</span> interpreter</figcaption>
</figure>
<p>In this post, we implement the parser for <span class="fancy">Co</span>. In the <a href="https://abhinavsarkar.net/posts/implementing-co-2/?mtm_campaign=feed">second part</a>, we create a first cut of the interpreter that supports the <a href="#basic-features">basic features</a> of <span class="fancy">Co</span>. In the third and fourth parts, we extend the interpreter to add support for coroutines and channels.</p>
<div class="note">
<p>The complete code for the parser is <a href="https://abhinavsarkar.net/code/co-parser.html?mtm_campaign=feed">here</a>. You can load it in GHCi using <a href="https://haskellstack.org/" target="_blank" rel="noopener">stack</a> (by running <code>stack co-interpreter.hs</code>), and follow along while reading this article<a href="#fn5" class="footnote-ref" id="fnref5" role="doc-noteref"><sup>5</sup></a>.</p>
</div>
<p>Let’s start with listing the extensions and imports needed<a href="#fn6" class="footnote-ref" id="fnref6" role="doc-noteref"><sup>6</sup></a>.</p>
<div class="sourceCode" id="cb10" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="ot">{-# LANGUAGE FlexibleContexts #-}</span></span>
<span id="cb10-2"><a href="#cb10-2" aria-hidden="true" tabindex="-1"></a><span class="ot">{-# LANGUAGE GeneralizedNewtypeDeriving #-}</span></span>
<span id="cb10-3"><a href="#cb10-3" aria-hidden="true" tabindex="-1"></a><span class="ot">{-# LANGUAGE LambdaCase #-}</span></span>
<span id="cb10-4"><a href="#cb10-4" aria-hidden="true" tabindex="-1"></a><span class="ot">{-# LANGUAGE RecordWildCards #-}</span></span>
<span id="cb10-5"><a href="#cb10-5" aria-hidden="true" tabindex="-1"></a><span class="kw">module</span> <span class="dt">CoInterpreter</span> <span class="kw">where</span></span>
<span id="cb10-6"><a href="#cb10-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb10-7"><a href="#cb10-7" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Concurrent</span> (forkIO, threadDelay)</span>
<span id="cb10-8"><a href="#cb10-8" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Concurrent.MVar.Lifted</span></span>
<span id="cb10-9"><a href="#cb10-9" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Monad</span> (foldM, unless, void, when)</span>
<span id="cb10-10"><a href="#cb10-10" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Monad.Base</span> (<span class="dt">MonadBase</span>, liftBase)</span>
<span id="cb10-11"><a href="#cb10-11" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Monad.Combinators.Expr</span> (<span class="dt">Operator</span> (..), makeExprParser)</span>
<span id="cb10-12"><a href="#cb10-12" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Monad.Cont</span> (<span class="dt">ContT</span>, <span class="dt">MonadCont</span>, callCC, runContT)</span>
<span id="cb10-13"><a href="#cb10-13" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Monad.Except</span> (<span class="dt">ExceptT</span>, <span class="dt">MonadError</span> (..), runExceptT)</span>
<span id="cb10-14"><a href="#cb10-14" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Monad.IO.Class</span> (<span class="dt">MonadIO</span>, liftIO)</span>
<span id="cb10-15"><a href="#cb10-15" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Control.Monad.State.Strict</span> (<span class="dt">MonadState</span>, <span class="dt">StateT</span>, evalStateT)</span>
<span id="cb10-16"><a href="#cb10-16" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Control.Monad.State.Strict</span> <span class="kw">as</span> <span class="dt">State</span></span>
<span id="cb10-17"><a href="#cb10-17" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Data.Foldable</span> (for_, traverse_)</span>
<span id="cb10-18"><a href="#cb10-18" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Data.IORef.Lifted</span></span>
<span id="cb10-19"><a href="#cb10-19" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Data.Map.Strict</span> <span class="kw">as</span> <span class="dt">Map</span></span>
<span id="cb10-20"><a href="#cb10-20" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Data.Maybe</span> (fromMaybe)</span>
<span id="cb10-21"><a href="#cb10-21" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Data.PQueue.Prio.Min</span> <span class="kw">as</span> <span class="dt">PQ</span></span>
<span id="cb10-22"><a href="#cb10-22" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Data.Time.Clock.POSIX</span> (getPOSIXTime)</span>
<span id="cb10-23"><a href="#cb10-23" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Data.Void</span> (<span class="dt">Void</span>)</span>
<span id="cb10-24"><a href="#cb10-24" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">System.Clock</span> (<span class="dt">Clock</span> (<span class="dt">Monotonic</span>), fromNanoSecs, getTime, <span class="dt">TimeSpec</span>)</span>
<span id="cb10-25"><a href="#cb10-25" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">System.Environment</span> (getArgs, getProgName)</span>
<span id="cb10-26"><a href="#cb10-26" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">System.IO</span> (hPutStrLn, stderr)</span>
<span id="cb10-27"><a href="#cb10-27" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Text.Megaparsec</span> <span class="kw">hiding</span> (runParser)</span>
<span id="cb10-28"><a href="#cb10-28" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Text.Megaparsec.Char</span></span>
<span id="cb10-29"><a href="#cb10-29" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="kw">qualified</span> <span class="dt">Text.Megaparsec.Char.Lexer</span> <span class="kw">as</span> <span class="dt">L</span></span>
<span id="cb10-30"><a href="#cb10-30" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="dt">Text.Pretty.Simple</span></span>
<span id="cb10-31"><a href="#cb10-31" aria-hidden="true" tabindex="-1"></a> ( <span class="dt">CheckColorTty</span> (<span class="op">..</span>),</span>
<span id="cb10-32"><a href="#cb10-32" aria-hidden="true" tabindex="-1"></a> <span class="dt">OutputOptions</span> (<span class="op">..</span>),</span>
<span id="cb10-33"><a href="#cb10-33" aria-hidden="true" tabindex="-1"></a> defaultOutputOptionsNoColor,</span>
<span id="cb10-34"><a href="#cb10-34" aria-hidden="true" tabindex="-1"></a> pPrintOpt,</span>
<span id="cb10-35"><a href="#cb10-35" aria-hidden="true" tabindex="-1"></a> )</span></code></pre></div>
<p>Next, let’s take a look at the <span class="fancy">Co</span> AST.</p>
<h2 data-track-content data-content-name="the-co-ast" data-content-piece="implementing-co-1" id="the-co-ast">The <span class="fancy">Co</span> AST</h2>
<p>Since <span class="fancy">Co</span> is an <a href="https://en.wikipedia.org/wiki/imperative_programming" target="_blank" rel="noopener">imperative programming</a> language, it is naturally <a href="https://en.wikipedia.org/wiki/Statement_(computer_science)" target="_blank" rel="noopener"><em>Statement</em></a> oriented. A statement describes how an action is to be executed. A <span class="fancy">Co</span> program is a list of top-level statements.</p>
<p>Statements have internal components called <a href="https://en.wikipedia.org/wiki/Expression_(computer_science)" target="_blank" rel="noopener"><em>Expressions</em></a>. Expressions evaluate to values at program run time. Let’s look at them first.</p>
<h3 id="expressions">Expressions</h3>
<p>We represent <span class="fancy">Co</span> expressions as a Haskell <a href="https://en.wikipedia.org/wiki/Algebraic_data_type" target="_blank" rel="noopener"><em>Algebraic data type</em></a> (ADT) with one constructor per expression type:</p>
<div class="sourceCode" id="cb11" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Expr</span></span>
<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">LNull</span></span>
<span id="cb11-3"><a href="#cb11-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LBool</span> <span class="dt">Bool</span></span>
<span id="cb11-4"><a href="#cb11-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LStr</span> <span class="dt">String</span></span>
<span id="cb11-5"><a href="#cb11-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LNum</span> <span class="dt">Integer</span></span>
<span id="cb11-6"><a href="#cb11-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Variable</span> <span class="dt">Identifier</span></span>
<span id="cb11-7"><a href="#cb11-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Binary</span> <span class="dt">BinOp</span> <span class="dt">Expr</span> <span class="dt">Expr</span></span>
<span id="cb11-8"><a href="#cb11-8" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Call</span> <span class="dt">Expr</span> [<span class="dt">Expr</span>]</span>
<span id="cb11-9"><a href="#cb11-9" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Lambda</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>]</span>
<span id="cb11-10"><a href="#cb11-10" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Receive</span> <span class="dt">Expr</span></span>
<span id="cb11-11"><a href="#cb11-11" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Eq</span>)</span>
<span id="cb11-12"><a href="#cb11-12" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb11-13"><a href="#cb11-13" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Identifier</span> <span class="ot">=</span> <span class="dt">String</span></span></code></pre></div>
<dl>
<dt><code class="sourceCode haskell"><span class="dt">LNull</span></code></dt>
<dd>
<p>The literal <code class="sourceCode javascript"><span class="kw">null</span></code>. Evaluates to the null value.</p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">LBool</span> <span class="dt">Bool</span></code></dt>
<dd>
<p>The boolean literals, <code class="sourceCode javascript"><span class="kw">true</span></code> and <code class="sourceCode javascript"><span class="kw">false</span></code>. Evaluate to their counterpart boolean values.</p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">LStr</span> <span class="dt">String</span></code></dt>
<dd>
<p>A string literal like <code class="sourceCode javascript"><span class="st">"towel"</span></code>. Evaluates to a string.</p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">LNum</span> <span class="dt">Integer</span></code></dt>
<dd>
<p>An integer literal like <code class="sourceCode javascript"><span class="dv">1</span></code> or <code class="sourceCode javascript"><span class="op">-</span><span class="dv">5</span></code>. Evaluates to an integer.</p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">Variable</span> <span class="dt">Identifier</span></code></dt>
<dd>
<p>A variable named by an identifier like <code>a1</code> or <code>sender</code>. Evaluates to the variable’s value at the point in the execution of code. An <code class="sourceCode haskell"><span class="dt">Identifier</span></code> is a string of alphanumeric characters, starting with an alpha character.</p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">Binary</span> <span class="dt">Op</span> <span class="dt">Expr</span> <span class="dt">Expr</span></code></dt>
<dd>
<p>A binary operation on two expressions. Example: <code class="sourceCode javascript"><span class="dv">1</span> <span class="op">+</span> <span class="dv">41</span></code> or <code class="sourceCode javascript"><span class="dv">2</span> <span class="op">==</span> <span class="st">"2"</span></code>. Supported binary operations are defined by the <code class="sourceCode haskell"><span class="dt">BinOp</span></code> enum: addition/concatenation, subtraction, multiplication, integer division, equality and inequality checks, and less-than and greater-than comparisons.</p>
</dd>
</dl>
<div class="sourceCode" id="cb12" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">BinOp</span> <span class="ot">=</span></span>
<span id="cb12-2"><a href="#cb12-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">Plus</span></span>
<span id="cb12-3"><a href="#cb12-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Minus</span></span>
<span id="cb12-4"><a href="#cb12-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Slash</span></span>
<span id="cb12-5"><a href="#cb12-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Star</span></span>
<span id="cb12-6"><a href="#cb12-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">Equals</span></span>
<span id="cb12-7"><a href="#cb12-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">NotEquals</span></span>
<span id="cb12-8"><a href="#cb12-8" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">LessThan</span></span>
<span id="cb12-9"><a href="#cb12-9" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">GreaterThan</span></span>
<span id="cb12-10"><a href="#cb12-10" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Eq</span>)</span></code></pre></div>
<dl>
<dt><code class="sourceCode haskell"><span class="dt">Call</span> <span class="dt">Expr</span> [<span class="dt">Expr</span>]</code></dt>
<dd>
<p>Calls the function obtained by evaluating the callee expression, with the given argument expressions. Examples: <code class="sourceCode javascript"><span class="fu">calcDistance</span>(start<span class="op">,</span> end)</code> or <code class="sourceCode javascript"><span class="fu">getResolver</span>(context)(template)</code>.</p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">Lambda</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>]</code></dt>
<dd>
<p>A function with the given parameter names and body statements. Example: <code class="sourceCode javascript"><span class="kw">function</span> (a<span class="op">,</span> b) { <span class="cf">return</span> a<span class="op">*</span>a <span class="op">+</span> b<span class="op">*</span>b<span class="op">;</span> }</code>.</p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">Receive</span> <span class="dt">Expr</span></code></dt>
<dd>
<p>Receives a value from a channel. Examples:</p>
<div class="sourceCode" id="cb13" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="co">// receive a value from the channel and prints it</span></span>
<span id="cb13-2"><a href="#cb13-2" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="op"><-</span> someChannel)<span class="op">;</span></span>
<span id="cb13-3"><a href="#cb13-3" aria-hidden="true" tabindex="-1"></a><span class="co">// receive a value from the channel and assigns it</span></span>
<span id="cb13-4"><a href="#cb13-4" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> x <span class="op">=</span> <span class="op"><-</span> someChannel<span class="op">;</span></span></code></pre></div>
</dd>
</dl>
<p>Next, we see how statements are represented in the AST.</p>
<h3 id="statements">Statements</h3>
<p>We represent the <span class="fancy">Co</span> statements as a Haskell ADT with one constructor per statement type:</p>
<div class="sourceCode" id="cb14" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="kw">data</span> <span class="dt">Stmt</span></span>
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true" tabindex="-1"></a> <span class="ot">=</span> <span class="dt">ExprStmt</span> <span class="dt">Expr</span></span>
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">VarStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></span>
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">AssignStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></span>
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">IfStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</span>
<span id="cb14-6"><a href="#cb14-6" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">WhileStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</span>
<span id="cb14-7"><a href="#cb14-7" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">FunctionStmt</span> <span class="dt">Identifier</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>]</span>
<span id="cb14-8"><a href="#cb14-8" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">ReturnStmt</span> (<span class="dt">Maybe</span> <span class="dt">Expr</span>)</span>
<span id="cb14-9"><a href="#cb14-9" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">YieldStmt</span></span>
<span id="cb14-10"><a href="#cb14-10" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">SpawnStmt</span> <span class="dt">Expr</span></span>
<span id="cb14-11"><a href="#cb14-11" aria-hidden="true" tabindex="-1"></a> <span class="op">|</span> <span class="dt">SendStmt</span> <span class="dt">Expr</span> <span class="dt">Expr</span></span>
<span id="cb14-12"><a href="#cb14-12" aria-hidden="true" tabindex="-1"></a> <span class="kw">deriving</span> (<span class="dt">Show</span>, <span class="dt">Eq</span>)</span>
<span id="cb14-13"><a href="#cb14-13" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb14-14"><a href="#cb14-14" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Program</span> <span class="ot">=</span> [<span class="dt">Stmt</span>]</span></code></pre></div>
<dl>
<dt><code class="sourceCode haskell"><span class="dt">ExprStmt</span> <span class="dt">Expr</span></code></dt>
<dd>
<p>A statement with just an expression. The expression’s value is thrown away after executing the statement. Example: <code class="sourceCode javascript"><span class="dv">1</span> <span class="op">+</span> <span class="dv">2</span><span class="op">;</span></code></p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">VarStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></code></dt>
<dd>
<p>Defines a new variable named by an identifier and sets it to an expression’s value. In <span class="fancy">Co</span>, variables must be initialized when being defined. Example: <code class="sourceCode javascript"><span class="kw">var</span> a <span class="op">=</span> <span class="dv">5</span><span class="op">;</span></code></p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">AssignStmt</span> <span class="dt">Identifier</span> <span class="dt">Expr</span></code></dt>
<dd>
<p>Sets an already defined variable named by an identifier to an expression’s value. Example: <code class="sourceCode javascript">a <span class="op">=</span> <span class="dv">55</span> <span class="op">+</span> <span class="st">"hello"</span><span class="op">;</span></code></p>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">IfStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</code></dt>
<dd>
<p>Executes a list of statements if the condition expression evaluates to a truthy value. In <span class="fancy">Co</span>, <code class="sourceCode javascript"><span class="kw">null</span></code> and <code class="sourceCode javascript"><span class="kw">false</span></code> values are non-truthy, and every other value is truthy. Also note that, there are no <code class="sourceCode javascript"><span class="cf">else</span></code> branches for <code class="sourceCode javascript"><span class="cf">if</span></code> statements in <span class="fancy">Co</span>. Example:</p>
<div class="sourceCode" id="cb15" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="cf">if</span> (a <span class="op">==</span> <span class="dv">1</span>) {</span>
<span id="cb15-2"><a href="#cb15-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"hello"</span>)<span class="op">;</span></span>
<span id="cb15-3"><a href="#cb15-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">WhileStmt</span> <span class="dt">Expr</span> [<span class="dt">Stmt</span>]</code></dt>
<dd>
<p>Executes a list of statements repeatedly while the condition expression evaluates to a truthy value. Example:</p>
<div class="sourceCode" id="cb16" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb16-1"><a href="#cb16-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> n <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb16-2"><a href="#cb16-2" aria-hidden="true" tabindex="-1"></a><span class="cf">while</span> (n <span class="op"><</span> <span class="dv">5</span>) {</span>
<span id="cb16-3"><a href="#cb16-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(n)<span class="op">;</span></span>
<span id="cb16-4"><a href="#cb16-4" aria-hidden="true" tabindex="-1"></a> n <span class="op">=</span> n <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb16-5"><a href="#cb16-5" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">FunctionStmt</span> <span class="dt">Identifier</span> [<span class="dt">Identifier</span>] [<span class="dt">Stmt</span>]</code></dt>
<dd>
<p>Defines a function with a name, a list of parameter names, and a list of body statements. Example:</p>
<div class="sourceCode" id="cb17" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">greet</span>(greeting) {</span>
<span id="cb17-2"><a href="#cb17-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(greeting <span class="op">+</span> <span class="st">" world"</span>)<span class="op">;</span></span>
<span id="cb17-3"><a href="#cb17-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">ReturnStmt</span> (<span class="dt">Maybe</span> <span class="dt">Expr</span>)</code></dt>
<dd>
<p>Returns from a function, optionally returning an expression’s value. Example:</p>
<div class="sourceCode" id="cb18" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb18-1"><a href="#cb18-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">square</span>(x) {</span>
<span id="cb18-2"><a href="#cb18-2" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> x <span class="op">*</span> x<span class="op">;</span></span>
<span id="cb18-3"><a href="#cb18-3" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">YieldStmt</span></code></dt>
<dd>
<p>Suspends the currently executing <abbr title="Thread of computation">ToC</abbr> so that some other <abbr title="Thread of computation">ToC</abbr> may run. The current <abbr title="Thread of computation">ToC</abbr> resumes later from statement next to the <code class="sourceCode javascript"><span class="kw">yield</span></code> statement. Example:</p>
<div class="sourceCode" id="cb19" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb19-1"><a href="#cb19-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">printNums</span>(start<span class="op">,</span> end) {</span>
<span id="cb19-2"><a href="#cb19-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> start<span class="op">;</span></span>
<span id="cb19-3"><a href="#cb19-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> end <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb19-4"><a href="#cb19-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(i)<span class="op">;</span></span>
<span id="cb19-5"><a href="#cb19-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">yield</span><span class="op">;</span></span>
<span id="cb19-6"><a href="#cb19-6" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb19-7"><a href="#cb19-7" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb19-8"><a href="#cb19-8" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">SpawnStmt</span> <span class="dt">Expr</span></code></dt>
<dd>
<p>Starts a new <abbr title="Thread of computation">ToC</abbr> in which the given expression is evaluated, which runs concurrently with all other running <abbr title="Threads of computation">ToCs</abbr>. Example:</p>
<div class="sourceCode" id="cb20" data-lang="co"><pre class="sourceCode javascript numberSource"><code class="sourceCode javascript"><span id="cb20-1"><a href="#cb20-1" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)<span class="op">;</span></span>
<span id="cb20-2"><a href="#cb20-2" aria-hidden="true" tabindex="-1"></a><span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)<span class="op">;</span></span></code></pre></div>
</dd>
<dt><code class="sourceCode haskell"><span class="dt">SendStmt</span> <span class="dt">Expr</span> <span class="dt">Expr</span></code></dt>
<dd>
<p>Sends an expressions’s value to a channel. Example:</p>
<div class="sourceCode" id="cb21"><pre class="sourceCode numberSource js"><code class="sourceCode javascript"><span id="cb21-1"><a href="#cb21-1" aria-hidden="true" tabindex="-1"></a><span class="dv">1</span> <span class="op">+</span> <span class="fu">square</span>(<span class="dv">3</span>) <span class="op">-></span> someChannel<span class="op">;</span></span></code></pre></div>
</dd>
</dl>
<p>The <span class="fancy">Co</span> AST is minimal but it serves our purpose. Next, let’s figure out how to actually parse source code to AST.</p>
<h2 data-track-content data-content-name="parsing" data-content-piece="implementing-co-1" id="parsing">Parsing</h2>
<p>Parsing is the process of taking textual input data and converting it to a data structure—often a hierarchal structure like AST—while checking the input for correct syntax. There are many ways of writing parsers<a href="#fn7" class="footnote-ref" id="fnref7" role="doc-noteref"><sup>7</sup></a>, <a href="https://en.wikipedia.org/wiki/Parser_Combinators" target="_blank" rel="noopener"><em>Parser Combinators</em></a><sup><a href="#ref-Hutton1992-vc" class="citation" title="Hutton, “Higher-Order Functions for Parsing.”
">@11</a></sup> being one of them. Parser combinators <a href="https://hackage.haskell.org/packages/#cat:Parsing" target="_blank" rel="noopener">libraries</a> are popular in Haskell because of their ease of use and succinctness. We are going to use one such library, <a href="https://hackage.haskell.org/package/megaparsec" target="_blank" rel="noopener">Megaparsec</a>, to create a parser for <span class="fancy">Co</span>.</p>
<p>Let’s start with writing some basic parsers for the <span class="fancy">Co</span> syntax using the Megaparsec parsers.</p>
<div class="sourceCode" id="cb22" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb22-1"><a href="#cb22-1" aria-hidden="true" tabindex="-1"></a><span class="kw">type</span> <span class="dt">Parser</span> <span class="ot">=</span> <span class="dt">Parsec</span> <span class="dt">Void</span> <span class="dt">String</span></span>
<span id="cb22-2"><a href="#cb22-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb22-3"><a href="#cb22-3" aria-hidden="true" tabindex="-1"></a><span class="ot">sc ::</span> <span class="dt">Parser</span> ()</span>
<span id="cb22-4"><a href="#cb22-4" aria-hidden="true" tabindex="-1"></a>sc <span class="ot">=</span> L.space space1 lineCmnt blockCmnt</span>
<span id="cb22-5"><a href="#cb22-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb22-6"><a href="#cb22-6" aria-hidden="true" tabindex="-1"></a> lineCmnt <span class="ot">=</span> L.skipLineComment <span class="st">"//"</span></span>
<span id="cb22-7"><a href="#cb22-7" aria-hidden="true" tabindex="-1"></a> blockCmnt <span class="ot">=</span> L.skipBlockComment <span class="st">"/*"</span> <span class="st">"*/"</span></span>
<span id="cb22-8"><a href="#cb22-8" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb22-9"><a href="#cb22-9" aria-hidden="true" tabindex="-1"></a><span class="ot">lexeme ::</span> <span class="dt">Parser</span> a <span class="ot">-></span> <span class="dt">Parser</span> a</span>
<span id="cb22-10"><a href="#cb22-10" aria-hidden="true" tabindex="-1"></a>lexeme <span class="ot">=</span> L.lexeme sc</span>
<span id="cb22-11"><a href="#cb22-11" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb22-12"><a href="#cb22-12" aria-hidden="true" tabindex="-1"></a><span class="ot">symbol ::</span> <span class="dt">String</span> <span class="ot">-></span> <span class="dt">Parser</span> <span class="dt">String</span></span>
<span id="cb22-13"><a href="#cb22-13" aria-hidden="true" tabindex="-1"></a>symbol <span class="ot">=</span> L.symbol sc</span>
<span id="cb22-14"><a href="#cb22-14" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb22-15"><a href="#cb22-15" aria-hidden="true" tabindex="-1"></a><span class="ot">reserved ::</span> <span class="dt">String</span> <span class="ot">-></span> <span class="dt">Parser</span> ()</span>
<span id="cb22-16"><a href="#cb22-16" aria-hidden="true" tabindex="-1"></a>reserved w <span class="ot">=</span> (lexeme <span class="op">.</span> try) <span class="op">$</span> string w <span class="op">*></span> notFollowedBy alphaNumChar</span>
<span id="cb22-17"><a href="#cb22-17" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb22-18"><a href="#cb22-18" aria-hidden="true" tabindex="-1"></a>parens,<span class="ot"> braces ::</span> <span class="dt">Parser</span> a <span class="ot">-></span> <span class="dt">Parser</span> a</span>
<span id="cb22-19"><a href="#cb22-19" aria-hidden="true" tabindex="-1"></a>parens <span class="ot">=</span> between (symbol <span class="st">"("</span>) (symbol <span class="st">")"</span>)</span>
<span id="cb22-20"><a href="#cb22-20" aria-hidden="true" tabindex="-1"></a>braces <span class="ot">=</span> between (symbol <span class="st">"{"</span>) (symbol <span class="st">"}"</span>)</span>
<span id="cb22-21"><a href="#cb22-21" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb22-22"><a href="#cb22-22" aria-hidden="true" tabindex="-1"></a>semi, identifier,<span class="ot"> stringLiteral ::</span> <span class="dt">Parser</span> <span class="dt">String</span></span>
<span id="cb22-23"><a href="#cb22-23" aria-hidden="true" tabindex="-1"></a>semi <span class="ot">=</span> symbol <span class="st">";"</span></span>
<span id="cb22-24"><a href="#cb22-24" aria-hidden="true" tabindex="-1"></a>identifier <span class="ot">=</span> lexeme ((<span class="op">:</span>) <span class="op"><$></span> letterChar <span class="op"><*></span> many alphaNumChar)</span>
<span id="cb22-25"><a href="#cb22-25" aria-hidden="true" tabindex="-1"></a>stringLiteral <span class="ot">=</span> char <span class="ch">'"'</span> <span class="op">>></span> manyTill L.charLiteral (char <span class="ch">'"'</span>) <span class="op"><*</span> sc</span>
<span id="cb22-26"><a href="#cb22-26" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb22-27"><a href="#cb22-27" aria-hidden="true" tabindex="-1"></a><span class="ot">integer ::</span> <span class="dt">Parser</span> <span class="dt">Integer</span></span>
<span id="cb22-28"><a href="#cb22-28" aria-hidden="true" tabindex="-1"></a>integer <span class="ot">=</span> lexeme (L.signed sc L.decimal)</span></code></pre></div>
<p>In the above code, <code class="sourceCode haskell"><span class="dt">Parser</span></code> is a concise type-alias for our parser type<a href="#fn8" class="footnote-ref" id="fnref8" role="doc-noteref"><sup>8</sup></a>.</p>
<p>The <code>sc</code> parser is the space consumer parser. It dictates what’s ignored while parsing the sources code. For <span class="fancy">Co</span>, we consider the Unicode space character and the control characters—tab, newline, carriage return, form feed, and vertical tab—as whitespaces. We use the <a href="https://hackage.haskell.org/package/megaparsec-9.0.1/docs/Text-Megaparsec-Char.html#v:space1" target="_blank" rel="noopener"><code>space1</code></a> parser to configure that. <code>sc</code> also lets us configure how to ignore line comments and block comments while parsing.</p>
<p>The <code>lexeme</code> combinator is for parsing <a href="https://en.wikipedia.org/wiki/Lexemes" target="_blank" rel="noopener"><em>Lexemes</em></a> while ignoring whitespaces and comments. It is implemented using the <a href="https://hackage.haskell.org/package/megaparsec-9.0.1/docs/Text-Megaparsec-Char-Lexer.html#v:lexeme" target="_blank" rel="noopener"><code>lexeme</code></a> combinator from Megaparsec.</p>
<p>The <code>symbol</code> combinator is for parsing symbols, that is, operators like <code class="sourceCode javascript"><span class="op">*</span></code> and <code class="sourceCode javascript"><span class="op">;</span></code>. It is implemented using the <a href="https://hackage.haskell.org/package/megaparsec-9.0.1/docs/Text-Megaparsec-Char-Lexer.html#v:symbol" target="_blank" rel="noopener"><code>symbol</code></a> combinator.</p>
<p>The <code>reserved</code> combinator is for parsing reserved keywords like <code class="sourceCode javascript"><span class="cf">if</span></code> and <code class="sourceCode javascript"><span class="cf">while</span></code>. It is implemented as a combination of <code>lexeme</code> combinator and <code>string</code> parser, while making sure that the reserved keyword is not a prefix of another word. It uses the <code>try</code> combinator to backtrack if that happens.</p>
<p>The <code>parens</code> and <code>braces</code> combinators are for parsing code surrounded by parentheses <code>(</code> and <code>)</code> and braces <code>{</code> and <code>}</code> respectively.</p>
<p>The <code class="sourceCode haskell">semi</code> parser matches a semicolon <code class="sourceCode javascript"><span class="op">;</span></code>. The <code>identifier</code> parser is for parsing identifiers in <span class="fancy">Co</span>. The <code>stringLiteral</code> parser matches a string literal like <code class="sourceCode javascript"><span class="st">"polyphonic ringtone"</span></code>. <code>integer</code> is the parser for <span class="fancy">Co</span> integers.</p>
<p>Let’s also write some functions to run the parsers and pretty-print the output in GHCi:</p>
<div class="sourceCode" id="cb23" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb23-1"><a href="#cb23-1" aria-hidden="true" tabindex="-1"></a><span class="ot">runParser ::</span> <span class="dt">Parser</span> a <span class="ot">-></span> <span class="dt">String</span> <span class="ot">-></span> <span class="dt">Either</span> <span class="dt">String</span> a</span>
<span id="cb23-2"><a href="#cb23-2" aria-hidden="true" tabindex="-1"></a>runParser parser code <span class="ot">=</span> <span class="kw">do</span></span>
<span id="cb23-3"><a href="#cb23-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">case</span> parse parser <span class="st">""</span> code <span class="kw">of</span></span>
<span id="cb23-4"><a href="#cb23-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">Left</span> err <span class="ot">-></span> <span class="dt">Left</span> <span class="op">$</span> errorBundlePretty err</span>
<span id="cb23-5"><a href="#cb23-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">Right</span> prog <span class="ot">-></span> <span class="dt">Right</span> prog</span>
<span id="cb23-6"><a href="#cb23-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb23-7"><a href="#cb23-7" aria-hidden="true" tabindex="-1"></a><span class="ot">pPrint ::</span> (<span class="dt">MonadIO</span> m, <span class="dt">Show</span> a) <span class="ot">=></span> a <span class="ot">-></span> m ()</span>
<span id="cb23-8"><a href="#cb23-8" aria-hidden="true" tabindex="-1"></a>pPrint <span class="ot">=</span></span>
<span id="cb23-9"><a href="#cb23-9" aria-hidden="true" tabindex="-1"></a> pPrintOpt <span class="dt">CheckColorTty</span> <span class="op">$</span></span>
<span id="cb23-10"><a href="#cb23-10" aria-hidden="true" tabindex="-1"></a> defaultOutputOptionsNoColor</span>
<span id="cb23-11"><a href="#cb23-11" aria-hidden="true" tabindex="-1"></a> { outputOptionsIndentAmount <span class="ot">=</span> <span class="dv">2</span>,</span>
<span id="cb23-12"><a href="#cb23-12" aria-hidden="true" tabindex="-1"></a> outputOptionsCompact <span class="ot">=</span> <span class="dt">True</span>,</span>
<span id="cb23-13"><a href="#cb23-13" aria-hidden="true" tabindex="-1"></a> outputOptionsCompactParens <span class="ot">=</span> <span class="dt">True</span></span>
<span id="cb23-14"><a href="#cb23-14" aria-hidden="true" tabindex="-1"></a> }</span></code></pre></div>
<p>That completes our basic setup for parsing. Let’s try them out in GHCi now:</p>
<div class="sourceCode" id="cb24" data-lang="ghci"><pre class="sourceCode lhs numberSource"><code class="sourceCode literatehaskell"><span id="cb24-1"><a href="#cb24-1" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> runParser identifier <span class="st">"num1 "</span></span>
<span id="cb24-2"><a href="#cb24-2" aria-hidden="true" tabindex="-1"></a>Right "num1"</span>
<span id="cb24-3"><a href="#cb24-3" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> runParser stringLiteral <span class="st">"\"val\" "</span></span>
<span id="cb24-4"><a href="#cb24-4" aria-hidden="true" tabindex="-1"></a>Right "val"</span>
<span id="cb24-5"><a href="#cb24-5" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> runParser integer <span class="st">"1 "</span></span>
<span id="cb24-6"><a href="#cb24-6" aria-hidden="true" tabindex="-1"></a>Right 1</span>
<span id="cb24-7"><a href="#cb24-7" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> runParser integer <span class="st">"-12 "</span></span>
<span id="cb24-8"><a href="#cb24-8" aria-hidden="true" tabindex="-1"></a>Right (-12)</span></code></pre></div>
<p>They work as expected. Next, off to parsing <span class="fancy">Co</span> expressions.</p>
<h3 id="parsing-expressions">Parsing Expressions</h3>
<p>Parsing <span class="fancy">Co</span> expressions to AST requires us to handle the <a href="https://en.wikipedia.org/wiki/Operator_associativity" target="_blank" rel="noopener"><em>Associativity</em></a> and <a href="https://en.wikipedia.org/wiki/Order_of_operations" target="_blank" rel="noopener"><em>Precedence</em></a> of the operators. Fortunately, Megaparsec makes it easy with the <a href="https://hackage.haskell.org/package/parser-combinators-1.3.0/docs/Control-Monad-Combinators-Expr.html#v:makeExprParser" target="_blank" rel="noopener"><code>makeExprParser</code></a> combinator. <code>makeExprParser</code> takes a parser to parse the terms and a table of operators, and creates the expression parser for us.</p>
<ul>
<li>Terms are parts of expressions which cannot be broken down further into sub-expressions. Examples in <span class="fancy">Co</span> are literals, variables, groupings and function calls.</li>
<li>The table of operators is a list of <a href="https://hackage.haskell.org/package/parser-combinators-1.3.0/docs/Control-Monad-Combinators-Expr.html#t:Operator" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">Operator</span></code></a> <code class="sourceCode haskell"><span class="dt">Parser</span> <span class="dt">Expr</span></code> lists ordered in descending precedence. All operators in one list have the same precedence but may have different associativity.</li>
</ul>
<p>This is a lot to take in but looking at the code makes it clear. First, the operator table:</p>
<div class="sourceCode" id="cb25" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb25-1"><a href="#cb25-1" aria-hidden="true" tabindex="-1"></a><span class="ot">operators ::</span> [[<span class="dt">Operator</span> <span class="dt">Parser</span> <span class="dt">Expr</span>]]</span>
<span id="cb25-2"><a href="#cb25-2" aria-hidden="true" tabindex="-1"></a>operators <span class="ot">=</span></span>
<span id="cb25-3"><a href="#cb25-3" aria-hidden="true" tabindex="-1"></a> [ [<span class="dt">Prefix</span> <span class="op">$</span> <span class="dt">Receive</span> <span class="op"><$</span> symbol <span class="st">"<-"</span>],</span>
<span id="cb25-4"><a href="#cb25-4" aria-hidden="true" tabindex="-1"></a> [ binary <span class="dt">Slash</span> <span class="op">$</span> symbol <span class="st">"/"</span>,</span>
<span id="cb25-5"><a href="#cb25-5" aria-hidden="true" tabindex="-1"></a> binary <span class="dt">Star</span> <span class="op">$</span> symbol <span class="st">"*"</span>],</span>
<span id="cb25-6"><a href="#cb25-6" aria-hidden="true" tabindex="-1"></a> [ binary <span class="dt">Plus</span> <span class="op">$</span> symbol <span class="st">"+"</span>,</span>
<span id="cb25-7"><a href="#cb25-7" aria-hidden="true" tabindex="-1"></a> binary <span class="dt">Minus</span> <span class="op">$</span> try (symbol <span class="st">"-"</span> <span class="op"><*</span> notFollowedBy (char <span class="ch">'>'</span>))</span>
<span id="cb25-8"><a href="#cb25-8" aria-hidden="true" tabindex="-1"></a> ],</span>
<span id="cb25-9"><a href="#cb25-9" aria-hidden="true" tabindex="-1"></a> [ binary <span class="dt">LessThan</span> <span class="op">$</span> symbol <span class="st">"<"</span>,</span>
<span id="cb25-10"><a href="#cb25-10" aria-hidden="true" tabindex="-1"></a> binary <span class="dt">GreaterThan</span> <span class="op">$</span> symbol <span class="st">">"</span></span>
<span id="cb25-11"><a href="#cb25-11" aria-hidden="true" tabindex="-1"></a> ],</span>
<span id="cb25-12"><a href="#cb25-12" aria-hidden="true" tabindex="-1"></a> [ binary <span class="dt">Equals</span> <span class="op">$</span> symbol <span class="st">"=="</span>,</span>
<span id="cb25-13"><a href="#cb25-13" aria-hidden="true" tabindex="-1"></a> binary <span class="dt">NotEquals</span> <span class="op">$</span> symbol <span class="st">"!="</span></span>
<span id="cb25-14"><a href="#cb25-14" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb25-15"><a href="#cb25-15" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb25-16"><a href="#cb25-16" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb25-17"><a href="#cb25-17" aria-hidden="true" tabindex="-1"></a> binary op symP <span class="ot">=</span> <span class="dt">InfixL</span> <span class="op">$</span> <span class="dt">Binary</span> op <span class="op"><$</span> symP</span></code></pre></div>
<p>The prefix operator <code class="sourceCode javascript"><span class="op"><-</span></code>, for receiving values from channels, is of highest precedence and hence the first in the table. Next are the binary operators <code class="sourceCode haskell"><span class="op">/</span></code> and <code class="sourceCode haskell"><span class="op">*</span></code> for integer division and multiplication respectively. Following it, are the binary <code class="sourceCode javascript"><span class="op">+</span></code> and <code class="sourceCode javascript"><span class="op">-</span></code> operators which are of same precedence. After that come the comparison operators <code class="sourceCode javascript"><span class="op"><</span></code> and <code class="sourceCode javascript"><span class="op">></span></code>. Finally, we have the lowest precedence operators, the equality and inequality checks <code class="sourceCode javascript"><span class="op">==</span></code> and <code class="sourceCode javascript"><span class="op">!=</span></code>.</p>
<p>Each operator also contains the parser to parse the operator symbol in the source code. All of them are self-explanatory except the parser for the <code class="sourceCode javascript"><span class="op">-</span></code> operator. The <code class="sourceCode javascript"><span class="op">-</span></code> parser is slightly complicated because we want it to not parse <code class="sourceCode javascript"><span class="op">-></span></code>, the channel send operator, the first character of which is same as the symbol for the <code class="sourceCode javascript"><span class="op">-</span></code> operator.</p>
<p>Moving on to the term parser next:</p>
<div class="sourceCode" id="cb26" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb26-1"><a href="#cb26-1" aria-hidden="true" tabindex="-1"></a><span class="ot">term ::</span> <span class="dt">Parser</span> <span class="dt">Expr</span></span>
<span id="cb26-2"><a href="#cb26-2" aria-hidden="true" tabindex="-1"></a>term <span class="ot">=</span> primary <span class="op">>>=</span> call</span>
<span id="cb26-3"><a href="#cb26-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">where</span></span>
<span id="cb26-4"><a href="#cb26-4" aria-hidden="true" tabindex="-1"></a> call e <span class="ot">=</span></span>
<span id="cb26-5"><a href="#cb26-5" aria-hidden="true" tabindex="-1"></a> ( lookAhead (symbol <span class="st">"("</span>)</span>
<span id="cb26-6"><a href="#cb26-6" aria-hidden="true" tabindex="-1"></a> <span class="op">>></span> symbol <span class="st">"("</span></span>
<span id="cb26-7"><a href="#cb26-7" aria-hidden="true" tabindex="-1"></a> <span class="op">>></span> <span class="dt">Call</span> e <span class="op"><$></span> sepBy expr (symbol <span class="st">","</span>) <span class="op"><*</span> symbol <span class="st">")"</span></span>
<span id="cb26-8"><a href="#cb26-8" aria-hidden="true" tabindex="-1"></a> <span class="op">>>=</span> call )</span>
<span id="cb26-9"><a href="#cb26-9" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="fu">pure</span> e</span>
<span id="cb26-10"><a href="#cb26-10" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb26-11"><a href="#cb26-11" aria-hidden="true" tabindex="-1"></a> primary <span class="ot">=</span> <span class="dt">LNull</span> <span class="op"><$</span> reserved <span class="st">"null"</span></span>
<span id="cb26-12"><a href="#cb26-12" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">LBool</span> <span class="dt">True</span> <span class="op"><$</span> reserved <span class="st">"true"</span></span>
<span id="cb26-13"><a href="#cb26-13" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">LBool</span> <span class="dt">False</span> <span class="op"><$</span> reserved <span class="st">"false"</span></span>
<span id="cb26-14"><a href="#cb26-14" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">LStr</span> <span class="op"><$></span> stringLiteral</span>
<span id="cb26-15"><a href="#cb26-15" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">LNum</span> <span class="op"><$></span> integer</span>
<span id="cb26-16"><a href="#cb26-16" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">Lambda</span></span>
<span id="cb26-17"><a href="#cb26-17" aria-hidden="true" tabindex="-1"></a> <span class="op"><$></span> (reserved <span class="st">"function"</span> <span class="op">*></span> parens (sepBy identifier <span class="op">$</span> symbol <span class="st">","</span>))</span>
<span id="cb26-18"><a href="#cb26-18" aria-hidden="true" tabindex="-1"></a> <span class="op"><*></span> braces (many stmt)</span>
<span id="cb26-19"><a href="#cb26-19" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">Variable</span> <span class="op"><$></span> identifier</span>
<span id="cb26-20"><a href="#cb26-20" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> parens expr</span></code></pre></div>
<p>The <code>term</code> parser uses the <code>primary</code> parser to parse the primary terms, and the <code>call</code> parser to parse the function calls. The <code>call</code> parser is recursive because the callee itself can be a chain of more functions calls, for example <code class="sourceCode javascript"><span class="fu">x</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">2</span>)(y)(<span class="st">"blah"</span>)()</code>. It starts with the output of the <code>primary</code> parser, and then tries to parse a function call. If it succeeds, it calls itself again to parse the next function call in the chain. If it fails, it returns the output of the previous round<a href="#fn9" class="footnote-ref" id="fnref9" role="doc-noteref"><sup>9</sup></a>.</p>
<p>The <code>primary</code> parser is a combination of smaller parsers—one for each type of primary terms in <span class="fancy">Co</span>—combined together using the <a href="https://hackage.haskell.org/package/base-4.12.0.0/docs/Control-Applicative.html#t:Alternative" target="_blank" rel="noopener"><code class="sourceCode haskell"><span class="dt">Alternative</span></code></a> instance of the parsers. It matches each parser one by one from the top until the match succeeds<a href="#fn10" class="footnote-ref" id="fnref10" role="doc-noteref"><sup>10</sup></a>. First, it tries to match for literals <code class="sourceCode javascript"><span class="kw">null</span></code>, <code class="sourceCode javascript"><span class="kw">true</span></code>, and <code class="sourceCode javascript"><span class="kw">false</span></code>, failing which it matches for string and integer literals. Then it matches for anonymous functions, variables, and expressions grouped in parentheses—in that order<a href="#fn11" class="footnote-ref" id="fnref11" role="doc-noteref"><sup>11</sup></a>.</p>
<p>That’s it! We finally write the expression parser:</p>
<div class="sourceCode" id="cb27" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb27-1"><a href="#cb27-1" aria-hidden="true" tabindex="-1"></a><span class="ot">expr ::</span> <span class="dt">Parser</span> <span class="dt">Expr</span></span>
<span id="cb27-2"><a href="#cb27-2" aria-hidden="true" tabindex="-1"></a>expr <span class="ot">=</span> makeExprParser term operators</span></code></pre></div>
<p>Let’s play with it in GHCi:</p>
<div class="sourceCode" id="cb28" data-lang="ghci"><pre class="sourceCode lhs numberSource"><code class="sourceCode literatehaskell"><span id="cb28-1"><a href="#cb28-1" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> pPrint <span class="op">$</span> runParser expr <span class="st">"1 + a < 9 - <- chan"</span></span>
<span id="cb28-2"><a href="#cb28-2" aria-hidden="true" tabindex="-1"></a>Right</span>
<span id="cb28-3"><a href="#cb28-3" aria-hidden="true" tabindex="-1"></a> ( Binary LessThan</span>
<span id="cb28-4"><a href="#cb28-4" aria-hidden="true" tabindex="-1"></a> ( Binary Plus ( LNum 1 ) ( Variable "a" ) )</span>
<span id="cb28-5"><a href="#cb28-5" aria-hidden="true" tabindex="-1"></a> ( Binary Minus ( LNum 9 ) ( Receive ( Variable "chan" ) ) ) )</span>
<span id="cb28-6"><a href="#cb28-6" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> pPrint <span class="op">$</span> runParser expr <span class="st">"funFun(null == \"ss\" + 12, true)"</span></span>
<span id="cb28-7"><a href="#cb28-7" aria-hidden="true" tabindex="-1"></a>Right</span>
<span id="cb28-8"><a href="#cb28-8" aria-hidden="true" tabindex="-1"></a> ( Call "funFun"</span>
<span id="cb28-9"><a href="#cb28-9" aria-hidden="true" tabindex="-1"></a> [ Binary Equals LNull</span>
<span id="cb28-10"><a href="#cb28-10" aria-hidden="true" tabindex="-1"></a> ( Binary Plus ( LStr "ss" ) ( LNum 12 ) )</span>
<span id="cb28-11"><a href="#cb28-11" aria-hidden="true" tabindex="-1"></a> , LBool True ] )</span>
<span id="cb28-12"><a href="#cb28-12" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> pPrint <span class="op">$</span> runParser expr <span class="st">"-99 - <- chan + funkyFun(a, false, \"hey\")"</span></span>
<span id="cb28-13"><a href="#cb28-13" aria-hidden="true" tabindex="-1"></a>Right</span>
<span id="cb28-14"><a href="#cb28-14" aria-hidden="true" tabindex="-1"></a> ( Binary Plus</span>
<span id="cb28-15"><a href="#cb28-15" aria-hidden="true" tabindex="-1"></a> ( Binary Minus ( LNum ( - 99 ) ) ( Receive ( Variable "chan" ) ) )</span>
<span id="cb28-16"><a href="#cb28-16" aria-hidden="true" tabindex="-1"></a> ( Call "funkyFun" [ Variable "a", LBool False, LStr "hey" ] ) )</span>
<span id="cb28-17"><a href="#cb28-17" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> pPrint <span class="op">$</span> runParser expr <span class="st">"(function (x) { print(x + 1); })(99)"</span></span>
<span id="cb28-18"><a href="#cb28-18" aria-hidden="true" tabindex="-1"></a>Right</span>
<span id="cb28-19"><a href="#cb28-19" aria-hidden="true" tabindex="-1"></a> ( Call</span>
<span id="cb28-20"><a href="#cb28-20" aria-hidden="true" tabindex="-1"></a> ( Lambda [ "x" ]</span>
<span id="cb28-21"><a href="#cb28-21" aria-hidden="true" tabindex="-1"></a> [ ExprStmt</span>
<span id="cb28-22"><a href="#cb28-22" aria-hidden="true" tabindex="-1"></a> ( Call</span>
<span id="cb28-23"><a href="#cb28-23" aria-hidden="true" tabindex="-1"></a> ( Variable "print" )</span>
<span id="cb28-24"><a href="#cb28-24" aria-hidden="true" tabindex="-1"></a> [ Binary Plus ( Variable "x" ) ( LNum 1 ) ] ) ] )</span>
<span id="cb28-25"><a href="#cb28-25" aria-hidden="true" tabindex="-1"></a> [ LNum 99 ] )</span></code></pre></div>
<p>Done! Onward, to parsing <span class="fancy">Co</span> statements.</p>
<h3 id="parsing-statements">Parsing Statements</h3>
<p>For parsing statements, we reuse the same trick we used for parsing expressions: combine smaller parsers for each statement type using <code class="sourceCode haskell"><span class="dt">Alternative</span></code>.</p>
<div class="sourceCode" id="cb29" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb29-1"><a href="#cb29-1" aria-hidden="true" tabindex="-1"></a><span class="ot">stmt ::</span> <span class="dt">Parser</span> <span class="dt">Stmt</span></span>
<span id="cb29-2"><a href="#cb29-2" aria-hidden="true" tabindex="-1"></a>stmt <span class="ot">=</span></span>
<span id="cb29-3"><a href="#cb29-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">IfStmt</span> <span class="op"><$></span> (reserved <span class="st">"if"</span> <span class="op">*></span> parens expr) <span class="op"><*></span> braces (many stmt)</span>
<span id="cb29-4"><a href="#cb29-4" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">WhileStmt</span> <span class="op"><$></span> (reserved <span class="st">"while"</span> <span class="op">*></span> parens expr) <span class="op"><*></span> braces (many stmt)</span>
<span id="cb29-5"><a href="#cb29-5" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">VarStmt</span> <span class="op"><$></span> (reserved <span class="st">"var"</span> <span class="op">*></span> identifier) <span class="op"><*></span> (symbol <span class="st">"="</span> <span class="op">*></span> expr <span class="op"><*</span> semi)</span>
<span id="cb29-6"><a href="#cb29-6" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">YieldStmt</span> <span class="op"><$</span> (reserved <span class="st">"yield"</span> <span class="op"><*</span> semi)</span>
<span id="cb29-7"><a href="#cb29-7" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">SpawnStmt</span> <span class="op"><$></span> (reserved <span class="st">"spawn"</span> <span class="op">*></span> expr <span class="op"><*</span> semi)</span>
<span id="cb29-8"><a href="#cb29-8" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">ReturnStmt</span> <span class="op"><$></span> (reserved <span class="st">"return"</span> <span class="op">*></span> optional expr <span class="op"><*</span> semi)</span>
<span id="cb29-9"><a href="#cb29-9" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">FunctionStmt</span></span>
<span id="cb29-10"><a href="#cb29-10" aria-hidden="true" tabindex="-1"></a> <span class="op"><$></span> try (reserved <span class="st">"function"</span> <span class="op">*></span> identifier)</span>
<span id="cb29-11"><a href="#cb29-11" aria-hidden="true" tabindex="-1"></a> <span class="op"><*></span> parens (sepBy identifier <span class="op">$</span> symbol <span class="st">","</span>)</span>
<span id="cb29-12"><a href="#cb29-12" aria-hidden="true" tabindex="-1"></a> <span class="op"><*></span> braces (many stmt)</span>
<span id="cb29-13"><a href="#cb29-13" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> try (<span class="dt">AssignStmt</span> <span class="op"><$></span> identifier <span class="op"><*></span> (symbol <span class="st">"="</span> <span class="op">*></span> expr <span class="op"><*</span> semi))</span>
<span id="cb29-14"><a href="#cb29-14" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> try (<span class="dt">SendStmt</span> <span class="op"><$></span> expr <span class="op"><*></span> (symbol <span class="st">"->"</span> <span class="op">*></span> expr <span class="op"><*</span> semi))</span>
<span id="cb29-15"><a href="#cb29-15" aria-hidden="true" tabindex="-1"></a> <span class="op"><|></span> <span class="dt">ExprStmt</span> <span class="op"><$></span> expr <span class="op"><*</span> semi</span></code></pre></div>
<p>Most of the statements start with keywords like <code class="sourceCode javascript"><span class="cf">if</span></code>, <code class="sourceCode javascript"><span class="kw">var</span></code>, <code class="sourceCode javascript"><span class="kw">function</span></code>, etc, so our parsers look for the keywords first using the <code>reserved</code> parser. If none of the keyword-starting statements match then we try matching for assignments, channel sends, and expression statements, in that order. The <code>stmt</code> parser uses the <code>expr</code> parser to parse expressions within statements. It also uses the combinators <a href="https://hackage.haskell.org/package/parser-combinators-1.3.0/docs/Control-Applicative-Combinators.html#v:many" target="_blank" rel="noopener"><code>many</code></a>, <a href="https://hackage.haskell.org/package/parser-combinators-1.3.0/docs/Control-Applicative-Combinators.html#v:optional" target="_blank" rel="noopener"><code>optional</code></a>, <a href="https://hackage.haskell.org/package/megaparsec-9.0.1/docs/Text-Megaparsec.html#v:try" target="_blank" rel="noopener"><code>try</code></a> and <a href="https://hackage.haskell.org/package/parser-combinators-1.3.0/docs/Control-Applicative-Combinators.html#v:sepBy" target="_blank" rel="noopener"><code>sepBy</code></a> from Megaparsec.</p>
<p>Finally, we put everything together to create the parser for <span class="fancy">Co</span>:</p>
<div class="sourceCode" id="cb30" data-lang="haskell"><pre class="sourceCode numberSource haskell"><code class="sourceCode haskell"><span id="cb30-1"><a href="#cb30-1" aria-hidden="true" tabindex="-1"></a><span class="ot">program ::</span> <span class="dt">Parser</span> <span class="dt">Program</span></span>
<span id="cb30-2"><a href="#cb30-2" aria-hidden="true" tabindex="-1"></a>program <span class="ot">=</span> sc <span class="op">*></span> many stmt <span class="op"><*</span> eof</span></code></pre></div>
<p>The <span class="fancy">Co</span> parser matches multiple top-level statements, starting with optional whitespace and ending with end-of-file.</p>
<p>It’s demo time. In GHCi, we parse a file and pretty-print the AST:</p>
<div class="sourceCode" id="cb31" data-lang="ghci"><pre class="sourceCode lhs numberSource"><code class="sourceCode literatehaskell"><span id="cb31-1"><a href="#cb31-1" aria-hidden="true" tabindex="-1"></a><span class="ot">></span> <span class="fu">readFile</span> <span class="st">"coroutines.co"</span> <span class="op">>>=</span> pPrint <span class="op">.</span> runParser program</span>
<span id="cb31-2"><a href="#cb31-2" aria-hidden="true" tabindex="-1"></a>Right</span>
<span id="cb31-3"><a href="#cb31-3" aria-hidden="true" tabindex="-1"></a> [ FunctionStmt "printNums"</span>
<span id="cb31-4"><a href="#cb31-4" aria-hidden="true" tabindex="-1"></a> [ "start", "end" ]</span>
<span id="cb31-5"><a href="#cb31-5" aria-hidden="true" tabindex="-1"></a> [ VarStmt "i"</span>
<span id="cb31-6"><a href="#cb31-6" aria-hidden="true" tabindex="-1"></a> ( Variable "start" )</span>
<span id="cb31-7"><a href="#cb31-7" aria-hidden="true" tabindex="-1"></a> , WhileStmt</span>
<span id="cb31-8"><a href="#cb31-8" aria-hidden="true" tabindex="-1"></a> ( Binary LessThan</span>
<span id="cb31-9"><a href="#cb31-9" aria-hidden="true" tabindex="-1"></a> ( Variable "i" )</span>
<span id="cb31-10"><a href="#cb31-10" aria-hidden="true" tabindex="-1"></a> ( Binary Plus ( Variable "end" ) ( LNum 1 ) ) )</span>
<span id="cb31-11"><a href="#cb31-11" aria-hidden="true" tabindex="-1"></a> [ ExprStmt</span>
<span id="cb31-12"><a href="#cb31-12" aria-hidden="true" tabindex="-1"></a> ( Call "print" [ Variable "i" ] )</span>
<span id="cb31-13"><a href="#cb31-13" aria-hidden="true" tabindex="-1"></a> , YieldStmt</span>
<span id="cb31-14"><a href="#cb31-14" aria-hidden="true" tabindex="-1"></a> , AssignStmt "i"</span>
<span id="cb31-15"><a href="#cb31-15" aria-hidden="true" tabindex="-1"></a> ( Binary Plus ( Variable "i" ) ( LNum 1 ) ) ] ]</span>
<span id="cb31-16"><a href="#cb31-16" aria-hidden="true" tabindex="-1"></a> , SpawnStmt</span>
<span id="cb31-17"><a href="#cb31-17" aria-hidden="true" tabindex="-1"></a> ( Call "printNums" [ LNum 1, LNum 4 ] )</span>
<span id="cb31-18"><a href="#cb31-18" aria-hidden="true" tabindex="-1"></a> , ExprStmt</span>
<span id="cb31-19"><a href="#cb31-19" aria-hidden="true" tabindex="-1"></a> ( Call "printNums" [ LNum 11, LNum 16 ] ) ]</span></code></pre></div>
<p>I’ve gone ahead and created a side-by-side view of the source code and corresponding AST for all three input files, aligned neatly for your pleasure of comparison. If you so wish, feast your eyes on them, and check for yourself that everything works correctly.</p>
<details>
<summary>
Source code and AST for <code>fib.co</code>
</summary>
<div class="sbs-code">
<div class="scrollable-table">
<table>
<colgroup>
<col style="width: 18%"></col>
<col style="width: 81%"></col>
</colgroup>
<tbody>
<tr class="odd">
<td><div class="sourceCode" id="cb32" data-lang="co"><pre class="sourceCode javascript small noNumberSource"><code class="sourceCode javascript"><span id="cb32-1"><a href="#cb32-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Fibonacci numbers</span></span>
<span id="cb32-2"><a href="#cb32-2" aria-hidden="true" tabindex="-1"></a><span class="co">// using a while loop</span></span>
<span id="cb32-3"><a href="#cb32-3" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> a <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb32-4"><a href="#cb32-4" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> b <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb32-5"><a href="#cb32-5" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> j <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb32-6"><a href="#cb32-6" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> temp <span class="op">=</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb32-7"><a href="#cb32-7" aria-hidden="true" tabindex="-1"></a><span class="cf">while</span> (j <span class="op"><</span> <span class="dv">6</span>) {</span>
<span id="cb32-8"><a href="#cb32-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(a)<span class="op">;</span></span>
<span id="cb32-9"><a href="#cb32-9" aria-hidden="true" tabindex="-1"></a> temp <span class="op">=</span> a<span class="op">;</span></span>
<span id="cb32-10"><a href="#cb32-10" aria-hidden="true" tabindex="-1"></a> a <span class="op">=</span> b<span class="op">;</span></span>
<span id="cb32-11"><a href="#cb32-11" aria-hidden="true" tabindex="-1"></a> b <span class="op">=</span> temp <span class="op">+</span> b<span class="op">;</span></span>
<span id="cb32-12"><a href="#cb32-12" aria-hidden="true" tabindex="-1"></a> j <span class="op">=</span> j <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb32-13"><a href="#cb32-13" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb32-14"><a href="#cb32-14" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb32-15"><a href="#cb32-15" aria-hidden="true" tabindex="-1"></a><span class="co">// Fibonacci numbers</span></span>
<span id="cb32-16"><a href="#cb32-16" aria-hidden="true" tabindex="-1"></a><span class="co">// using recursive function call</span></span>
<span id="cb32-17"><a href="#cb32-17" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">fib</span>(n) {</span>
<span id="cb32-18"><a href="#cb32-18" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (n <span class="op"><</span> <span class="dv">2</span>) {</span>
<span id="cb32-19"><a href="#cb32-19" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> n<span class="op">;</span></span>
<span id="cb32-20"><a href="#cb32-20" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb32-21"><a href="#cb32-21" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="fu">fib</span>(n <span class="op">-</span> <span class="dv">2</span>)</span>
<span id="cb32-22"><a href="#cb32-22" aria-hidden="true" tabindex="-1"></a> <span class="op">+</span> <span class="fu">fib</span>(n <span class="op">-</span> <span class="dv">1</span>)<span class="op">;</span></span>
<span id="cb32-23"><a href="#cb32-23" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb32-24"><a href="#cb32-24" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb32-25"><a href="#cb32-25" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> i <span class="op">=</span> <span class="dv">0</span><span class="op">;</span></span>
<span id="cb32-26"><a href="#cb32-26" aria-hidden="true" tabindex="-1"></a><span class="cf">while</span> (i <span class="op"><</span> <span class="dv">6</span>) {</span>
<span id="cb32-27"><a href="#cb32-27" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="fu">fib</span>(i))<span class="op">;</span></span>
<span id="cb32-28"><a href="#cb32-28" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb32-29"><a href="#cb32-29" aria-hidden="true" tabindex="-1"></a>}</span></code></pre></div></td>
<td><div class="sourceCode" id="cb33" data-lang="haskell"><pre class="sourceCode haskell small noNumberSource"><code class="sourceCode haskell"><span id="cb33-1"><a href="#cb33-1" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb33-2"><a href="#cb33-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb33-3"><a href="#cb33-3" aria-hidden="true" tabindex="-1"></a>[ <span class="dt">VarStmt</span> <span class="st">"a"</span> ( <span class="dt">LNum</span> <span class="dv">0</span> )</span>
<span id="cb33-4"><a href="#cb33-4" aria-hidden="true" tabindex="-1"></a>, <span class="dt">VarStmt</span> <span class="st">"b"</span> ( <span class="dt">LNum</span> <span class="dv">1</span> )</span>
<span id="cb33-5"><a href="#cb33-5" aria-hidden="true" tabindex="-1"></a>, <span class="dt">VarStmt</span> <span class="st">"j"</span> ( <span class="dt">LNum</span> <span class="dv">0</span> )</span>
<span id="cb33-6"><a href="#cb33-6" aria-hidden="true" tabindex="-1"></a>, <span class="dt">VarStmt</span> <span class="st">"temp"</span> <span class="dt">LNull</span></span>
<span id="cb33-7"><a href="#cb33-7" aria-hidden="true" tabindex="-1"></a>, <span class="dt">WhileStmt</span> ( <span class="dt">Binary</span> <span class="dt">LessThan</span> ( <span class="dt">Variable</span> <span class="st">"j"</span> ) ( <span class="dt">LNum</span> <span class="dv">6</span> ) )</span>
<span id="cb33-8"><a href="#cb33-8" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">ExprStmt</span> ( <span class="dt">Call</span> <span class="st">"print"</span> [ <span class="dt">Variable</span> <span class="st">"a"</span> ] )</span>
<span id="cb33-9"><a href="#cb33-9" aria-hidden="true" tabindex="-1"></a> , <span class="dt">AssignStmt</span> <span class="st">"temp"</span> ( <span class="dt">Variable</span> <span class="st">"a"</span> )</span>
<span id="cb33-10"><a href="#cb33-10" aria-hidden="true" tabindex="-1"></a> , <span class="dt">AssignStmt</span> <span class="st">"a"</span> ( <span class="dt">Variable</span> <span class="st">"b"</span> )</span>
<span id="cb33-11"><a href="#cb33-11" aria-hidden="true" tabindex="-1"></a> , <span class="dt">AssignStmt</span> <span class="st">"b"</span> ( <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Variable</span> <span class="st">"temp"</span> ) ( <span class="dt">Variable</span> <span class="st">"b"</span> ) )</span>
<span id="cb33-12"><a href="#cb33-12" aria-hidden="true" tabindex="-1"></a> , <span class="dt">AssignStmt</span> <span class="st">"j"</span> ( <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Variable</span> <span class="st">"j"</span> ) ( <span class="dt">LNum</span> <span class="dv">1</span> ) )</span>
<span id="cb33-13"><a href="#cb33-13" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb33-14"><a href="#cb33-14" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb33-15"><a href="#cb33-15" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb33-16"><a href="#cb33-16" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb33-17"><a href="#cb33-17" aria-hidden="true" tabindex="-1"></a>, <span class="dt">FunctionStmt</span> <span class="st">"fib"</span> [ <span class="st">"n"</span> ]</span>
<span id="cb33-18"><a href="#cb33-18" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">IfStmt</span> ( <span class="dt">Binary</span> <span class="dt">LessThan</span> ( <span class="dt">Variable</span> <span class="st">"n"</span> ) ( <span class="dt">LNum</span> <span class="dv">2</span> ) )</span>
<span id="cb33-19"><a href="#cb33-19" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">ReturnStmt</span> ( <span class="dt">Just</span> ( <span class="dt">Variable</span> <span class="st">"n"</span> ) )</span>
<span id="cb33-20"><a href="#cb33-20" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb33-21"><a href="#cb33-21" aria-hidden="true" tabindex="-1"></a> , <span class="dt">ReturnStmt</span> ( <span class="dt">Just</span> ( <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Call</span> <span class="st">"fib"</span> [ <span class="dt">Binary</span> <span class="dt">Minus</span> ( <span class="dt">Variable</span> <span class="st">"n"</span> ) ( <span class="dt">LNum</span> <span class="dv">2</span> ) ] )</span>
<span id="cb33-22"><a href="#cb33-22" aria-hidden="true" tabindex="-1"></a> ( <span class="dt">Call</span> <span class="st">"fib"</span> [ <span class="dt">Binary</span> <span class="dt">Minus</span> ( <span class="dt">Variable</span> <span class="st">"n"</span> ) ( <span class="dt">LNum</span> <span class="dv">1</span> ) ] ) ) )</span>
<span id="cb33-23"><a href="#cb33-23" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb33-24"><a href="#cb33-24" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb33-25"><a href="#cb33-25" aria-hidden="true" tabindex="-1"></a>, <span class="dt">VarStmt</span> <span class="st">"i"</span> ( <span class="dt">LNum</span> <span class="dv">0</span> )</span>
<span id="cb33-26"><a href="#cb33-26" aria-hidden="true" tabindex="-1"></a>, <span class="dt">WhileStmt</span> ( <span class="dt">Binary</span> <span class="dt">LessThan</span> ( <span class="dt">Variable</span> <span class="st">"i"</span> ) ( <span class="dt">LNum</span> <span class="dv">6</span> ) )</span>
<span id="cb33-27"><a href="#cb33-27" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">ExprStmt</span> ( <span class="dt">Call</span> <span class="st">"print"</span> [ <span class="dt">Call</span> <span class="st">"fib"</span> [ <span class="dt">Variable</span> <span class="st">"i"</span> ] ] )</span>
<span id="cb33-28"><a href="#cb33-28" aria-hidden="true" tabindex="-1"></a> , <span class="dt">AssignStmt</span> <span class="st">"i"</span> ( <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Variable</span> <span class="st">"i"</span> ) ( <span class="dt">LNum</span> <span class="dv">1</span> ) )</span>
<span id="cb33-29"><a href="#cb33-29" aria-hidden="true" tabindex="-1"></a> ] ]</span></code></pre></div></td>
</tr>
</tbody>
</table>
</div>
</div>
</details>
<details>
<summary>
Source code and AST for <code>coroutines.co</code>
</summary>
<div class="sbs-code">
<div class="scrollable-table">
<table>
<colgroup>
<col style="width: 25%"></col>
<col style="width: 75%"></col>
</colgroup>
<tbody>
<tr class="odd">
<td><div class="sourceCode" id="cb34" data-lang="co"><pre class="sourceCode javascript small noNumberSource"><code class="sourceCode javascript"><span id="cb34-1"><a href="#cb34-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">printNums</span>(start<span class="op">,</span> end) {</span>
<span id="cb34-2"><a href="#cb34-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> i <span class="op">=</span> start<span class="op">;</span></span>
<span id="cb34-3"><a href="#cb34-3" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (i <span class="op"><</span> end <span class="op">+</span> <span class="dv">1</span>) {</span>
<span id="cb34-4"><a href="#cb34-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(i)<span class="op">;</span></span>
<span id="cb34-5"><a href="#cb34-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">yield</span><span class="op">;</span></span>
<span id="cb34-6"><a href="#cb34-6" aria-hidden="true" tabindex="-1"></a> i <span class="op">=</span> i <span class="op">+</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb34-7"><a href="#cb34-7" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb34-8"><a href="#cb34-8" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb34-9"><a href="#cb34-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb34-10"><a href="#cb34-10" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">printNums</span>(<span class="dv">1</span><span class="op">,</span> <span class="dv">4</span>)<span class="op">;</span></span>
<span id="cb34-11"><a href="#cb34-11" aria-hidden="true" tabindex="-1"></a><span class="fu">printNums</span>(<span class="dv">11</span><span class="op">,</span> <span class="dv">16</span>)<span class="op">;</span></span></code></pre></div></td>
<td><div class="sourceCode" id="cb35" data-lang="haskell"><pre class="sourceCode haskell small noNumberSource"><code class="sourceCode haskell"><span id="cb35-1"><a href="#cb35-1" aria-hidden="true" tabindex="-1"></a>[ <span class="dt">FunctionStmt</span> <span class="st">"printNums"</span> [ <span class="st">"start"</span>, <span class="st">"end"</span> ]</span>
<span id="cb35-2"><a href="#cb35-2" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">VarStmt</span> <span class="st">"i"</span> ( <span class="dt">Variable</span> <span class="st">"start"</span> )</span>
<span id="cb35-3"><a href="#cb35-3" aria-hidden="true" tabindex="-1"></a> , <span class="dt">WhileStmt</span> ( <span class="dt">Binary</span> <span class="dt">LessThan</span> ( <span class="dt">Variable</span> <span class="st">"i"</span> ) ( <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Variable</span> <span class="st">"end"</span> ) ( <span class="dt">LNum</span> <span class="dv">1</span> ) )</span>
<span id="cb35-4"><a href="#cb35-4" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">ExprStmt</span> ( <span class="dt">Call</span> <span class="st">"print"</span> [ <span class="dt">Variable</span> <span class="st">"i"</span> ] )</span>
<span id="cb35-5"><a href="#cb35-5" aria-hidden="true" tabindex="-1"></a> , <span class="dt">YieldStmt</span></span>
<span id="cb35-6"><a href="#cb35-6" aria-hidden="true" tabindex="-1"></a> , <span class="dt">AssignStmt</span> <span class="st">"i"</span> ( <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Variable</span> <span class="st">"i"</span> ) ( <span class="dt">LNum</span> <span class="dv">1</span> ) )</span>
<span id="cb35-7"><a href="#cb35-7" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb35-8"><a href="#cb35-8" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb35-9"><a href="#cb35-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb35-10"><a href="#cb35-10" aria-hidden="true" tabindex="-1"></a>, <span class="dt">SpawnStmt</span> ( <span class="dt">Call</span> <span class="st">"printNums"</span> [ <span class="dt">LNum</span> <span class="dv">1</span>, <span class="dt">LNum</span> <span class="dv">4</span> ] )</span>
<span id="cb35-11"><a href="#cb35-11" aria-hidden="true" tabindex="-1"></a>, <span class="dt">ExprStmt</span> ( <span class="dt">Call</span> <span class="st">"printNums"</span> [ <span class="dt">LNum</span> <span class="dv">11</span>, <span class="dt">LNum</span> <span class="dv">16</span> ] ) ]</span></code></pre></div></td>
</tr>
</tbody>
</table>
</div>
</div>
</details>
<details>
<summary>
Source code and AST for <code>pingpong.co</code>
</summary>
<div class="sbs-code">
<div class="scrollable-table">
<table>
<colgroup>
<col style="width: 20%"></col>
<col style="width: 79%"></col>
</colgroup>
<tbody>
<tr class="odd">
<td><div class="sourceCode" id="cb36" data-lang="co"><pre class="sourceCode javascript small noNumberSource"><code class="sourceCode javascript"><span id="cb36-1"><a href="#cb36-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> chan <span class="op">=</span> <span class="fu">newChannel</span>()<span class="op">;</span></span>
<span id="cb36-2"><a href="#cb36-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb36-3"><a href="#cb36-3" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">player</span>(name) {</span>
<span id="cb36-4"><a href="#cb36-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> n <span class="op">=</span> <span class="kw">null</span><span class="op">;</span></span>
<span id="cb36-5"><a href="#cb36-5" aria-hidden="true" tabindex="-1"></a> <span class="cf">while</span> (<span class="kw">true</span>) {</span>
<span id="cb36-6"><a href="#cb36-6" aria-hidden="true" tabindex="-1"></a> n <span class="op">=</span> <span class="op"><-</span> chan<span class="op">;</span></span>
<span id="cb36-7"><a href="#cb36-7" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (n <span class="op">==</span> <span class="st">"done"</span>) {</span>
<span id="cb36-8"><a href="#cb36-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" done"</span>)<span class="op">;</span></span>
<span id="cb36-9"><a href="#cb36-9" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span><span class="op">;</span></span>
<span id="cb36-10"><a href="#cb36-10" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb36-11"><a href="#cb36-11" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb36-12"><a href="#cb36-12" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" "</span> <span class="op">+</span> n)<span class="op">;</span></span>
<span id="cb36-13"><a href="#cb36-13" aria-hidden="true" tabindex="-1"></a> <span class="cf">if</span> (n <span class="op">==</span> <span class="dv">0</span>) {</span>
<span id="cb36-14"><a href="#cb36-14" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(name <span class="op">+</span> <span class="st">" done"</span>)<span class="op">;</span></span>
<span id="cb36-15"><a href="#cb36-15" aria-hidden="true" tabindex="-1"></a> <span class="st">"done"</span> <span class="op">-></span> chan<span class="op">;</span></span>
<span id="cb36-16"><a href="#cb36-16" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span><span class="op">;</span></span>
<span id="cb36-17"><a href="#cb36-17" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb36-18"><a href="#cb36-18" aria-hidden="true" tabindex="-1"></a> n <span class="op">-</span> <span class="dv">1</span> <span class="op">-></span> chan<span class="op">;</span></span>
<span id="cb36-19"><a href="#cb36-19" aria-hidden="true" tabindex="-1"></a> }</span>
<span id="cb36-20"><a href="#cb36-20" aria-hidden="true" tabindex="-1"></a>}</span>
<span id="cb36-21"><a href="#cb36-21" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb36-22"><a href="#cb36-22" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">player</span>(<span class="st">"ping"</span>)<span class="op">;</span></span>
<span id="cb36-23"><a href="#cb36-23" aria-hidden="true" tabindex="-1"></a>spawn <span class="fu">player</span>(<span class="st">"pong"</span>)<span class="op">;</span></span>
<span id="cb36-24"><a href="#cb36-24" aria-hidden="true" tabindex="-1"></a><span class="dv">10</span> <span class="op">-></span> chan<span class="op">;</span></span></code></pre></div></td>
<td><div class="sourceCode" id="cb37" data-lang="haskell"><pre class="sourceCode haskell small noNumberSource"><code class="sourceCode haskell"><span id="cb37-1"><a href="#cb37-1" aria-hidden="true" tabindex="-1"></a>[ <span class="dt">VarStmt</span> <span class="st">"chan"</span> ( <span class="dt">Call</span> <span class="st">"newChannel"</span> [] )</span>
<span id="cb37-2"><a href="#cb37-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb37-3"><a href="#cb37-3" aria-hidden="true" tabindex="-1"></a>, <span class="dt">FunctionStmt</span> <span class="st">"player"</span> [ <span class="st">"name"</span> ]</span>
<span id="cb37-4"><a href="#cb37-4" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">VarStmt</span> <span class="st">"n"</span> <span class="dt">LNull</span></span>
<span id="cb37-5"><a href="#cb37-5" aria-hidden="true" tabindex="-1"></a> , <span class="dt">WhileStmt</span> ( <span class="dt">LBool</span> <span class="dt">True</span> )</span>
<span id="cb37-6"><a href="#cb37-6" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">AssignStmt</span> <span class="st">"n"</span> ( <span class="dt">Receive</span> ( <span class="dt">Variable</span> <span class="st">"chan"</span> ) )</span>
<span id="cb37-7"><a href="#cb37-7" aria-hidden="true" tabindex="-1"></a> , <span class="dt">IfStmt</span> ( <span class="dt">Binary</span> <span class="dt">Equals</span> ( <span class="dt">Variable</span> <span class="st">"n"</span> ) ( <span class="dt">LStr</span> <span class="st">"done"</span> ) )</span>
<span id="cb37-8"><a href="#cb37-8" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">ExprStmt</span> ( <span class="dt">Call</span> <span class="st">"print"</span> [ <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Variable</span> <span class="st">"name"</span> ) ( <span class="dt">LStr</span> <span class="st">" done"</span> ) ] )</span>
<span id="cb37-9"><a href="#cb37-9" aria-hidden="true" tabindex="-1"></a> , <span class="dt">ReturnStmt</span> <span class="dt">Nothing</span></span>
<span id="cb37-10"><a href="#cb37-10" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb37-11"><a href="#cb37-11" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb37-12"><a href="#cb37-12" aria-hidden="true" tabindex="-1"></a> , <span class="dt">ExprStmt</span> ( <span class="dt">Call</span> <span class="st">"print"</span> [ <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Variable</span> <span class="st">"name"</span> ) ( <span class="dt">LStr</span> <span class="st">" "</span> ) ) ( <span class="dt">Variable</span> <span class="st">"n"</span> ) ] )</span>
<span id="cb37-13"><a href="#cb37-13" aria-hidden="true" tabindex="-1"></a> , <span class="dt">IfStmt</span> ( <span class="dt">Binary</span> <span class="dt">Equals</span> ( <span class="dt">Variable</span> <span class="st">"n"</span> ) ( <span class="dt">LNum</span> <span class="dv">0</span> ) )</span>
<span id="cb37-14"><a href="#cb37-14" aria-hidden="true" tabindex="-1"></a> [ <span class="dt">ExprStmt</span> ( <span class="dt">Call</span> <span class="st">"print"</span> [ <span class="dt">Binary</span> <span class="dt">Plus</span> ( <span class="dt">Variable</span> <span class="st">"name"</span> ) ( <span class="dt">LStr</span> <span class="st">" done"</span> ) ] )</span>
<span id="cb37-15"><a href="#cb37-15" aria-hidden="true" tabindex="-1"></a> , <span class="dt">SendStmt</span> ( <span class="dt">LStr</span> <span class="st">"done"</span> ) <span class="st">"chan"</span></span>
<span id="cb37-16"><a href="#cb37-16" aria-hidden="true" tabindex="-1"></a> , <span class="dt">ReturnStmt</span> <span class="dt">Nothing</span></span>
<span id="cb37-17"><a href="#cb37-17" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb37-18"><a href="#cb37-18" aria-hidden="true" tabindex="-1"></a> , <span class="dt">SendStmt</span> ( <span class="dt">Binary</span> <span class="dt">Minus</span> ( <span class="dt">Variable</span> <span class="st">"n"</span> ) ( <span class="dt">LNum</span> <span class="dv">1</span> ) ) <span class="st">"chan"</span></span>
<span id="cb37-19"><a href="#cb37-19" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb37-20"><a href="#cb37-20" aria-hidden="true" tabindex="-1"></a> ]</span>
<span id="cb37-21"><a href="#cb37-21" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb37-22"><a href="#cb37-22" aria-hidden="true" tabindex="-1"></a>, <span class="dt">SpawnStmt</span> ( <span class="dt">Call</span> <span class="st">"player"</span> [ <span class="dt">LStr</span> <span class="st">"ping"</span> ] )</span>
<span id="cb37-23"><a href="#cb37-23" aria-hidden="true" tabindex="-1"></a>, <span class="dt">SpawnStmt</span> ( <span class="dt">Call</span> <span class="st">"player"</span> [ <span class="dt">LStr</span> <span class="st">"pong"</span> ] )</span>
<span id="cb37-24"><a href="#cb37-24" aria-hidden="true" tabindex="-1"></a>, <span class="dt">SendStmt</span> ( <span class="dt">LNum</span> <span class="dv">10</span> ) <span class="st">"chan"</span> ]</span></code></pre></div></td>
</tr>
</tbody>
</table>
</div>
</div>
</details>
<hr></hr>
<p>That’s all for now. In about 200 lines of code, we have implemented the complete parser for <span class="fancy">Co</span>. In the <a href="https://abhinavsarkar.net/posts/implementing-co-2/?mtm_campaign=feed">next part</a>, we’ll implement a tree-walking interpreter for the <span class="fancy">Co</span> AST that supports the <a href="#basic-features">basic features</a>.</p>
<p>The full code for the parser can be seen <a href="https://abhinavsarkar.net/code/co-parser.html?mtm_campaign=feed">here</a>.</p>
<h2 class="notoc" data-track-content data-content-name="acknowledgements" data-content-piece="implementing-co-1" id="acknowledgements">Acknowledgements</h2>
<p>Many thanks to <a href="https://www.deobald.ca/" target="_blank" rel="noopener">Steven Deobald</a> for reviewing a draft of this article.</p>
<div id="refs" class="references csl-bib-body hanging-indent" role="list">
<div id="ref-Bartel2011-ap" class="csl-entry" role="listitem">
Bartel, Joe. <span>“<span>Non-Preemptive</span> Multitasking.”</span> <em>The Computer Journal</em>, no. 30 (May 2011): 37–38, 28. <a href="http://cini.classiccmp.org/pdf/HT68K/HT68K%20TCJ30p37.pdf" target="_blank" rel="noopener">http://cini.classiccmp.org/pdf/HT68K/HT68K%20TCJ30p37.pdf</a>.
</div>
<div id="ref-Hoare1986-ih" class="csl-entry" role="listitem">
Hoare, C A R. <em>Communicating Sequential Processes</em>. Prentice Hall, 1986. <a href="https://doi.org/10.1145/359576.359585" target="_blank" rel="noopener">https://doi.org/10.1145/359576.359585</a>.
</div>
<div id="ref-Hutton1992-vc" class="csl-entry" role="listitem">
Hutton, Graham. <span>“Higher-Order Functions for Parsing.”</span> <em>Journal of Functional Programming</em> 2, no. 3 (1992): 323–43. <a href="https://doi.org/10.1017/S0956796800000411" target="_blank" rel="noopener">https://doi.org/10.1017/S0956796800000411</a>.
</div>
<div id="ref-Knuth1997-rv" class="csl-entry" role="listitem">
Knuth, Donald E. <span>“Coroutines.”</span> In <em>The Art of Computer Programming: Volume 1: Fundamental Algorithms</em>, 3rd ed., 193–200. Addison Wesley, 1997.
</div>
<div id="ref-Watson2017" class="csl-entry" role="listitem">
Watson, Des. <span>“Approaches to Syntax Analysis.”</span> In <em>A Practical Approach to Compiler Construction</em>, 75–93. Springer International Publishing, 2017. <a href="https://doi.org/10.1007/978-3-319-52789-5_4" target="_blank" rel="noopener">https://doi.org/10.1007/978-3-319-52789-5_4</a>.
</div>
</div>
<section id="footnotes" class="footnotes footnotes-end-of-document" role="doc-endnotes">
<hr></hr>
<ol>
<li id="fn1"><p>Coroutines are often conflated with <a href="https://en.wikipedia.org/wiki/Fiber_(computer_science)" target="_blank" rel="noopener"><em>Fibers</em></a>. Some implementations of coroutines call themselves fibers instead. Quoting Wikipedia:</p>
<blockquote>
<p>Fibers describe essentially the same concept as coroutines. The distinction, if
there is any, is that coroutines are a language-level construct, a form of control
flow, while fibers are a systems-level construct, viewed as threads that happen
to not run in parallel. It is contentious which of the two concepts has priority:
fibers may be viewed as an implementation of coroutines, or as a substrate on which
to implement coroutines.</p>
</blockquote>
<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></li>
<li id="fn2"><p><a href="https://en.wikipedia.org/wiki/Green_threads" target="_blank" rel="noopener"><em>Green threads</em></a> are another popular solution for lightweight concurrency. Unlike coroutines, green threads are preemtable. Unlike <a href="https://en.wikipedia.org/wiki/Thread_(computing)" target="_blank" rel="noopener"><em>Threads</em></a>, they are scheduled by the language runtime instead of the operating system.<a href="#fnref2" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn3"><p><code>print</code> is a built-in function in <span class="fancy">Co</span>. It behaves same as the <code>console.log</code> function in JavaScript, or the <a href="https://hackage.haskell.org/package/base-4.15.0.0/docs/Prelude.html#v:print" target="_blank" rel="noopener"><code>print</code></a> function in Haskell.<a href="#fnref3" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn4"><p>The complete code for the interpreter is <a href="https://abhinavsarkar.net/code/co-interpreter.html?mtm_campaign=feed">here</a>.<a href="#fnref4" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn5"><p>If you are a Cabal user instead, the process is a little longer. First run:</p>
<pre class="plain"><code>cabal repl \
--build-depends "megaparsec ^>= 9.0" \
--build-depends "lifted-base ^>= 0.2" \
--build-depends "clock ^>= 0.8" \
--build-depends "pqueue ^>= 1.4" \
--build-depends "pretty-simple ^>= 4.0"</code></pre>
<p>Then in the started GHCi REPL, run <code>:load co-interpreter.hs</code>. You should be able to use the functions in the interpreter now.<a href="#fnref5" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn6"><p>We are using:</p>
<ul>
<li>the <a href="https://hackage.haskell.org/package/containers" target="_blank" rel="noopener">containers</a> and <a href="https://hackage.haskell.org/package/pqueue" target="_blank" rel="noopener">pqueue</a> libraries for data structures,</li>
<li>the <a href="https://hackage.haskell.org/package/clock" target="_blank" rel="noopener">clock</a> library for getting system time,</li>
<li>the <a href="https://hackage.haskell.org/package/megaparsec" target="_blank" rel="noopener">megaparsec</a> library for parsing,</li>
<li>the <a href="https://hackage.haskell.org/package/mtl" target="_blank" rel="noopener">mtl</a> and <a href="https://hackage.haskell.org/package/lifted-base" target="_blank" rel="noopener">lifted-base</a> libraries for the Monad stack and operations, and</li>
<li>the <a href="https://hackage.haskell.org/package/pretty-simple" target="_blank" rel="noopener">pretty-simple</a> library for pretty printing data structures.</li>
</ul>
<a href="#fnref6" class="footnote-back" role="doc-backlink">↩︎</a></li>
<li id="fn7"><p>I explore parsing in more depth by writing a JSON parser from scratch in Haskell in <a href="https://abhinavsarkar.net/posts/json-parsing-from-scratch-in-haskell/?mtm_campaign=feed">one of my previous posts</a>.<a href="#fnref7" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn8"><p>I followed <a href="https://github.com/mrkkrp/megaparsec-site/blob/master/tutorials/parsing-simple-imperative-language.md" target="_blank" rel="noopener">this</a> (outdated) Megaparsec tutorial for writing a parser for an imperative language. <a href="https://markkarpov.com/tutorial/megaparsec.html" target="_blank" rel="noopener">This tutorial</a> is also helpful in learning the intricacies of Megaparsec.<a href="#fnref8" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn9"><p><code>term</code> is a <a href="https://en.wikipedia.org/wiki/Recursive_descent_parser" target="_blank" rel="noopener"><em>Recursive descent parser</em></a>. If we directly include call parsing as an alternative in the <code>primary</code> parser, we would run into the <a href="https://en.wikipedia.org/wiki/Left_recursion" target="_blank" rel="noopener"><em>Left recursion</em></a> problem, causing the parser to hang in infinite loop.<a href="#fnref9" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn10"><p>In the parsing parlance this is called <em><a href="https://en.wikipedia.org/wiki/Backtracking" target="_blank" rel="noopener">Backtracking</a></em>. The alternative is to do a <em><a href="https://en.wikipedia.org/wiki/Parsing#Lookahead" target="_blank" rel="noopener">Lookahead</a></em><sup><a href="#ref-Watson2017" class="citation" title="(Watson, “Approaches to Syntax Analysis”)
">@14</a></sup>.<a href="#fnref10" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn11"><p>An earlier version of this post used the <code>symbol</code> parser for parsing keywords instead of the <code>reserved</code> parser. That would cause the parser to fail to parse variable names like <code>nullx</code> that start with a keyword. Using the <code>reserved</code> parser fixes this issue because of the built-in backtracking using <code>try</code>.<a href="#fnref11" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
</ol>
</section><p>If you liked this post, please <a href="https://abhinavsarkar.net/posts/implementing-co-1/?mtm_campaign=feed#syndications">leave a comment</a>.</p><img referrerpolicy="no-referrer-when-downgrade" src="https://anna.abhinavsarkar.net/matomo.php?idsite=1&rec=1" style="border:0" alt="" />2021-04-24T00:00:00Z<p>Many major programming languages these days support some lightweight concurrency primitives. The most recent popular ones are <a href="https://en.wikipedia.org/wiki/Go_(programming_language)#Concurrency:_goroutines_and_channels" target="_blank" rel="noopener">Goroutines</a> in <a href="https://golang.org/" target="_blank" rel="noopener">Go</a>, <a href="https://kotlinlang.org/docs/coroutines-basics.html" target="_blank" rel="noopener">Coroutines</a> in <a href="https://kotlinlang.org/" target="_blank" rel="noopener">Kotlin</a> and <a href="https://rust-lang.github.io/async-book/01_getting_started/02_why_async.html" target="_blank" rel="noopener">Async</a> in <a href="https://www.rust-lang.org/" target="_blank" rel="noopener">Rust</a>. Let’s explore some of these concepts in detail by implementing a programming language with support for coroutines and Go-style channels.</p>