Ricardo Martins2020-03-16T14:59:33-05:00http://ricardomartins.cc/Ricardo Martinshttp://ricardomartins.cc/2016/08/03/convenient_and_idiomatic_conversions_in_rustConvenient and idiomatic conversions in Rust2016-08-03T00:00:00-05:002016-08-03T00:00:00-05:00Ricardo Martinshttp://ricardomartins.cc/<div class="summary">
<h1 id="key-takeaways">Key takeaways</h1>
<ul>
<li>The traits in <code class="highlighter-rouge">std::convert</code> provide a uniform API for converting values to other types</li>
<li><code class="highlighter-rouge">From<T></code> and <code class="highlighter-rouge">Into<U></code> are for conversions that <em>cannot</em> fail and consume the original value</li>
<li><code class="highlighter-rouge">From<T> for U</code> converts a value of type <code class="highlighter-rouge">T</code> into one of type <code class="highlighter-rouge">U</code></li>
<li><code class="highlighter-rouge">Into<U> for T</code> inverts <code class="highlighter-rouge">From<T></code>’s subject-object relationship</li>
<li>Implementing <code class="highlighter-rouge">From<T> for U</code> gives us an automatically derived <code class="highlighter-rouge">Into<U> for T</code> implementation</li>
<li><code class="highlighter-rouge">TryFrom<T></code> and <code class="highlighter-rouge">TryInto<U></code> are the equivalent traits for conversions that may fail</li>
<li><code class="highlighter-rouge">AsRef<T></code> and <code class="highlighter-rouge">AsMut<T></code> represent cheap reference-to-reference conversions, with some similarities to <code class="highlighter-rouge">Borrow<T></code> and <code class="highlighter-rouge">BorrowMut<T></code></li>
</ul>
</div>
<h1 id="introduction">Introduction</h1>
<p>We all convert data from one representation to another with some regularity. There are several situations where this need pops up: converting a wide array of types into a more convenient type, converting “foreign” error types to our libraries’ error types, and encoding and decoding network packets of our custom protocols. The first situation is probably the most common. For instance, in some cases a plain <code class="highlighter-rouge">Vec<T></code> is a convenient representation, so there are readily available ways to convert values of other types, such as <code class="highlighter-rouge">VecDeque<T></code>, <code class="highlighter-rouge">BinaryHeap<T></code>, <code class="highlighter-rouge">&[T]</code>, and <code class="highlighter-rouge">&str</code>, into <code class="highlighter-rouge">Vec<T></code>.</p>
<p>Naturally, there is more one way to convert types in Rust, each with advantages and disadvantages. We could:</p>
<ul>
<li>build the target types ourselves with struct literals, but that’s tedious, repetitive and exposes implementation details;</li>
<li>create specialized constructors for each source type (e.g.: <code class="highlighter-rouge">new_from_vec_deque</code>, <code class="highlighter-rouge">new_from_binary_heap</code>, <code class="highlighter-rouge">new_from_slice</code>), but that’s just as tedious and we might miss some cases anyway;</li>
<li>write generic constructors that accept a certain trait, but that might still cover less cases than we need and require additional constructors;</li>
<li>cast enum variants to integers and vice-versa, but those conversions may have <a href="https://github.com/rust-lang/rust/issues/18154">unexpected results</a>;</li>
<li>…</li>
</ul>
<p>You get the idea: there are myriad ways of converting types, but many of them suck. <em>There has to be a better way!</em></p>
<p>In this article, we’ll explore how to do it in a more idiomatic way — and if you read the key takeaways you already know how 😉. The traits in the <a href="https://doc.rust-lang.org/std/convert/index.html"><code class="highlighter-rouge">std::convert</code></a> module — <a href="https://doc.rust-lang.org/std/convert/trait.From.html"><code class="highlighter-rouge">From<T></code></a>, <a href="https://doc.rust-lang.org/std/convert/trait.Into.html"><code class="highlighter-rouge">Into<U></code></a>, <a href="https://doc.rust-lang.org/std/convert/trait.TryFrom.html"><code class="highlighter-rouge">TryFrom<T></code></a>, <a href="https://doc.rust-lang.org/std/convert/trait.TryInto.html"><code class="highlighter-rouge">TryInto<U></code></a>, <a href="https://doc.rust-lang.org/std/convert/trait.AsRef.html"><code class="highlighter-rouge">AsRef<U></code></a>, and <a href="https://doc.rust-lang.org/std/convert/trait.AsMut.html"><code class="highlighter-rouge">AsMut<U></code></a> — have this exact purpose. Those traits provide a uniform API for type conversion, and we’ll be exploring how we can leverage them to achieve a consistent and ergonomic API. Once you know about them, you’ll start noticing them everywhere in the documentation. I hope that, by the end of this article you’ll probably appreciate them as much as I do.</p>
<h1 id="from-and-into"><code class="highlighter-rouge">From</code> and <code class="highlighter-rouge">Into</code></h1>
<p><a href="https://doc.rust-lang.org/std/convert/trait.From.html"><code class="highlighter-rouge">From<T></code></a> represents the conversion of a value of type <code class="highlighter-rouge">T</code> into a target type (<code class="highlighter-rouge">impl From<T> for TargetType</code>). This conversion may or may not be computationally expensive, but we can usually assume it isn’t cheap. Let’s have a look at its <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/convert.rs#L156-L161">definition</a>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="p">[</span><span class="nf">stable</span><span class="p">(</span><span class="n">feature</span> <span class="o">=</span> <span class="s">"rust1"</span><span class="p">,</span> <span class="n">since</span> <span class="o">=</span> <span class="s">"1.0.0"</span><span class="p">)]</span>
<span class="k">pub</span> <span class="k">trait</span> <span class="n">From</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">:</span> <span class="n">Sized</span> <span class="p">{</span>
<span class="c">/// Performs the conversion.</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">fn</span> <span class="nf">from</span><span class="p">(</span><span class="n">T</span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="highlighter-rouge">From<T></code> contains a single method signature, <code class="highlighter-rouge">from()</code>, which we’ll have to implement to perform the conversion. Inspecting <code class="highlighter-rouge">from()</code>’s signature, we can tell that it moves (or consumes) the argument. Its return value, <code class="highlighter-rouge">Self</code>, also clues us in to the fact that the conversion <em>may not fail</em>. Later in this article, we’ll look into <code class="highlighter-rouge">TryFrom<T></code> for conversions that may fail. <code class="highlighter-rouge">From<T></code> is also a reflexive trait, which means that conversion of a value into its own type (<code class="highlighter-rouge">From<T> for T</code>) is <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/convert.rs#L239-L243">implemented</a> and returns the argument without modification.</p>
<p>Reading on, we arrive at the symmetrical companion trait of <code class="highlighter-rouge">From</code>, <a href="https://doc.rust-lang.org/std/convert/trait.Into.html"><code class="highlighter-rouge">Into<T></code></a>. Like <code class="highlighter-rouge">From</code>, <code class="highlighter-rouge">Into</code> has a short <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/convert.rs#L129-L134">definition</a>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="p">[</span><span class="nf">stable</span><span class="p">(</span><span class="n">feature</span> <span class="o">=</span> <span class="s">"rust1"</span><span class="p">,</span> <span class="n">since</span> <span class="o">=</span> <span class="s">"1.0.0"</span><span class="p">)]</span>
<span class="k">pub</span> <span class="k">trait</span> <span class="n">Into</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">:</span> <span class="n">Sized</span> <span class="p">{</span>
<span class="c">/// Performs the conversion.</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">fn</span> <span class="nf">into</span><span class="p">(</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">T</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>As we can see in the definition, <code class="highlighter-rouge">Into::into()</code> consumes <code class="highlighter-rouge">self</code> and returns <code class="highlighter-rouge">T</code>, the opposite of <code class="highlighter-rouge">From::from()</code>, which consumes an argument <code class="highlighter-rouge">T</code> and returning <code class="highlighter-rouge">Self</code>. Compare both ways of converting values:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">// `from` can be called from either the `From` trait or the target type.</span>
<span class="c">// Calling from the target type makes our intention clearer.</span>
<span class="k">let</span> <span class="n">converted_value</span> <span class="o">=</span> <span class="nn">From</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">original_value</span><span class="p">);</span>
<span class="k">let</span> <span class="n">converted_value</span> <span class="o">=</span> <span class="nn">TargetType</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">original_value</span><span class="p">);</span>
<span class="c">// `into` is usually called directly on the original value, but we can</span>
<span class="c">// also call it from the Into trait or the source type:</span>
<span class="k">let</span> <span class="n">converted_value</span> <span class="o">=</span> <span class="n">original_value</span><span class="nf">.into</span><span class="p">();</span>
<span class="k">let</span> <span class="n">converted_value</span> <span class="o">=</span> <span class="nn">Into</span><span class="p">::</span><span class="nf">into</span><span class="p">(</span><span class="n">original_value</span><span class="p">);</span>
</code></pre></div></div>
<p>While <code class="highlighter-rouge">From::from()</code> focuses on the target type, <code class="highlighter-rouge">Into::into()</code> focuses on the original value; yet both express the same conversion. All the conversions above are equivalent, choosing one of them is a matter of taste. Personally, I prefer using <code class="highlighter-rouge">TargetType::from(value)</code> and <code class="highlighter-rouge">value.into()</code>. The former makes our intention clearer, while the latter is shorter than <code class="highlighter-rouge">Into::into(value)</code>. Note that we might need to add type annotations to disambiguate the intended target type if we opt any form other than <code class="highlighter-rouge">TargetType::from()</code>, which clearly indicates it.</p>
<p>A nice thing about implementing <code class="highlighter-rouge">From<T> for U</code> is that it implies <code class="highlighter-rouge">Into<U>
for T</code>, which means we get an automatic <code class="highlighter-rouge">Into</code> implementation <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/convert.rs#L231-L237">for free</a> (the opposite isn’t true):</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">// From implies Into</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="p">,</span> <span class="n">U</span><span class="o">></span> <span class="n">Into</span><span class="o"><</span><span class="n">U</span><span class="o">></span> <span class="k">for</span> <span class="n">T</span> <span class="n">where</span> <span class="n">U</span><span class="p">:</span> <span class="n">From</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">into</span><span class="p">(</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">U</span> <span class="p">{</span>
<span class="nn">U</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>A secondary advantage of having an <code class="highlighter-rouge">Into</code> implementation (it doesn’t matter if it’s explicit or automatic) is that we can use it to broaden function arguments from a specific type to any type can be converted into the target type, as shown in the following example:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">// Instead of targetting a specific type like this:</span>
<span class="k">fn</span> <span class="nf">do_something</span><span class="p">(</span><span class="n">value</span><span class="p">:</span> <span class="n">TargetType</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">)</span> <span class="p">{</span>
<span class="c">// ...</span>
<span class="p">}</span>
<span class="c">// We can broaden the accepted types with the following:</span>
<span class="k">fn</span> <span class="n">do_something</span><span class="o"><</span><span class="n">U</span><span class="p">:</span> <span class="n">Into</span><span class="o"><</span><span class="n">TargetType</span><span class="o"><</span><span class="n">T</span><span class="o">>>></span><span class="p">(</span><span class="n">value</span><span class="p">:</span> <span class="n">U</span><span class="p">)</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">converted_value</span> <span class="o">=</span> <span class="n">value</span><span class="nf">.into</span><span class="p">();</span>
<span class="c">// ...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Alright, that’s enough theory. A couple of examples will make it easier to understand how all this works in practice.</p>
<h2 id="example-sortedvect">Example: <code class="highlighter-rouge">SortedVec<T></code></h2>
<p>Suppose we have a sorted vector type, <code class="highlighter-rouge">SortedVec<T></code>. Since it’s a general data structure, building a <code class="highlighter-rouge">SortedVec<T></code> from slice- and list-like types makes sense, so we’ll implement those conversions:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">/// Our simple sorted vector structure is just a wrapper around a Vec</span>
<span class="k">struct</span> <span class="n">SortedVec</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">(</span><span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">);</span>
<span class="c">/// Converting slices into SortedVec is pretty much expected.</span>
<span class="k">impl</span><span class="o"><</span><span class="nv">'a</span><span class="p">,</span> <span class="n">T</span><span class="p">:</span> <span class="nb">Ord</span> <span class="o">+</span> <span class="n">Clone</span><span class="o">></span> <span class="n">From</span><span class="o"><&</span><span class="nv">'a</span> <span class="p">[</span><span class="n">T</span><span class="p">]</span><span class="o">></span> <span class="k">for</span> <span class="n">SortedVec</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">from</span><span class="p">(</span><span class="n">slice</span><span class="p">:</span> <span class="o">&</span><span class="p">[</span><span class="n">T</span><span class="p">])</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">vec</span> <span class="o">=</span> <span class="n">slice</span><span class="nf">.to_owned</span><span class="p">();</span>
<span class="n">vec</span><span class="nf">.sort</span><span class="p">();</span>
<span class="nf">SortedVec</span><span class="p">(</span><span class="n">vec</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c">/// Converting a Vec is also expected.</span>
<span class="c">/// We can sort the vector in place and then put it inside SortedVec.</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="nb">Ord</span> <span class="o">+</span> <span class="n">Clone</span><span class="o">></span> <span class="n">From</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">>></span> <span class="k">for</span> <span class="n">SortedVec</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">from</span><span class="p">(</span><span class="k">mut</span> <span class="n">vec</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="n">vec</span><span class="nf">.sort</span><span class="p">();</span>
<span class="nf">SortedVec</span><span class="p">(</span><span class="n">vec</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c">/// Converting a LinkedList also makes sense, but it has no</span>
<span class="c">/// slice representation, so we'll have to rely on its iterator.</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="nb">Ord</span> <span class="o">+</span> <span class="n">Clone</span><span class="o">></span> <span class="n">From</span><span class="o"><</span><span class="n">LinkedList</span><span class="o"><</span><span class="n">T</span><span class="o">>></span> <span class="k">for</span> <span class="n">SortedVec</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">from</span><span class="p">(</span><span class="n">list</span><span class="p">:</span> <span class="n">LinkedList</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">vec</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="o">=</span> <span class="n">list</span><span class="nf">.iter</span><span class="p">()</span><span class="nf">.cloned</span><span class="p">()</span><span class="nf">.collect</span><span class="p">();</span>
<span class="n">vec</span><span class="nf">.sort</span><span class="p">();</span>
<span class="nf">SortedVec</span><span class="p">(</span><span class="n">vec</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Now, you might protest that the conversion from <code class="highlighter-rouge">Vec<T></code> is redundant, because we can get a slice from the vector and then convert the slice. That’s absolutely correct, dear reader. However, the implementation above avoids cloning the vector, and, in my opinion, hiding any intermediate steps leads to a more pleasant API.</p>
<p>As a result of the trait implementations above, we can call <code class="highlighter-rouge">SortedVec::from()</code> without caring if the argument is a slice, <code class="highlighter-rouge">Vec</code> or <code class="highlighter-rouge">LinkedList</code>.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">vec</span> <span class="o">=</span> <span class="nd">vec!</span><span class="p">[</span><span class="mi">1u8</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">];</span>
<span class="c">// Convert a slice</span>
<span class="k">let</span> <span class="n">sorted</span> <span class="o">=</span> <span class="nn">SortedVec</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="o">&</span><span class="n">vec</span><span class="p">[</span><span class="mi">1</span><span class="o">..</span><span class="p">]);</span>
<span class="c">// ... a vector</span>
<span class="k">let</span> <span class="n">sorted</span> <span class="o">=</span> <span class="nn">SortedVec</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">vec</span><span class="p">);</span>
<span class="c">// ... a linked list</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">linked_list</span><span class="p">:</span> <span class="n">LinkedList</span><span class="o"><</span><span class="nb">u8</span><span class="o">></span> <span class="o">=</span> <span class="nn">LinkedList</span><span class="p">::</span><span class="nf">new</span><span class="p">();</span>
<span class="n">linked_list</span><span class="nf">.extend</span><span class="p">(</span><span class="o">&</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]);</span>
<span class="k">let</span> <span class="n">sorted</span> <span class="o">=</span> <span class="nn">SortedVec</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="n">linked_list</span><span class="p">);</span>
</code></pre></div></div>
<p>We can also go in the opposite direction and implement conversions from <code class="highlighter-rouge">SortedVec<T></code> into other types (for instance, into <code class="highlighter-rouge">Vec<T></code>). However, there are some restrictions about implementing traits for non-local, generic types — check <a href="https://doc.rust-lang.org/error-index.html#E0210">error 0210</a> and the related <a href="https://github.com/rust-lang/rfcs/pull/1023">Rust RFC 1023</a>. As a rule of thumb, if the non-local type isn’t generic over some type parameter, you can implement <code class="highlighter-rouge">From</code> for it.</p>
<h2 id="example-packettype">Example: <code class="highlighter-rouge">PacketType</code></h2>
<p>Let’s take a different example. Suppose we are now implementing a library for a network protocol where the first byte in a packet header tells us the packet type. A reasonable solution is representing the packet types with an enumeration, where each variant maps to a packet type. For instance:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">/// Represents a packet type.</span>
<span class="c">/// Associated with each variant is its raw numeric representation.</span>
<span class="k">enum</span> <span class="n">PacketType</span> <span class="p">{</span>
<span class="n">Data</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="c">// packet carries a data payload</span>
<span class="n">Fin</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span> <span class="c">// signals the end of a connection</span>
<span class="n">State</span> <span class="o">=</span> <span class="mi">2</span><span class="p">,</span> <span class="c">// signals acknowledgment of a packet</span>
<span class="n">Reset</span> <span class="o">=</span> <span class="mi">3</span><span class="p">,</span> <span class="c">// forcibly terminates a connection</span>
<span class="n">Syn</span> <span class="o">=</span> <span class="mi">4</span><span class="p">,</span> <span class="c">// initiates a new connection with a peer</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Given this representation, how shall we convert to and from the byte representation?</p>
<p>The traditional way, very common in C and C++ programs, is to simply cast the values from one type to another. That can also be done in Rust; for instance, converting <code class="highlighter-rouge">PacketType::Data</code> into a byte is as simple as <code class="highlighter-rouge">PacketType::Data as
u8</code>. That seems to take care of encoding a <code class="highlighter-rouge">PacketType</code> into a byte representation, but we aren’t done yet.</p>
<p>Did you notice that each <code class="highlighter-rouge">PacketType</code> variant has an associated value? They define the variants’ representation in the generated code. If we followed the usual Rust style and didn’t assign the variants any values, the numeric representation of each variant would depend on the order they are declared, which can lead to errors if we simply cast enum variants into numeric types. A better way to convert the enum variants to the correct values is an explicit <code class="highlighter-rouge">match</code>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">impl</span> <span class="n">From</span><span class="o"><</span><span class="n">PacketType</span><span class="o">></span> <span class="k">for</span> <span class="nb">u8</span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">from</span><span class="p">(</span><span class="n">original</span><span class="p">:</span> <span class="n">PacketType</span><span class="p">)</span> <span class="k">-></span> <span class="nb">u8</span> <span class="p">{</span>
<span class="k">match</span> <span class="n">original</span> <span class="p">{</span>
<span class="nn">PacketType</span><span class="p">::</span><span class="n">Data</span> <span class="k">=></span> <span class="mi">0</span><span class="p">,</span>
<span class="nn">PacketType</span><span class="p">::</span><span class="n">Fin</span> <span class="k">=></span> <span class="mi">1</span><span class="p">,</span>
<span class="nn">PacketType</span><span class="p">::</span><span class="n">State</span> <span class="k">=></span> <span class="mi">2</span><span class="p">,</span>
<span class="nn">PacketType</span><span class="p">::</span><span class="n">Reset</span> <span class="k">=></span> <span class="mi">3</span><span class="p">,</span>
<span class="nn">PacketType</span><span class="p">::</span><span class="n">Syn</span> <span class="k">=></span> <span class="mi">4</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Pretty straightforward, right? Since the mapping from <code class="highlighter-rouge">PacketType</code> to <code class="highlighter-rouge">u8</code> is contained in the implementation of <code class="highlighter-rouge">From</code>, we can remove the values assigned to <code class="highlighter-rouge">PacketType</code>’s variants, resulting in a cleaner enum definition.</p>
<h3 id="what-about-the-opposite-conversion">What about the opposite conversion?</h3>
<p>According to the <a href="https://www.rust-lang.org/en-US/faq.html#how-can-i-convert-a-c-style-enum-to-an-integer">Frequently Asked Questions</a>, converting an enum into an integer can be achieved with a cast, as we saw. However, the opposite conversion can (and I argue that, in many cases, it <em>should</em>) be made with a <code class="highlighter-rouge">match</code> statement. For ease of use and better ergonomics, implementing <code class="highlighter-rouge">From<T></code> for conversions in both directions is usually a good idea.</p>
<p>Casting a <code class="highlighter-rouge">PacketType</code> to <code class="highlighter-rouge">u8</code> is generally safe and correct, with the caveats we saw before, because for every <code class="highlighter-rouge">PacketType</code> variant, there’s a corresponding representation compatible with <code class="highlighter-rouge">u8</code>. However, the reverse is decidedly <strong>not</strong> true: converting an <code class="highlighter-rouge">u8</code> value without a corresponding <code class="highlighter-rouge">PacketType</code> variant is undefined behavior! Quoth the <a href="https://doc.rust-lang.org/reference.html#behavior-considered-undefined">Rust reference</a>:</p>
<blockquote>
<p><strong>Behavior considered undefined</strong></p>
<ul>
<li>Invalid values in primitive types, even in private fields/locals:
<ul>
<li>A discriminant in an enum not included in the type definition</li>
</ul>
</li>
</ul>
</blockquote>
<p>Although we can map any <code class="highlighter-rouge">PacketType</code> variant into an <code class="highlighter-rouge">u8</code> value, we can’t do the reverse and map any <code class="highlighter-rouge">u8</code> into a <code class="highlighter-rouge">PacketType</code>: there are too many <code class="highlighter-rouge">u8s</code> and not enough <code class="highlighter-rouge">PacketTypes</code>!</p>
<p>So for the <code class="highlighter-rouge">u8</code> to <code class="highlighter-rouge">PacketType</code> conversion, we can’t simply match on <code class="highlighter-rouge">u8</code> value and return the appropriate <code class="highlighter-rouge">PacketType</code> variant like we did for the opposite conversion. We need a way to signal that the conversion failed, but calling <code class="highlighter-rouge">panic!()</code> is not an acceptable option. We need a fallible <code class="highlighter-rouge">From</code>.</p>
<h1 id="do-or-do-not-there-is-no-try">“Do or do not; there is no <code class="highlighter-rouge">Try</code>”</h1>
<p>We saw that the conversions made by <code class="highlighter-rouge">From</code> and <code class="highlighter-rouge">Into</code> must not fail. However, sometimes we deal with types that don’t fully map onto one another, so we need fallible versions of those traits. Fortunately, there’s both <code class="highlighter-rouge">TryFrom</code> and <code class="highlighter-rouge">TryInto</code>, which return a <code class="highlighter-rouge">Result<TargetType, ErrorType></code>. Both live in <code class="highlighter-rouge">std::convert</code> along with their infallible siblings, but their exact details and implications are <a href="https://github.com/rust-lang/rust/issues/33417">still under debate</a>, which means they’re still marked as unstable. To use them, we can restrict ourselves to the nightly version of the compiler, use the <a href="https://crates.io/crates/try_from"><code class="highlighter-rouge">try_from</code> crate</a>, or paste <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/convert.rs#L163-L185">their definitions</a> somewhere in our crates (they’re really short).</p>
<p>Let’s have a look at <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/convert.rs#L177-L185"><code class="highlighter-rouge">TryFrom</code>’s definition</a> (as of Rust 1.10.0):</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="p">[</span><span class="nf">unstable</span><span class="p">(</span><span class="n">feature</span> <span class="o">=</span> <span class="s">"try_from"</span><span class="p">,</span> <span class="n">issue</span> <span class="o">=</span> <span class="s">"33417"</span><span class="p">)]</span>
<span class="k">pub</span> <span class="k">trait</span> <span class="n">TryFrom</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">:</span> <span class="n">Sized</span> <span class="p">{</span>
<span class="c">/// The type returned in the event of a conversion error.</span>
<span class="k">type</span> <span class="nb">Err</span><span class="p">;</span>
<span class="c">/// Performs the conversion.</span>
<span class="k">fn</span> <span class="nf">try_from</span><span class="p">(</span><span class="n">T</span><span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><</span><span class="n">Self</span><span class="p">,</span> <span class="nn">Self</span><span class="p">::</span><span class="nb">Err</span><span class="o">></span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>First we have a stability <a href="https://doc.rust-lang.org/reference.html#attributes">attribute</a> marking the trait as unstable, followed by the trait definition itself. We can see it has an associated type, <code class="highlighter-rouge">Err</code>, for the cases where the conversion fails. As expected, we have a <code class="highlighter-rouge">try_from</code> method instead of <code class="highlighter-rouge">from</code>, which returns <code class="highlighter-rouge">Result<Self, Self::Err></code> instead of <code class="highlighter-rouge">Self</code>.</p>
<p>Keeping with our example, we would have:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">impl</span> <span class="n">TryFrom</span><span class="o"><</span><span class="nb">u8</span><span class="o">></span> <span class="k">for</span> <span class="n">PacketType</span> <span class="p">{</span>
<span class="k">type</span> <span class="nb">Err</span> <span class="o">=</span> <span class="n">ParseError</span><span class="p">;</span>
<span class="k">fn</span> <span class="nf">try_from</span><span class="p">(</span><span class="n">original</span><span class="p">:</span> <span class="nb">u8</span><span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><</span><span class="n">Self</span><span class="p">,</span> <span class="nn">Self</span><span class="p">::</span><span class="nb">Err</span><span class="o">></span> <span class="p">{</span>
<span class="k">match</span> <span class="n">original</span> <span class="p">{</span>
<span class="mi">0</span> <span class="k">=></span> <span class="nf">Ok</span><span class="p">(</span><span class="nn">PacketType</span><span class="p">::</span><span class="n">Data</span><span class="p">),</span>
<span class="mi">1</span> <span class="k">=></span> <span class="nf">Ok</span><span class="p">(</span><span class="nn">PacketType</span><span class="p">::</span><span class="n">Fin</span><span class="p">),</span>
<span class="mi">2</span> <span class="k">=></span> <span class="nf">Ok</span><span class="p">(</span><span class="nn">PacketType</span><span class="p">::</span><span class="n">State</span><span class="p">),</span>
<span class="mi">3</span> <span class="k">=></span> <span class="nf">Ok</span><span class="p">(</span><span class="nn">PacketType</span><span class="p">::</span><span class="n">Reset</span><span class="p">),</span>
<span class="mi">4</span> <span class="k">=></span> <span class="nf">Ok</span><span class="p">(</span><span class="nn">PacketType</span><span class="p">::</span><span class="n">Syn</span><span class="p">),</span>
<span class="n">n</span> <span class="k">=></span> <span class="nf">Err</span><span class="p">(</span><span class="nn">ParseError</span><span class="p">::</span><span class="nf">InvalidPacketType</span><span class="p">(</span><span class="n">n</span><span class="p">))</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>In this example, we return the corresponding <code class="highlighter-rouge">PacketType</code> variant for values which can be mapped and an error for the remaining ones. This error type preserves the original value, which is potentially useful for debugging purposes, but we could just discard it instead.</p>
<h1 id="asref-and-asmut"><code class="highlighter-rouge">AsRef</code> and <code class="highlighter-rouge">AsMut</code></h1>
<p>Last but not least, we’re going to examine the remaining traits in the <code class="highlighter-rouge">std::convert</code> module: <a href="https://doc.rust-lang.org/std/convert/trait.AsRef.html"><code class="highlighter-rouge">AsRef<T></code></a> and <a href="https://doc.rust-lang.org/std/convert/trait.AsMut.html"><code class="highlighter-rouge">AsMut<T></code></a>. Like the other traits in this module, they are used to implement conversions among types. However, whereas the other traits consume values and may perform costly operations, <code class="highlighter-rouge">AsRef<T></code> and <code class="highlighter-rouge">AsMut<T></code> are used to implement cheap, reference-to-reference conversions.</p>
<p>As you have probably guessed from their names, <code class="highlighter-rouge">AsRef<T></code> converts an immutable reference to a value into another immutable reference, while <code class="highlighter-rouge">AsMut<T></code> does the same for mutable references.</p>
<p>Since they’re both very similar, we’re going to explore them at the same time. Let’s start with <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/convert.rs#L77-L99">their definitions</a>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="p">[</span><span class="nf">stable</span><span class="p">(</span><span class="n">feature</span> <span class="o">=</span> <span class="s">"rust1"</span><span class="p">,</span> <span class="n">since</span> <span class="o">=</span> <span class="s">"1.0.0"</span><span class="p">)]</span>
<span class="k">pub</span> <span class="k">trait</span> <span class="n">AsRef</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="p">{</span>
<span class="c">/// Performs the conversion.</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">fn</span> <span class="nf">as_ref</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="o">&</span><span class="n">T</span><span class="p">;</span>
<span class="p">}</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">pub</span> <span class="k">trait</span> <span class="n">AsMut</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="p">{</span>
<span class="c">/// Performs the conversion.</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">fn</span> <span class="nf">as_mut</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span> <span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="o">&</span><span class="k">mut</span> <span class="n">T</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Both take references to <code class="highlighter-rouge">self</code> and return references to the target type with the same mutability as <code class="highlighter-rouge">self</code>. Using these traits requires no more than calling <code class="highlighter-rouge">as_ref()</code> or <code class="highlighter-rouge">as_mut()</code> on a value, depending on which conversion we need, like so: <code class="highlighter-rouge">value.as_ref()</code>.</p>
<p>Implementing <code class="highlighter-rouge">AsRef<T></code> and <code class="highlighter-rouge">AsMut<T></code> is sensible and easy when the source type is a wrapper around the target type, like the <code class="highlighter-rouge">SortedVec<T></code> example we used before. Since <code class="highlighter-rouge">SortedVec<T></code> relies on a <code class="highlighter-rouge">Vec<T></code>, implementing both traits is painless:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">/// SortedVec<T> is a tuple struct, containing a single Vec<T>.</span>
<span class="k">struct</span> <span class="n">SortedVec</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">(</span><span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">);</span>
<span class="c">/// Implementing AsRef<Vec<T>> for SortedVec<T> only requires</span>
<span class="c">/// returning a reference to SortedVec<T>'s single field.</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">AsRef</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">>></span> <span class="k">for</span> <span class="n">SortedVec</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">as_ref</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="o">&</span><span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="o">&</span><span class="k">self</span><span class="err">.</span><span class="mi">0</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c">/// Implementing AsMut<Vec<T>> is just as easy.</span>
<span class="c">/// Note that this allows the user to mutate the underlying Vec</span>
<span class="c">/// such that it's no longer sorted, so you might want to avoid</span>
<span class="c">/// implementing this trait!</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">AsMut</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">>></span> <span class="k">for</span> <span class="n">SortedVec</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">as_mut</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span> <span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="o">&</span><span class="k">mut</span> <span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="o">&</span><span class="k">mut</span> <span class="k">self</span><span class="err">.</span><span class="mi">0</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="highlighter-rouge">AsRef<T></code> and <code class="highlighter-rouge">AsMut<T></code> also allow us to broaden the argument type from a specific reference type to any type that can be cheaply converted to the target reference type, just like <code class="highlighter-rouge">Into<T></code>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="n">manipulate_vector</span><span class="o"><</span><span class="n">T</span><span class="p">,</span> <span class="n">V</span><span class="p">:</span> <span class="n">AsRef</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="n">T</span><span class="o">>>></span><span class="p">(</span><span class="n">vec</span><span class="p">:</span> <span class="n">V</span><span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><</span><span class="nb">usize</span><span class="p">,</span> <span class="p">()</span><span class="o">></span> <span class="p">{</span>
<span class="c">// ...</span>
<span class="p">}</span>
<span class="c">// Now we can call `manipulate_vector` with a Vec<T> or anything that can</span>
<span class="c">// be cheaply converted to Vec<T>, such as SortedVec<T>.</span>
<span class="k">let</span> <span class="n">sorted_vec</span> <span class="o">=</span> <span class="nn">SortedVec</span><span class="p">::</span><span class="nf">from</span><span class="p">(</span><span class="nd">vec!</span><span class="p">[</span><span class="mi">1u8</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]);</span>
<span class="k">match</span> <span class="nf">manipulate_vector</span><span class="p">(</span><span class="n">sorted_vec</span><span class="p">)</span> <span class="p">{</span>
<span class="c">// ...</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="highlighter-rouge">AsRef<T></code> and <code class="highlighter-rouge">AsMut<T></code> are very similar to <a href="https://doc.rust-lang.org/std/borrow/trait.Borrow.html"><code class="highlighter-rouge">Borrow<T></code></a> and <a href="https://doc.rust-lang.org/std/borrow/trait.BorrowMut.html"><code class="highlighter-rouge">BorrowMut<T></code></a>, but semantically different. The Rust Programming Language Book discusses those <a href="https://doc.rust-lang.org/book/borrow-and-asref.html">differences in detail</a>, but as a rule of thumb, we choose <code class="highlighter-rouge">AsRef<T></code> and <code class="highlighter-rouge">AsMut<T></code> when we want to convert references or when writing generic code, and <code class="highlighter-rouge">Borrow<T></code> and <code class="highlighter-rouge">BorrowMut<T></code> when we wish to disregard whether a value is owned or borrowed (for instance, we might want a value to have the same <a href="https://doc.rust-lang.org/std/hash/trait.Hash.html">hash</a> independently of it being owned or not).</p>
<p>There are a few interesting <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/convert.rs#L191-L229">generic implementations</a> for <code class="highlighter-rouge">AsRef<T></code> and <code class="highlighter-rouge">AsMut<T></code>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">// As lifts over &</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">impl</span><span class="o"><</span><span class="nv">'a</span><span class="p">,</span> <span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="p">,</span> <span class="n">U</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="n">AsRef</span><span class="o"><</span><span class="n">U</span><span class="o">></span> <span class="k">for</span> <span class="o">&</span><span class="nv">'a</span> <span class="n">T</span> <span class="n">where</span> <span class="n">T</span><span class="p">:</span> <span class="n">AsRef</span><span class="o"><</span><span class="n">U</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">as_ref</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="o">&</span><span class="n">U</span> <span class="p">{</span>
<span class="o"><</span><span class="n">T</span> <span class="k">as</span> <span class="n">AsRef</span><span class="o"><</span><span class="n">U</span><span class="o">>></span><span class="p">::</span><span class="nf">as_ref</span><span class="p">(</span><span class="o">*</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c">// As lifts over &mut</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">impl</span><span class="o"><</span><span class="nv">'a</span><span class="p">,</span> <span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="p">,</span> <span class="n">U</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="n">AsRef</span><span class="o"><</span><span class="n">U</span><span class="o">></span> <span class="k">for</span> <span class="o">&</span><span class="nv">'a</span> <span class="k">mut</span> <span class="n">T</span> <span class="n">where</span> <span class="n">T</span><span class="p">:</span> <span class="n">AsRef</span><span class="o"><</span><span class="n">U</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">as_ref</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="o">&</span><span class="n">U</span> <span class="p">{</span>
<span class="o"><</span><span class="n">T</span> <span class="k">as</span> <span class="n">AsRef</span><span class="o"><</span><span class="n">U</span><span class="o">>></span><span class="p">::</span><span class="nf">as_ref</span><span class="p">(</span><span class="o">*</span><span class="k">self</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c">// AsMut lifts over &mut</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">impl</span><span class="o"><</span><span class="nv">'a</span><span class="p">,</span> <span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="p">,</span> <span class="n">U</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="n">AsMut</span><span class="o"><</span><span class="n">U</span><span class="o">></span> <span class="k">for</span> <span class="o">&</span><span class="nv">'a</span> <span class="k">mut</span> <span class="n">T</span> <span class="n">where</span> <span class="n">T</span><span class="p">:</span> <span class="n">AsMut</span><span class="o"><</span><span class="n">U</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">as_mut</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span> <span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="o">&</span><span class="k">mut</span> <span class="n">U</span> <span class="p">{</span>
<span class="p">(</span><span class="o">*</span><span class="k">self</span><span class="p">)</span><span class="nf">.as_mut</span><span class="p">()</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Those generic implementations may look intimidating, but looks are deceiving. Reading them slowly, we can see the traits are implemented for <em>references</em> to types that implement <code class="highlighter-rouge">AsRef<U></code> or <code class="highlighter-rouge">AsMut<U></code> (<code class="highlighter-rouge">&'a T where T: AsRef<U></code>, <code class="highlighter-rouge">&'a
mut T where T: AsRef<U></code> and <code class="highlighter-rouge">&'a mut T where T: AsMut<U></code>). We can also see that every implementation dereferences the argument, which is a reference.</p>
<p>The result is rather useful: these trait implementations make references to references (to references to references…) behave as if they were simple, direct references. That is to say, they make multiple-level deep references such as <code class="highlighter-rouge">&&&&vec</code> (in the case of the first implementation) and <code class="highlighter-rouge">&&&& mut vec</code> (in the case of the second) equivalent to <code class="highlighter-rouge">&vec</code>, while the third implementation makes <code class="highlighter-rouge">&mut &mut vec</code> equivalent to <code class="highlighter-rouge">&mut vec</code>. After those conversions, any compatible conversions we explicitly implemented can be applied.</p>
<h1 id="closing-thoughts">Closing thoughts</h1>
<p>In this article we dove into <code class="highlighter-rouge">std::convert</code> and explored how we can use its traits — <code class="highlighter-rouge">From<T></code>, <code class="highlighter-rouge">Into<T></code>, <code class="highlighter-rouge">TryFrom<T></code>, <code class="highlighter-rouge">TryInto<T></code>, <code class="highlighter-rouge">AsRef<T></code> and <code class="highlighter-rouge">AsMut<T></code> — to achieve a uniform type conversion API. The table below summarizes the characteristics of those traits.</p>
<div class="center">
<table>
<thead>
<tr>
<th> </th>
<th>receives</th>
<th>returns</th>
<th>can fail?</th>
</tr>
</thead>
<tbody>
<tr>
<td><code class="highlighter-rouge">From<T></code></td>
<td><code class="highlighter-rouge">T</code></td>
<td><code class="highlighter-rouge">Self</code></td>
<td>❌</td>
</tr>
<tr>
<td><code class="highlighter-rouge">TryFrom<T></code></td>
<td><code class="highlighter-rouge">T</code></td>
<td><code class="highlighter-rouge">Result<Self, E></code></td>
<td>✅</td>
</tr>
<tr>
<td><code class="highlighter-rouge">Into<T></code></td>
<td><code class="highlighter-rouge">self</code></td>
<td><code class="highlighter-rouge">T</code></td>
<td>❌</td>
</tr>
<tr>
<td><code class="highlighter-rouge">TryInto<T></code></td>
<td><code class="highlighter-rouge">self</code></td>
<td><code class="highlighter-rouge">Result<T, E></code></td>
<td>✅</td>
</tr>
<tr>
<td><code class="highlighter-rouge">AsRef<T></code></td>
<td><code class="highlighter-rouge">&self</code></td>
<td><code class="highlighter-rouge">&T</code></td>
<td>❌</td>
</tr>
<tr>
<td><code class="highlighter-rouge">AsMut<T></code></td>
<td><code class="highlighter-rouge">&mut</code> <code class="highlighter-rouge">self</code></td>
<td><code class="highlighter-rouge">&mut</code> <code class="highlighter-rouge">T</code></td>
<td>❌</td>
</tr>
</tbody>
</table>
</div>
<p>In short:</p>
<ul>
<li><code class="highlighter-rouge">From<T></code>, <code class="highlighter-rouge">Into<T></code>, and their fallible counterparts, <code class="highlighter-rouge">TryFrom<T></code> and <code class="highlighter-rouge">TryInto<T></code>, operate on <em>values</em> and perform conversions that may be costly;</li>
<li><code class="highlighter-rouge">AsRef<T></code> and <code class="highlighter-rouge">AsMut<T></code>, on the other hand, take references to values and perform cheap reference-to-reference conversions;</li>
<li>We can look at <code class="highlighter-rouge">From::from()</code> as if it were a woodchipper that we can feed with any of the approved types of wood;</li>
<li><code class="highlighter-rouge">From<T> for U</code> implies <code class="highlighter-rouge">Into<U> for T</code>, it’s usually preferable to implement only the former and get the other for free,</li>
<li>We can make our methods more general by using those traits as type constraints.</li>
</ul>
<p>Now that you know about these traits, go ahead and use them in your crates. Your API will be more ergonomic and idiomatic, and its users will appreciate the convenience.</p>
<p>I hope you found this article useful! What should we explore next? Tell me on Twitter (<a href="https://twitter.com/meqif">@meqif</a>) or send me an email (<a href="mailto:words@ricardomartins.cc">words@ricardomartins.cc</a>). You can also discuss the article on <a href="https://www.reddit.com/r/rust/comments/4w14yd/convenient_and_idiomatic_conversions_in_rust/">reddit</a>. If you don’t want to miss the next articles, sign up for my newsletter in the form below. 👇</p>
http://ricardomartins.cc/2016/07/11/interior-mutability-behind-the-curtainInterior mutability in Rust, part 3: behind the curtain2016-07-11T00:00:00-05:002016-07-11T00:00:00-05:00Ricardo Martinshttp://ricardomartins.cc/<div class="summary">
<h1 id="key-takeaways">Key takeaways</h1>
<ul>
<li><code class="highlighter-rouge">UnsafeCell</code> is the keystone for building interior mutability types (<code class="highlighter-rouge">Cell</code>, <code class="highlighter-rouge">RefCell</code>, <code class="highlighter-rouge">RwLock</code> and <code class="highlighter-rouge">Mutex</code>)</li>
<li><code class="highlighter-rouge">UnsafeCell</code> wraps around a value and provides a raw mutable pointer to it</li>
<li>It depends on a special compiler path that avoids undefined behavior related to the raw pointer</li>
<li>It lacks synchronization primitives and has a very bare-bones API</li>
<li><code class="highlighter-rouge">Cell</code> and <code class="highlighter-rouge">RefCell</code> provide convenient APIs to access their <code class="highlighter-rouge">UnsafeCell</code>’s inner value, with negligible-to-none run-time overhead</li>
<li><code class="highlighter-rouge">RwLock</code> and <code class="highlighter-rouge">Mutex</code> provide synchronized access to <code class="highlighter-rouge">UnsafeCell</code>’s inner value</li>
<li>Avoid using <code class="highlighter-rouge">UnsafeCell</code> except when you’re absolutely sure you don’t want any of the other types</li>
</ul>
</div>
<p><em>This article is part of a series about interior mutability in Rust. You can read <a href="/2016/06/08/interior-mutability">part 1 here</a> and <a href="/2016/06/25/interior-mutability-thread-safety">part 2 here</a></em>.</p>
<h1 id="introduction">Introduction</h1>
<p>In the previous articles, we looked at interior mutability in Rust from a practical standpoint, but we didn’t explore the magic that made it work. In this article we’ll pull back the curtain and <a href="https://en.wikipedia.org/wiki/The_Wizard_of_Oz_(1939_film)">stare at the man lurking behind</a>.</p>
<p>A good place to start is reading the definition of <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L155-L161"><code class="highlighter-rouge">Cell</code></a>, <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L318-L325"><code class="highlighter-rouge">RefCell</code></a>, <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libstd/sync/rwlock.rs#L68-L73"><code class="highlighter-rouge">RwLock</code></a>, and <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libstd/sync/mutex.rs#L115-L125"><code class="highlighter-rouge">Mutex</code></a> to gather some clues. Luckily, a pattern quickly emerges: all of them contain a field with a common type: <code class="highlighter-rouge">UnsafeCell</code>.<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup> We’ll first explore what it is, then figure out how it is used by those types.</p>
<h1 id="what-is-unsafecell">What is <code class="highlighter-rouge">UnsafeCell</code>?</h1>
<p><a href="https://doc.rust-lang.org/std/cell/struct.UnsafeCell.html"><code class="highlighter-rouge">UnsafeCell</code></a> is defined in the documentation as “the core primitive for interior mutability in Rust”. Indeed, we stumbled on it every time we looked at the definition of types that provide interior mutability. However, for all its apparent power, its definition seems rather boring:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="p">[</span><span class="n">lang</span> <span class="o">=</span> <span class="s">"unsafe_cell"</span><span class="p">]</span>
<span class="nd">#[stable(feature</span> <span class="nd">=</span> <span class="s">"rust1"</span><span class="nd">,</span> <span class="nd">since</span> <span class="nd">=</span> <span class="s">"1.0.0"</span><span class="nd">)]</span>
<span class="k">pub</span> <span class="k">struct</span> <span class="n">UnsafeCell</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="p">{</span>
<span class="n">value</span><span class="p">:</span> <span class="n">T</span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div></div>
<p>That’s it? The keystone for interior mutability looks like a <a href="https://doc.rust-lang.org/book/structs.html#tuple-structs">newtype struct</a> with some weird stuff before?</p>
<p>Those strange lines right above the structure definition are compiler attributes. We can mostly ignore the second, as it simply marks the function as stable since version 1.0.0 of the compiler, meaning it can be used in any channel (stable, beta, and nightly). The first one is more interesting: it’s a <a href="https://doc.rust-lang.org/book/lang-items.html">lang item</a>, a sort of special wink and nudge to the compiler. I’ll get to it later.</p>
<p>Apart from <code class="highlighter-rouge">new()</code> and <code class="highlighter-rouge">into_inner()</code>, which are humdrum methods, there’s also a <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L954-L956"><code class="highlighter-rouge">get()</code></a> method:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">fn</span> <span class="nf">get</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="o">*</span><span class="k">mut</span> <span class="n">T</span> <span class="p">{</span>
<span class="o">&</span><span class="k">self</span><span class="py">.value</span> <span class="k">as</span> <span class="o">*</span><span class="k">const</span> <span class="n">T</span> <span class="k">as</span> <span class="o">*</span><span class="k">mut</span> <span class="n">T</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Even though it’s a small method, we can tell it’s somewhat unusual: it takes an immutable reference to the inner value (<code class="highlighter-rouge">&self.value</code>) and then <a href="https://doc.rust-lang.org/book/casting-between-types.html#coercion">casts</a> it twice: first into a raw constant pointer (<code class="highlighter-rouge">*const T</code>), and then into a raw mutable pointer (<code class="highlighter-rouge">*mut T</code>), which is returned to the caller.</p>
<p>Raw pointers, eh? That’s new. Let’s take a look.</p>
<h2 id="raw-pointers">Raw pointers</h2>
<p><a href="https://doc.rust-lang.org/book/raw-pointers.html">Raw pointers</a> are similar to references, which we are used to. Both point to a memory address, but there are important differences. References are smart: they have safety guarantees (such as pointing to valid memory and never being null), borrow checking and lifetimes. Raw pointers, on the other hand, are… well, raw. They have none of those features, and simply point to a memory address, like pointers in C. Most importantly, they don’t have aliasing or mutability guarantees, unlike references. There are two kinds of raw pointers: constant (<code class="highlighter-rouge">*const T</code>) and mutable (<code class="highlighter-rouge">*mut T</code>). The difference between them is that mutation is not directly allowed on constant pointers, but we can easily cast them into mutable pointers and then mutate the value.</p>
<p>You might have noticed that <code class="highlighter-rouge">get()</code> isn’t marked <code class="highlighter-rouge">unsafe</code>, even though we are coercing an immutable reference into a raw mutable pointer and dropping Rust’s safety guarantees along the way. Shouldn’t this trip up some alarms?</p>
<p>Coercing (i.e., converting) an immutable reference into a raw constant pointer isn’t too bad. It could be argued that it’s safe, since the immutability of the reference is preserved and we’re starting from a reference, which has the safety guarantees we saw above, so the resulting raw pointer won’t be invalid or null. Going from a raw constant pointer to a raw <em>mutable</em> pointer, on the other hand, seems much worse, because we’re suddenly saying it’s fine to mutate a value we have no business mutating. Going from there back to a <em>mutable</em> reference is <a href="https://doc.rust-lang.org/nomicon/transmutes.html">definitively bad</a>.</p>
<p>Thinking some more about it, those conversions aren’t actually dangerous by themselves, as they have no side-effects on their own. It’s actually using (i.e., dereferencing) the raw pointers that’s potentially dangerous. As such, when we try to dereference the pointer returned by <code class="highlighter-rouge">get()</code> like in <a href="https://play.rust-lang.org/?gist=e7ba86944009a4564fb74aa931c0eb79">this example</a>, the compiler tells us that we’re about to do something potentially dangerous:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>error: dereference of raw pointer requires unsafe function or block [E0133]
</code></pre></div></div>
<p>Surrounding the dereference with an unsafe block signals to the compiler that we’re taking responsibility for all potential badness that may come. In the example, we know the pointer is safe because it was created from a reference and, most importantly, we didn’t mutate its value, so it’ll work <a href="https://play.rust-lang.org/?gist=0ce6d15aba4fe5b21bd6c667a853791a">as expected</a>.</p>
<p>Even so, mutating non-mutable data is considered <a href="https://doc.rust-lang.org/reference.html#behavior-considered-undefined">undefined behavior</a>:</p>
<blockquote>
<p>Mutating non-mutable data (that is, data reached through a shared reference or data owned by a let binding), unless that data is contained within an UnsafeCell<U>.</p>
</blockquote>
<p>So, we’re back at the beginning: what makes <code class="highlighter-rouge">UnsafeCell</code> special?</p>
<h2 id="-a-dash-of-magic-">✨ A dash of magic ✨</h2>
<p>Remember the lang item that preceded <code class="highlighter-rouge">UnsafeCell</code>’s definition? Here it is again:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="p">[</span><span class="n">lang</span> <span class="o">=</span> <span class="s">"unsafe_cell"</span><span class="p">]</span>
</code></pre></div></div>
<p>It triggers special treatment by the compiler: <code class="highlighter-rouge">UnsafeCell</code> gets a special marker that follows it during the various compilation phases, until it finally triggers a <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/librustc_trans/abi.rs#L369-L382">particular code path</a>. This section of code checks for the presence of that marker and, if it is present, avoids setting two <a href="http://llvm.org">LLVM</a> attributes that get applied to regular types: namely, that the value is read-only and there are no aliases. Those attributes, when present, may allow some additional optimization by LLVM, which can interfere with the behavior we want for <code class="highlighter-rouge">UnsafeCell</code>.</p>
<p>Now, that doesn’t mean we absolutely can’t get similar results without the lang item — we can<sup id="fnref:2"><a href="#fn:2" class="footnote">2</a></sup>, but that’s purely coincidental and relies on undefined behavior which may eat our laundry, depending on the compiler’s mood (i.e., the generated code may change unexpectedly).</p>
<p>That aside, notice that there is no synchronization inside <code class="highlighter-rouge">get()</code>, which makes <code class="highlighter-rouge">UnsafeCell</code> unsafe to be used by multiple threads and is thus marked <code class="highlighter-rouge">!Sync</code>.</p>
<p>A <a href="https://play.rust-lang.org/?gist=1c993b661b3194f5554f8751e946894e&version=nightly&backtrace=0">little experiment</a> shows that using an <code class="highlighter-rouge">UnsafeCell</code> incurs no run-time costs after compiler optimization, and is nearly indistinguishable from a “raw” variable, as expected from Rust’s zero-overhead abstractions.</p>
<p>And that’s it for <code class="highlighter-rouge">UnsafeCell</code>. It’s a little amazing, but this tiny structure with a very simple method and some compiler support is enough to build upon and create all the other interior mutability types we explored before. A dash of magic is enough. ✨</p>
<h1 id="how-is-it-used">How is it used?</h1>
<p>Now that we know what <code class="highlighter-rouge">UnsafeCell</code> is like, let’s explore how the other types build upon it.</p>
<h2 id="cell"><code class="highlighter-rouge">Cell</code></h2>
<p>Let’s start with <code class="highlighter-rouge">Cell</code>, the simplest of the bunch. <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L155-L161">Its definition</a> is exceedingly succinct:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">struct</span> <span class="n">Cell</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="n">value</span><span class="p">:</span> <span class="n">UnsafeCell</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This suggests that <code class="highlighter-rouge">Cell</code> merely provides a nicer API to access the inner value contained in <code class="highlighter-rouge">UnsafeCell</code>. If we take a gander at <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L194-L196"><code class="highlighter-rouge">Cell::get</code></a> and <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L211-L215"><code class="highlighter-rouge">Cell::set</code></a>, we can see that <code class="highlighter-rouge">UnsafeCell::get()</code> is enough to implement both methods, and that the implementation details, such as the unsafe blocks, are hidden from the user:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">fn</span> <span class="nf">get</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">T</span> <span class="p">{</span>
<span class="k">unsafe</span><span class="p">{</span> <span class="o">*</span><span class="k">self</span><span class="py">.value</span><span class="nf">.get</span><span class="p">()</span> <span class="p">}</span>
<span class="p">}</span>
<span class="k">pub</span> <span class="k">fn</span> <span class="nf">set</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">,</span> <span class="n">value</span><span class="p">:</span> <span class="n">T</span><span class="p">)</span> <span class="p">{</span>
<span class="k">unsafe</span> <span class="p">{</span>
<span class="o">*</span><span class="k">self</span><span class="py">.value</span><span class="nf">.get</span><span class="p">()</span> <span class="o">=</span> <span class="n">value</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>There’s an important detail in <code class="highlighter-rouge">Cell</code>’s <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L163">impl declaration</a>: it’s restricted to <code class="highlighter-rouge">Copy</code> types.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="p">:</span><span class="nb">Copy</span><span class="o">></span> <span class="n">Cell</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
</code></pre></div></div>
<p>Without this restriction, we would be able to create instances of <code class="highlighter-rouge">Cell</code> around <code class="highlighter-rouge">!Copy</code> types, such as <code class="highlighter-rouge">&mut T</code> (mutable references), which is a <a href="https://doc.rust-lang.org/std/marker/trait.Copy.html#when-can-my-type-not-be-copy">terrible idea</a>: doing so would create aliased mutable references and break the aliasing rules.</p>
<h2 id="refcell"><code class="highlighter-rouge">RefCell</code></h2>
<p><code class="highlighter-rouge">RefCell</code> is only slightly more complicated than <code class="highlighter-rouge">Cell</code>. It contains an <code class="highlighter-rouge">UnsafeCell</code> field with the inner value, just like <code class="highlighter-rouge">Cell</code>, and a <code class="highlighter-rouge">Cell</code> with a borrow flag to track the borrow state:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">struct</span> <span class="n">RefCell</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="p">{</span>
<span class="n">borrow</span><span class="p">:</span> <span class="n">Cell</span><span class="o"><</span><span class="n">BorrowFlag</span><span class="o">></span><span class="p">,</span>
<span class="n">value</span><span class="p">:</span> <span class="n">UnsafeCell</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div></div>
<p>We saw in the <a href="/2016/06/08/interior-mutability">first article</a> of the series that we need to call <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L438-L446"><code class="highlighter-rouge">borrow</code></a> or <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L486-L494"><code class="highlighter-rouge">borrow_mut</code></a> on <code class="highlighter-rouge">RefCell</code> before accessing the value inside. Both methods have similar implementations and convert the raw pointer they get from <code class="highlighter-rouge">UnsafeCell::get()</code> to the value back into a reference with <a href="https://doc.rust-lang.org/book/raw-pointers.html#references-and-raw-pointers"><code class="highlighter-rouge">&*</code></a>. For brevity’s sake, we’ll look at just <code class="highlighter-rouge">RefCell::borrow()</code>, since <code class="highlighter-rouge">RefCell::borrow_mut()</code> is very similar.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">fn</span> <span class="nf">borrow</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Ref</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">match</span> <span class="nn">BorrowRef</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="py">.borrow</span><span class="p">)</span> <span class="p">{</span>
<span class="nf">Some</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="k">=></span> <span class="n">Ref</span> <span class="p">{</span>
<span class="n">value</span><span class="p">:</span> <span class="k">unsafe</span> <span class="p">{</span> <span class="o">&*</span><span class="k">self</span><span class="py">.value</span><span class="nf">.get</span><span class="p">()</span> <span class="p">},</span>
<span class="n">borrow</span><span class="p">:</span> <span class="n">b</span><span class="p">,</span>
<span class="p">},</span>
<span class="nb">None</span> <span class="k">=></span> <span class="nd">panic!</span><span class="p">(</span><span class="s">"RefCell<T> already mutably borrowed"</span><span class="p">),</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>There’s the call to <code class="highlighter-rouge">UnsafeCell::get()</code> and conversion back into a reference. Where’s the update to <code class="highlighter-rouge">RefCell</code>’s borrow state, though?</p>
<p>Reading <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L593-L604"><code class="highlighter-rouge">BorrowRef::new()</code></a> makes things clear. Roughly, when that method is called, it checks <code class="highlighter-rouge">RefCell</code>’s <code class="highlighter-rouge">borrow</code> field (which it received as argument) and updates it if the value can be borrowed. Since the borrow field is a <code class="highlighter-rouge">Cell</code>, <code class="highlighter-rouge">BorrowRef</code> can mutate it, even though it received an immutable reference to it!</p>
<p><a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L486-L494"><code class="highlighter-rouge">RefCell::borrow_mut()</code></a> is very similar, except that it calls <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libcore/cell.rs#L824-L835"><code class="highlighter-rouge">BorrowRefMut::new()</code></a> instead of <code class="highlighter-rouge">BorrowRef::new()</code>, and returns a mutable reference (to be precise, it returns a <code class="highlighter-rouge">RefMut</code>, which implements <code class="highlighter-rouge">DerefMut</code>).</p>
<p>We can conclude, then, that <code class="highlighter-rouge">Cell</code> and <code class="highlighter-rouge">RefCell</code> are lightweight wrappers around <code class="highlighter-rouge">UnsafeCell</code>, giving us a convenient API to access <code class="highlighter-rouge">UnsafeCell</code>’s inner value and shielding us from dangerous pointer dereferences. In the case of <code class="highlighter-rouge">Cell</code>, there’s no run-time cost after optimization, while <code class="highlighter-rouge">RefCell</code>’s dynamic borrow checking mechanism incurs a small overhead. Additionally, <code class="highlighter-rouge">Cell</code> and <code class="highlighter-rouge">RefCell</code> are “tainted” by <code class="highlighter-rouge">UnsafeCell</code>’s <code class="highlighter-rouge">!Sync</code> marker.</p>
<h2 id="rwlock-and-mutex"><code class="highlighter-rouge">RwLock</code> and <code class="highlighter-rouge">Mutex</code></h2>
<p>In contrast to <code class="highlighter-rouge">Cell</code> and <code class="highlighter-rouge">RefCell</code>, <code class="highlighter-rouge">RwLock</code> and <code class="highlighter-rouge">Mutex</code> are more complex data structures which provide synchronized access to the inner value, allowing them to implement <code class="highlighter-rouge">Sync</code> and honor that guarantee. We won’t be diving deep into them, but looking at <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libstd/sync/rwlock.rs#L195-L200"><code class="highlighter-rouge">RwLock::read()</code></a> we can see similarities to <code class="highlighter-rouge">RefCell::borrow()</code>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">fn</span> <span class="nf">read</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">LockResult</span><span class="o"><</span><span class="n">RwLockReadGuard</span><span class="o"><</span><span class="n">T</span><span class="o">>></span> <span class="p">{</span>
<span class="k">unsafe</span> <span class="p">{</span>
<span class="k">self</span><span class="py">.inner.lock</span><span class="nf">.read</span><span class="p">();</span>
<span class="nn">RwLockReadGuard</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="o">&*</span><span class="k">self</span><span class="py">.inner</span><span class="p">,</span> <span class="o">&</span><span class="k">self</span><span class="py">.data</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Untangling that a little, we can see that first an inner lock<sup id="fnref:3"><a href="#fn:3" class="footnote">3</a></sup> is locked, then a <code class="highlighter-rouge">ReadLockReadGuard</code> is created. Similarly to what <code class="highlighter-rouge">BorrowRef</code> does for <code class="highlighter-rouge">RefCell</code>, <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libstd/sync/rwlock.rs#L461-L470"><code class="highlighter-rouge">RwLockReadGuard::new()</code></a> converts the raw pointer it got after calling <code class="highlighter-rouge">UnsafeCell::get()</code> into a reference:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">unsafe</span> <span class="k">fn</span> <span class="nf">new</span><span class="p">(</span><span class="n">lock</span><span class="p">:</span> <span class="o">&</span><span class="nv">'rwlock</span> <span class="n">StaticRwLock</span><span class="p">,</span> <span class="n">data</span><span class="p">:</span> <span class="o">&</span><span class="nv">'rwlock</span> <span class="n">UnsafeCell</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">)</span>
<span class="k">-></span> <span class="n">LockResult</span><span class="o"><</span><span class="n">RwLockReadGuard</span><span class="o"><</span><span class="nv">'rwlock</span><span class="p">,</span> <span class="n">T</span><span class="o">>></span> <span class="p">{</span>
<span class="nn">poison</span><span class="p">::</span><span class="nf">map_result</span><span class="p">(</span><span class="n">lock</span><span class="py">.poison</span><span class="nf">.borrow</span><span class="p">(),</span> <span class="p">|</span><span class="n">_</span><span class="p">|</span> <span class="p">{</span>
<span class="n">RwLockReadGuard</span> <span class="p">{</span>
<span class="n">__lock</span><span class="p">:</span> <span class="n">lock</span><span class="p">,</span>
<span class="n">__data</span><span class="p">:</span> <span class="o">&*</span><span class="n">data</span><span class="nf">.get</span><span class="p">(),</span>
<span class="p">}</span>
<span class="p">})</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Both <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libstd/sync/rwlock.rs#L247-L252"><code class="highlighter-rouge">RwLock::write()</code></a> and <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libstd/sync/mutex.rs#L237-L242"><code class="highlighter-rouge">Mutex::lock()</code></a> follow a similar pattern to <code class="highlighter-rouge">RwLock::read()</code>, so there’s no need to explore them individually.</p>
<h1 id="wrapping-up">Wrapping up</h1>
<p>In this article we learned that far from relying on arcane magic, interior mutability is achieved thanks to <code class="highlighter-rouge">UnsafeCell</code>, a neat little structure that allows the creation of more ergonomic abstractions on top of it.</p>
<p>Since its API involves unsafe operations, using it directly is a little cumbersome. In nearly every case where we need interior mutability, we are better served by <code class="highlighter-rouge">Cell</code>, <code class="highlighter-rouge">RefCell</code>, <code class="highlighter-rouge">RwLock</code>, and <code class="highlighter-rouge">Mutex</code> instead. There are a few cases where you might want to avoid the overhead of those types by <code class="highlighter-rouge">UnsafeCell</code> directly, such as implementing new locks and concurrent data structures, but those aren’t common tasks unless you work in academia. 😉</p>
<p>Whew! This concludes the series on interior mutability. I hope you enjoyed it! Special thanks to everyone who has commented on both this and the previous articles. Your support and corrections are deeply appreciated. 💜</p>
<p>What should we explore next? Tell me on Twitter (<a href="https://twitter.com/meqif">@meqif</a>) or send me an email (<a href="mailto:words@ricardomartins.cc">words@ricardomartins.cc</a>). You can also discuss the article on <a href="https://www.reddit.com/r/rust/comments/4sbax3/interior_mutability_in_rust_part_3_behind_the/">reddit</a>. If you don’t want to miss the next articles, sign up for my newsletter in the form below. 👇</p>
<div class="footnotes">
<ol>
<li id="fn:1">
<p>Many other types depend on <code class="highlighter-rouge">UnsafeCell</code>, such as <code class="highlighter-rouge">Condvar</code>, the <code class="highlighter-rouge">Sender</code> and <code class="highlighter-rouge">Receiver</code> structures used created by <code class="highlighter-rouge">std::sync::mpsc::channel</code>, thread-local variables (created with the <code class="highlighter-rouge">thread_local</code> macro), private data structures used in the compiler implementation, and some others I won’t bother to list. <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
<li id="fn:2">
<p>Compare the result when running <a href="https://play.rust-lang.org/?gist=dce59aa6a210a9c5010b921b1579559b&version=stable&backtrace=0">this example</a> in debug mode and release mode. In release mode the compiler assumed that the value is never changed, so the call to <code class="highlighter-rouge">println!</code> doesn’t show the update! Thanks to <a href="https://www.reddit.com/r/rust/comments/4sbax3/interior_mutability_in_rust_part_3_behind_the/d5864zn">/u/derKha</a> and <a href="https://www.reddit.com/r/rust/comments/4sbax3/interior_mutability_in_rust_part_3_behind_the/d588nt5">/u/notriddle</a> for their examples. <a href="#fnref:2" class="reversefootnote">↩</a></p>
</li>
<li id="fn:3">
<p>That inner lock is actually a native OS lock. You can read the implementation for the <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libstd/sys/unix/rwlock.rs">Unix one</a> and the <a href="https://github.com/rust-lang/rust/blob/1.10.0/src/libstd/sys/windows/rwlock.rs">Windows one</a>. Both rely on <code class="highlighter-rouge">UnsafeCell</code> as well. <a href="#fnref:3" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
http://ricardomartins.cc/2016/06/25/interior-mutability-thread-safetyInterior mutability in Rust, part 2: thread safety2016-06-25T00:00:00-05:002016-06-25T00:00:00-05:00Ricardo Martinshttp://ricardomartins.cc/<div class="summary">
<h1 id="key-takeaways">Key takeaways</h1>
<ul>
<li>We can have thread-safe interior mutability through <code class="highlighter-rouge">Mutex</code>, <code class="highlighter-rouge">RwLock</code> and the various <code class="highlighter-rouge">Atomic*</code> types in <code class="highlighter-rouge">std::sync</code></li>
<li><code class="highlighter-rouge">Mutex</code> allows only one thread at a time, has no direct thread-unsafe counterpart, but can be thought of as only giving <code class="highlighter-rouge">&mut T</code> references</li>
<li><code class="highlighter-rouge">RwLock</code> is equivalent to <code class="highlighter-rouge">RefCell</code>, and also allows multiple readers or one writer</li>
<li>The <code class="highlighter-rouge">Atomic</code> types are equivalent to <code class="highlighter-rouge">Cell</code></li>
<li><code class="highlighter-rouge">std::sync::Arc</code> is the thread-safe version of <code class="highlighter-rouge">std::rc::Rc</code>, necessary to share references among threads</li>
<li>These thread-safe types have additional run-time cost compared to their unsynchronized counterparts</li>
<li>In Rust, we lock/protect data, not code</li>
</ul>
</div>
<p><em>This article is part of a series about interior mutability in Rust. You can read <a href="/2016/06/08/interior-mutability">part 1 here</a> and <a href="/2016/07/11/interior-mutability-behind-the-curtain">part 3 here</a></em>.</p>
<h1 id="introduction">Introduction</h1>
<p>In the <a href="/2016/06/08/interior-mutability">previous article</a>, we looked into <a href="https://doc.rust-lang.org/std/cell/struct.Cell.html"><code class="highlighter-rouge">Cell</code></a> and <a href="https://doc.rust-lang.org/std/cell/struct.RefCell.html"><code class="highlighter-rouge">RefCell</code></a> as a way to achieve interior mutability—the ability to mutate particular fields in a structure regardless of its exterior (explicit or declared) mutability.</p>
<p>However, neither <code class="highlighter-rouge">Cell</code> nor <code class="highlighter-rouge">RefCell</code> are appropriate to share data among threads. Since <code class="highlighter-rouge">Cell</code> has no borrow checking mechanisms and its operations aren’t <a href="https://en.wikipedia.org/wiki/Linearizability">atomic</a>, it’s possible to have <a href="https://en.wikipedia.org/wiki/Race_condition#Software">race conditions</a> where we read outdated values and lose updates. <code class="highlighter-rouge">RefCell</code>, on the other hand, has run-time borrow checking, but will panic when there are conflicting borrows, such as borrowing a mutably borrowed value. Additionally, <code class="highlighter-rouge">RefCell</code> uses a <code class="highlighter-rouge">Cell</code> to keep track of the borrow state. This means that even if you were careful and checked it before borrowing, you would risk mutably borrowing the value in multiple threads simultaneously, because its run-time borrow checks aren’t atomic either.</p>
<p>Because of those issues, both <code class="highlighter-rouge">Cell</code> and <code class="highlighter-rouge">RefCell</code> are marked <a href="https://doc.rust-lang.org/std/marker/trait.Sync.html"><code class="highlighter-rouge">!Sync</code></a>, which means they’re unsafe to be used in more than one thread.<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup></p>
<p>Additionally, we need to share references to the cell among threads, but the reference counter type we explored previously, <code class="highlighter-rouge">Rc</code>, is also not appropriate to be used in this scenario. The reference counter fields inside <code class="highlighter-rouge">Rc</code> are wrapped by <code class="highlighter-rouge">Cell</code>, so they would get erroneous values in a threaded program sooner or later. The presence of <code class="highlighter-rouge">!Sync</code> fields is “infectious”. Since <code class="highlighter-rouge">Rc</code> contains two <code class="highlighter-rouge">Cell</code> fields, their <code class="highlighter-rouge">!Sync</code> markers propagate to the entire <code class="highlighter-rouge">Rc</code> structure. <code class="highlighter-rouge">Rc</code> is also marked <a href="https://doc.rust-lang.org/std/marker/trait.Send.html"><code class="highlighter-rouge">!Send</code></a> — unsafe to send (move) to other threads. Likewise, <code class="highlighter-rouge">!Send</code> is just as “infectious” as <code class="highlighter-rouge">!Sync</code>.</p>
<p>We could implement both traits ourselves in our types, but since those traits are rather important, they’re marked as unsafe. So, we need to tell the Rust compiler we know what we’re doing by prefixing their <code class="highlighter-rouge">impl</code> declarations with the <code class="highlighter-rouge">unsafe</code> keyword. For instance, if we wanted to make the nodes in the <a href="https://play.rust-lang.org/?gist=9ccf40fae2347519fcae7dd42ddf5ed6">example</a> of the previous article <code class="highlighter-rouge">Send</code> and <code class="highlighter-rouge">Sync</code>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">unsafe</span> <span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="nb">Send</span> <span class="k">for</span> <span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{}</span>
<span class="k">unsafe</span> <span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">Sync</span> <span class="k">for</span> <span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{}</span>
</code></pre></div></div>
<p>This does make the compiler shut up and allows us to get on with it, but what does it mean?</p>
<p><code class="highlighter-rouge">Send</code> and <code class="highlighter-rouge">Sync</code> are automatically derived by the compiler for most types. If a type contains a <code class="highlighter-rouge">!Send</code> or <code class="highlighter-rouge">!Sync</code> field, the “taint” always spreads to the parent types. By explicitly implementing those traits, we’re implicitly telling the user of our API that our types are thread-safe (<code class="highlighter-rouge">Sync</code>) and that they can be moved between threads safely (<code class="highlighter-rouge">Send</code>).</p>
<p>When we say our types are <code class="highlighter-rouge">Send</code> and <code class="highlighter-rouge">Sync</code> without actually introducing synchronization mechanisms, we’re disrespecting the “contracts” associated with those traits and misleading the user. They’d get run-time panics or bad results due to race conditions. Not a happy situation, at all.</p>
<h1 id="thread-safe-interior-mutability">Thread-safe interior mutability</h1>
<p>Fortunately, Rust gives us good tools to achieve interior mutability in a thread-safe way without much effort. More than that, it does so in such a way that the borrow checker will have our backs most of the time, so we can’t shoot ourselves in the feet.</p>
<p>One of the really nice aspects about Rust is that once you get the hang of the borrow system, you can use the same reasoning with interior mutability in a single thread (<code class="highlighter-rouge">Cell</code> and <code class="highlighter-rouge">RefCell</code>) and in concurrent programs.</p>
<h2 id="for-copy-values">For <code class="highlighter-rouge">Copy</code> values</h2>
<p>For <code class="highlighter-rouge">Copy</code> values (eg., integers), instead of <code class="highlighter-rouge">Cell</code>, we have <a href="https://doc.rust-lang.org/std/sync/atomic/index.html"><code class="highlighter-rouge">Atomic</code></a> types (<code class="highlighter-rouge">std::sync::atomic::*</code>) that rely on assembly instructions to prevent data races:</p>
<ul>
<li><a href="https://doc.rust-lang.org/std/sync/atomic/struct.AtomicBool.html"><code class="highlighter-rouge">AtomicBool</code></a>, a boolean type,</li>
<li><a href="https://doc.rust-lang.org/std/sync/atomic/struct.AtomicIsize.html"><code class="highlighter-rouge">AtomicIsize</code></a>, a signed integer type,</li>
<li><a href="https://doc.rust-lang.org/std/sync/atomic/struct.AtomicUsize.html"><code class="highlighter-rouge">AtomicUsize</code></a>, an unsigned integer type, and</li>
<li><a href="https://doc.rust-lang.org/std/sync/atomic/struct.AtomicPtr.html"><code class="highlighter-rouge">AtomicPtr</code></a>, a raw pointer type.</li>
</ul>
<p>Even though there are only four types, you can can build on <code class="highlighter-rouge">AtomicPtr</code> to implement additional ones. Alternatively, you can use crates, such as the <a href="https://crates.io/crates/atom/"><code class="highlighter-rouge">atom</code> crate</a>, to do so with a simple API. Quick disclaimer: I haven’t tried <code class="highlighter-rouge">atom</code> but the example in <a href="https://github.com/slide-rs/atom/blob/master/readme.md">its README</a> looks nice, and a quick glance at the source code looks exactly like what I would expect.</p>
<p>Using an atomic type is a bit more involved than doing the same with a <code class="highlighter-rouge">Cell</code>. Taking again the naive reference counter example of the previous article, using an <code class="highlighter-rouge">AtomicUsize</code> we’d have:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">sync</span><span class="p">::</span><span class="nn">atomic</span><span class="p">::{</span><span class="n">AtomicUsize</span><span class="p">,</span> <span class="n">Ordering</span><span class="p">};</span>
<span class="k">struct</span> <span class="n">NaiveRc</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="n">reference_count</span><span class="p">:</span> <span class="n">AtomicUsize</span><span class="p">,</span>
<span class="n">inner_value</span><span class="p">:</span> <span class="n">T</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">Clone</span> <span class="k">for</span> <span class="n">NaiveRc</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">clone</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="k">self</span><span class="py">.reference_count</span><span class="nf">.fetch_add</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nn">Ordering</span><span class="p">::</span><span class="n">Relaxed</span><span class="p">);</span>
<span class="c">// ...</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>As you can see, instead of simply assigning a new value to <code class="highlighter-rouge">reference_count</code>, we called <code class="highlighter-rouge">fetch_add</code> to atomically increment it. The first parameter is the increment size, while the second is new. An <a href="https://doc.rust-lang.org/std/sync/struct.RwLock.html"><code class="highlighter-rouge">Ordering</code></a> tells the compiler (and the CPU) how much freedom it has to reorder instructions. I won’t delve into that, as the official documentation explains it in sufficient detail.</p>
<h2 id="for-non-copy-values">For non-Copy values</h2>
<p>For non-copy values, <a href="https://doc.rust-lang.org/std/sync/struct.RwLock.html"><code class="highlighter-rouge">std::sync::RwLock</code></a> is the counterpart to <code class="highlighter-rouge">RefCell</code>. Like <code class="highlighter-rouge">RefCell</code>, <code class="highlighter-rouge">RwLock</code> has semantics very similar to our old friend the borrow system, and allows either several “readers” or one “writer”, but not both at the same time, nor several “writers”.</p>
<p>Unlike <code class="highlighter-rouge">RefCell</code>, however, <code class="highlighter-rouge">RwLock</code> doesn’t panic when there are incompatible borrows: if a thread needs a mutable reference, it will just have to wait until the other threads release the lock (i.e., stop using the borrowed values).</p>
<p>We can get shared, read-only references with <code class="highlighter-rouge">read</code> (equivalent to <code class="highlighter-rouge">borrow</code> in <code class="highlighter-rouge">RefCell</code>), or exclusive, mutable references with <code class="highlighter-rouge">write</code> (<code class="highlighter-rouge">borrow_mut</code>, respectively).<sup id="fnref:2"><a href="#fn:2" class="footnote">2</a></sup></p>
<p>Converting the <a href="https://play.rust-lang.org/?gist=9ccf40fae2347519fcae7dd42ddf5ed6">graph example</a> in the previous article to use <code class="highlighter-rouge">RwLock</code> instead of <code class="highlighter-rouge">RefCell</code> is straightforward: replace <code class="highlighter-rouge">RefCell</code> declarations with <code class="highlighter-rouge">RwLock</code>, change <code class="highlighter-rouge">borrow</code> to <code class="highlighter-rouge">read</code> and <code class="highlighter-rouge">borrow_mut</code> to <code class="highlighter-rouge">write</code>. We also need to replace <code class="highlighter-rouge">Rc</code> with <code class="highlighter-rouge">Arc</code> to be able to move references to other threads, but I’ll describe that later.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="n">thread</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">sync</span><span class="p">::{</span><span class="nb">Arc</span><span class="p">,</span> <span class="n">RwLock</span><span class="p">};</span>
<span class="c">// Represents a reference to a node.</span>
<span class="c">// This makes the code less repetitive to write and easier to read.</span>
<span class="k">type</span> <span class="n">NodeRef</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="o">=</span> <span class="nb">Arc</span><span class="o"><</span><span class="n">RwLock</span><span class="o"><</span><span class="n">_Node</span><span class="o"><</span><span class="n">T</span><span class="o">>>></span><span class="p">;</span>
<span class="c">// The private representation of a node.</span>
<span class="k">struct</span> <span class="n">_Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="n">inner_value</span><span class="p">:</span> <span class="n">T</span><span class="p">,</span>
<span class="n">adjacent</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="n">NodeRef</span><span class="o"><</span><span class="n">T</span><span class="o">>></span><span class="p">,</span>
<span class="p">}</span>
<span class="c">// The public representation of a node, with some syntactic sugar.</span>
<span class="k">struct</span> <span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">(</span><span class="n">NodeRef</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">);</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="c">// Creates a new node with no edges.</span>
<span class="k">fn</span> <span class="nf">new</span><span class="p">(</span><span class="n">inner</span><span class="p">:</span> <span class="n">T</span><span class="p">)</span> <span class="k">-></span> <span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">let</span> <span class="n">node</span> <span class="o">=</span> <span class="n">_Node</span> <span class="p">{</span> <span class="n">inner_value</span><span class="p">:</span> <span class="n">inner</span><span class="p">,</span> <span class="n">adjacent</span><span class="p">:</span> <span class="nd">vec!</span><span class="p">[]</span> <span class="p">};</span>
<span class="nf">Node</span><span class="p">(</span><span class="nn">Arc</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="nn">RwLock</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="n">node</span><span class="p">)))</span>
<span class="p">}</span>
<span class="c">// Adds a directed edge from this node to other node.</span>
<span class="k">fn</span> <span class="nf">add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">,</span> <span class="n">other</span><span class="p">:</span> <span class="o">&</span><span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">)</span> <span class="p">{</span>
<span class="k">self</span><span class="err">.</span><span class="mi">0</span>
<span class="nf">.write</span><span class="p">()</span>
<span class="nf">.expect</span><span class="p">(</span><span class="s">"Failed to acquire a write lock on node"</span><span class="p">)</span>
<span class="py">.adjacent</span><span class="nf">.push</span><span class="p">(</span><span class="n">other</span><span class="err">.</span><span class="mi">0</span><span class="nf">.clone</span><span class="p">());</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">struct</span> <span class="n">Graph</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="n">nodes</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">>></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">Graph</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">with_nodes</span><span class="p">(</span><span class="n">nodes</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">>></span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="n">Graph</span> <span class="p">{</span> <span class="n">nodes</span><span class="p">:</span> <span class="n">nodes</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="c">// Create some nodes</span>
<span class="k">let</span> <span class="n">node_1</span> <span class="o">=</span> <span class="nn">Node</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="k">let</span> <span class="n">node_2</span> <span class="o">=</span> <span class="nn">Node</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">2</span><span class="p">);</span>
<span class="k">let</span> <span class="n">node_3</span> <span class="o">=</span> <span class="nn">Node</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">3</span><span class="p">);</span>
<span class="c">// Connect some of the nodes (with directed edges)</span>
<span class="n">node_1</span><span class="nf">.add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="n">node_2</span><span class="p">);</span>
<span class="n">node_1</span><span class="nf">.add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="n">node_3</span><span class="p">);</span>
<span class="n">node_2</span><span class="nf">.add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="n">node_1</span><span class="p">);</span>
<span class="n">node_3</span><span class="nf">.add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="n">node_1</span><span class="p">);</span>
<span class="c">// Add nodes to graph</span>
<span class="k">let</span> <span class="n">graph</span> <span class="o">=</span> <span class="nn">Arc</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="nn">Graph</span><span class="p">::</span><span class="nf">with_nodes</span><span class="p">(</span><span class="nd">vec!</span><span class="p">[</span><span class="n">node_1</span><span class="p">,</span> <span class="n">node_2</span><span class="p">,</span> <span class="n">node_3</span><span class="p">]));</span>
<span class="c">// Spawn a new thread that will print information about every node in the</span>
<span class="c">// graph.</span>
<span class="c">// The new scope makes this block more obviously different from the code</span>
<span class="c">// surrounding it and lets us group variables that will be moved into the</span>
<span class="c">// new thread, such as "graph".</span>
<span class="k">let</span> <span class="n">guard</span> <span class="o">=</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">graph</span> <span class="o">=</span> <span class="n">graph</span><span class="nf">.clone</span><span class="p">();</span>
<span class="k">let</span> <span class="n">message</span> <span class="o">=</span> <span class="s">"Failed to acquire a read lock"</span><span class="p">;</span>
<span class="nn">thread</span><span class="p">::</span><span class="nf">spawn</span><span class="p">(</span><span class="k">move</span> <span class="p">||</span> <span class="p">{</span>
<span class="k">for</span> <span class="n">_</span> <span class="n">in</span> <span class="mi">0</span><span class="o">..</span><span class="mi">10</span> <span class="p">{</span>
<span class="c">// Show every node in the graph and list their neighbors</span>
<span class="k">for</span> <span class="n">node</span> <span class="n">in</span> <span class="o">&</span><span class="n">graph</span><span class="py">.nodes</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="err">.</span><span class="mi">0</span><span class="nf">.read</span><span class="p">()</span><span class="nf">.expect</span><span class="p">(</span><span class="o">&</span><span class="n">message</span><span class="p">);</span>
<span class="k">let</span> <span class="n">value</span> <span class="o">=</span> <span class="n">node</span><span class="py">.inner_value</span><span class="p">;</span>
<span class="k">let</span> <span class="n">neighbours</span> <span class="o">=</span> <span class="n">node</span><span class="py">.adjacent</span>
<span class="nf">.iter</span><span class="p">()</span>
<span class="nf">.map</span><span class="p">(|</span><span class="n">n</span><span class="p">|</span> <span class="n">n</span><span class="nf">.read</span><span class="p">()</span><span class="nf">.expect</span><span class="p">(</span><span class="o">&</span><span class="n">message</span><span class="p">)</span><span class="py">.inner_value</span><span class="p">)</span>
<span class="py">.collect</span><span class="p">::</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="n">_</span><span class="o">>></span><span class="p">();</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"node ({}) is connected to: {:?}"</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">neighbours</span><span class="p">);</span>
<span class="p">}</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"-------------"</span><span class="p">);</span>
<span class="c">// Give the main thread a chance to run</span>
<span class="nn">thread</span><span class="p">::</span><span class="nf">yield_now</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">})</span>
<span class="p">};</span>
<span class="k">for</span> <span class="n">_</span> <span class="n">in</span> <span class="mi">0</span><span class="o">..</span><span class="mi">10</span> <span class="p">{</span>
<span class="c">// Update the value of every node in the graph</span>
<span class="k">for</span> <span class="n">node</span> <span class="n">in</span> <span class="o">&</span><span class="n">graph</span><span class="py">.nodes</span> <span class="p">{</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="err">.</span><span class="mi">0</span><span class="nf">.write</span><span class="p">()</span><span class="nf">.expect</span><span class="p">(</span><span class="s">"Failed to acquire a write lock"</span><span class="p">);</span>
<span class="n">node</span><span class="py">.inner_value</span> <span class="o">+=</span> <span class="mi">10</span><span class="p">;</span>
<span class="p">}</span>
<span class="c">// Give the other thread a chance to run</span>
<span class="nn">thread</span><span class="p">::</span><span class="nf">yield_now</span><span class="p">();</span>
<span class="p">}</span>
<span class="c">// Wait for the other thread to end</span>
<span class="n">guard</span><span class="nf">.join</span><span class="p">()</span><span class="nf">.expect</span><span class="p">(</span><span class="s">"Error joining thread"</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>You can view the differences between the single- and multi-threaded versions in this <a href="https://gist.github.com/meqif/bd8a85654b230e8ecd1d3344fc707de9/revisions">gist</a>.</p>
<p>Apart from creating a new thread that prints the graph’s information and a new loop in the main thread that updates the node’s values, the only important differences are:</p>
<ul>
<li><code class="highlighter-rouge">Arc<RwLock<_>></code> instead of <code class="highlighter-rouge">Rc<RefCell<_>></code></li>
<li>Every call to <code class="highlighter-rouge">borrow</code> and <code class="highlighter-rouge">borrow_mut</code> was replaced with <code class="highlighter-rouge">read</code> and <code class="highlighter-rouge">write</code>, respectively.</li>
</ul>
<p>I encourage you to make these changes yourself starting from the <a href="https://play.rust-lang.org/?gist=9ccf40fae2347519fcae7dd42ddf5ed6">previous article’s code</a>. Making small, incremental changes and compiling the intermediate versions usually helps me get an intuitive feel for how the API works.</p>
<p>Note that, unlike <code class="highlighter-rouge">borrow</code> and <code class="highlighter-rouge">borrow_mut</code>, <code class="highlighter-rouge">read</code> and <code class="highlighter-rouge">write</code> return a <code class="highlighter-rouge">LockResult</code>, which is a type alias for <code class="highlighter-rouge">Result<Guard, PoisonError<Guard>></code>, and requires us to <code class="highlighter-rouge">match</code>, <code class="highlighter-rouge">unwrap</code> or <code class="highlighter-rouge">expect</code> it. The <code class="highlighter-rouge">Guard</code> is automatically coerced into a reference, so we can pretty much ignore it.</p>
<p>In my experience, you rarely need to deal with the error case for either <code class="highlighter-rouge">read</code> or <code class="highlighter-rouge">write</code>, as it only happens if another thread with a mutable reference (i.e., a successful <code class="highlighter-rouge">write</code>) panics. In that case, you have a more serious bug elsewhere that you need to take care of.</p>
<p>Both <code class="highlighter-rouge">read</code> and <code class="highlighter-rouge">write</code> will block the thread until it gets the requested lock. Since you may not want to wait indefinitely, you can also call <code class="highlighter-rouge">try_read</code> and <code class="highlighter-rouge">try_write</code> instead, which won’t block and return an error if they couldn’t get a lock.</p>
<p>Besides <code class="highlighter-rouge">RwLock</code>, there is also <a href="https://doc.rust-lang.org/std/sync/struct.Mutex.html"><code class="highlighter-rouge">std::sync::Mutex</code></a>, whose name comes from “mutual exclusion”, because it guarantees mutually exclusive access to the wrapped value, i.e., only one thread can access it at a time. Because of this, it’s always safe to mutate the value once you get access to it.</p>
<p>We can look at <code class="highlighter-rouge">Mutex</code> as if it were an <code class="highlighter-rouge">RwLock</code> without <code class="highlighter-rouge">read</code>, only able to give the caller mutable references. In this sense, it’s more constraining than the regular borrow system or an <code class="highlighter-rouge">RwLock</code>, which allow either multiple simultaneous readers (with immutable references) or only one writer (a mutable reference) at a time. Even if we want an innocent immutable reference, we must get full permission to the inner value.</p>
<p>As there is only one kind of borrow for <code class="highlighter-rouge">Mutex</code> values, <code class="highlighter-rouge">read</code> and <code class="highlighter-rouge">write</code> are replaced by a single <code class="highlighter-rouge">lock</code> method, which will block the thread until the current lock owner releases it (i.e., the other borrow ends). Like with <code class="highlighter-rouge">RwLock</code>, if you don’t want to block the thread when the value isn’t available, you can call <code class="highlighter-rouge">try_lock</code> instead, which will either give you the lock/mutable reference or an error (<code class="highlighter-rouge">Err</code>).<sup id="fnref:3"><a href="#fn:3" class="footnote">3</a></sup></p>
<h2 id="reference-counting">Reference counting</h2>
<p>As I mentioned earlier, <code class="highlighter-rouge">std::rc::Rc</code> lacks synchronization control, which makes it unsafe to be used by multiple threads. Its thread-safe counterpart is <a href="https://doc.rust-lang.org/std/sync/struct.Arc.html"><code class="highlighter-rouge">Arc</code></a>, which lives in <code class="highlighter-rouge">std::sync</code>, along with <code class="highlighter-rouge">RwLock</code> and <code class="highlighter-rouge">Mutex</code>.</p>
<p><code class="highlighter-rouge">Arc</code> is very similar to <code class="highlighter-rouge">Rc</code> but relies on <code class="highlighter-rouge">AtomicUsize</code> for the reference counter, which makes it safe to be updated by more than one thread, unlike <code class="highlighter-rouge">Rc</code>, which uses <code class="highlighter-rouge">Cell<usize></code>.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">struct</span> <span class="nb">Rc</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="p">{</span>
<span class="n">ptr</span><span class="p">:</span> <span class="n">Shared</span><span class="o"><</span><span class="n">RcBox</span><span class="o"><</span><span class="n">T</span><span class="o">>></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">struct</span> <span class="n">RcBox</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="p">{</span>
<span class="n">strong</span><span class="p">:</span> <span class="n">Cell</span><span class="o"><</span><span class="nb">usize</span><span class="o">></span><span class="p">,</span>
<span class="n">weak</span><span class="p">:</span> <span class="n">Cell</span><span class="o"><</span><span class="nb">usize</span><span class="o">></span><span class="p">,</span>
<span class="n">value</span><span class="p">:</span> <span class="n">T</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">pub</span> <span class="k">struct</span> <span class="nb">Arc</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="p">{</span>
<span class="n">ptr</span><span class="p">:</span> <span class="n">Shared</span><span class="o"><</span><span class="n">ArcInner</span><span class="o"><</span><span class="n">T</span><span class="o">>></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">struct</span> <span class="n">ArcInner</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="o">?</span><span class="n">Sized</span><span class="o">></span> <span class="p">{</span>
<span class="n">strong</span><span class="p">:</span> <span class="nn">atomic</span><span class="p">::</span><span class="n">AtomicUsize</span><span class="p">,</span>
<span class="n">weak</span><span class="p">:</span> <span class="nn">atomic</span><span class="p">::</span><span class="n">AtomicUsize</span><span class="p">,</span>
<span class="n">data</span><span class="p">:</span> <span class="n">T</span><span class="p">,</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="highlighter-rouge">Arc</code>’s API is identical to <code class="highlighter-rouge">Rc</code>’s, which makes replacing one with the other a simple matter of searching and replacing the name and correcting the import.</p>
<h1 id="closing-thoughts">Closing thoughts</h1>
<p>In the previous article, we learned that interior mutability in Rust can be achieved through <code class="highlighter-rouge">Cell</code> and <code class="highlighter-rouge">RefCell</code> in single-threaded programs, supported by <code class="highlighter-rouge">Rc</code> where necessary. In this article, we saw the same can be done in a safe way in concurrent programs with <code class="highlighter-rouge">Atomic</code> types and <code class="highlighter-rouge">RwLock</code>, with the help of <code class="highlighter-rouge">Arc</code>.</p>
<div class="center">
<table>
<thead>
<tr>
<th> </th>
<th>Single thread</th>
<th>Multiple threads</th>
</tr>
</thead>
<tbody>
<tr>
<td>Copy value</td>
<td><code class="highlighter-rouge">Cell</code></td>
<td><code class="highlighter-rouge">Atomic*</code></td>
</tr>
<tr>
<td>Non-Copy value</td>
<td><code class="highlighter-rouge">RefCell</code></td>
<td><code class="highlighter-rouge">RwLock</code>, <code class="highlighter-rouge">Mutex</code></td>
</tr>
<tr>
<td>Reference counter</td>
<td><code class="highlighter-rouge">Rc</code></td>
<td><code class="highlighter-rouge">Arc</code></td>
</tr>
</tbody>
</table>
</div>
<p>The table above summarizes the types to be used in single- and multi-threaded scenarios.</p>
<div class="center">
<table>
<thead>
<tr>
<th>Type of access</th>
<th>Borrow checker</th>
<th><code class="highlighter-rouge">RefCell</code></th>
<th><code class="highlighter-rouge">RwLock</code></th>
<th><code class="highlighter-rouge">Mutex</code></th>
</tr>
</thead>
<tbody>
<tr>
<td>shared / read-only</td>
<td><code class="highlighter-rouge">&T</code></td>
<td><code class="highlighter-rouge">borrow</code></td>
<td><code class="highlighter-rouge">read</code></td>
<td>-</td>
</tr>
<tr>
<td>exclusive / writable</td>
<td><code class="highlighter-rouge">&mut T</code></td>
<td><code class="highlighter-rouge">borrow_mut</code></td>
<td><code class="highlighter-rouge">write</code></td>
<td><code class="highlighter-rouge">lock</code></td>
</tr>
</tbody>
</table>
</div>
<p>This second table highlights the similarities between the borrow checker, <code class="highlighter-rouge">RefCell</code>, <code class="highlighter-rouge">RwLock</code> and, to a lesser degree, <code class="highlighter-rouge">Mutex</code>.</p>
<p>You could ask, “Why bother opting for <code class="highlighter-rouge">RefCell</code> and <code class="highlighter-rouge">Rc</code> if <code class="highlighter-rouge">RwLock</code> and <code class="highlighter-rouge">Arc</code> have identical semantics and very similar APIs?”</p>
<p>Unfortunately, the types we explored in this article (<code class="highlighter-rouge">Atomic</code> types, <code class="highlighter-rouge">RwLock</code>, <code class="highlighter-rouge">Mutex</code> and <code class="highlighter-rouge">Arc</code>) depend on synchronization primitives with higher run-time overhead than their naive counterparts, and we’ll want to avoid them whenever possible.</p>
<p>We can have a regular <code class="highlighter-rouge">Rc</code> and <code class="highlighter-rouge">RefCell</code> combination for variables that aren’t shared with other threads, and their synchronized versions for the bits you want to parallelize. Because the API and semantics are similar in both cases, we will have little cognitive overhead using both.</p>
<p>Another important point we must also pay attention to, is the semantics of the wrapping. For instance, <code class="highlighter-rouge">Arc<Vec<RwLock<T>>></code> is different from <code class="highlighter-rouge">Arc<RwLock<Vec<T>>></code>. With the first, we can’t concurrently mutate the vector itself, but we can mutate its stored values. This is a consequence of <code class="highlighter-rouge">Arc</code> implementing <code class="highlighter-rouge">Deref</code> but not <code class="highlighter-rouge">DerefMut</code>, meaning we can only get immutable references to the vector (which contains lockable elements). With the second form, we get an immutable reference to <code class="highlighter-rouge">RwLock</code>, but since it can give us both kinds of references through <code class="highlighter-rouge">read</code> and <code class="highlighter-rouge">write</code>, we can mutate the vector, adding or removing elements. However, we lose the ability to concurrently mutate the values: once a thread gets a mutable reference to the vector, the others will have to wait, while the first form allows us to have one thread mutating each element in parallel.</p>
<p>In short, <code class="highlighter-rouge">Arc<Vec<RwLock<T>>></code> allows us to mutate all elements of the vector in parallel if we wish to do so, while <code class="highlighter-rouge">Arc<RwLock<Vec<T>>></code> allows only one thread to modify the vector (and its values), leaving other threads waiting for the lock.</p>
<p>If <code class="highlighter-rouge">T</code> is itself wrapped by <code class="highlighter-rouge">Arc</code>, we would be unable to mutate the stored values at all (because <code class="highlighter-rouge">Arc</code> coerces only to immutable references). We would need a monstrosity like <code class="highlighter-rouge">Arc<RwLock<Vec<Arc<RwLock<T>>>>></code> to be able to concurrently mutate both the vector and its elements, but we should take this a hint to rethink how you want to parallelize your code.</p>
<p>Exploring interior mutability in a concurrent environment made me realize that Rust is different from other languages regarding locks. Whereas in other languages we use locks to protect code fragments, in Rust we use them to protect access to data.</p>
<p>In addition, locks are so similar in use to Rust’s borrow mechanisms that they feel like a thread-safe generalization of those mechanisms. Personally, I find this way much easier to reason about than the classic, C way.</p>
<p>Finally, I know it’s tempting, but don’t throw interior mutability everywhere just to make the borrow checker shut up. Consider carefully whether the situation really calls for interior mutability or a refactor of your data structures. This goes doubly so for concurrent programs, not only because <code class="highlighter-rouge">RwLock</code>, <code class="highlighter-rouge">Mutex</code> and <code class="highlighter-rouge">Arc</code> incur additional run-time costs, but also because synchronization is easy to mess up and leave you with race conditions. Fortunately, Rust’s borrow checker gives us precious guidance and makes them much less likely. Race conditions are nasty to debug, be especially careful about dropping your locks as soon as possible.</p>
<p>That’s it. Now you know enough to effectively employ interior mutability in your programs, whether they are single- or multi-threaded. Well done! 🎉</p>
<p>My thanks to <a href="https://www.reddit.com/r/rust/comments/4puabs/interior_mutability_in_rust_part_2_thread_safety/d4octag">/u/Manishearth</a>, <a href="https://www.reddit.com/r/rust/comments/4puabs/interior_mutability_in_rust_part_2_thread_safety/d4o0cna">/u/Steel_Neuron</a>, and <a href="https://www.reddit.com/r/rust/comments/4puabs/interior_mutability_in_rust_part_2_thread_safety/d4pgcxy">/u/diwic</a> for their feedback.</p>
<p>I hope you found this article useful and/or interesting. As always, if you found a mistake or have any questions, please ping me on Twitter (<a href="https://twitter.com/meqif">@meqif</a>) or send me an email (<a href="mailto:words@ricardomartins.cc">words@ricardomartins.cc</a>). You can also join the discussion on <a href="https://www.reddit.com/r/rust/comments/4puabs/interior_mutability_in_rust_part_2_thread_safety/">reddit</a>.</p>
<div class="footnotes">
<ol>
<li id="fn:1">
<p>We can implement a marker trait or its negation for a type. Currently this only works with marker traits, such as <code class="highlighter-rouge">Sync</code> and <code class="highlighter-rouge">Send</code>, which don’t have associated methods but provide useful type information. There is work being done toward generalizing this feature, called negative traits, to any trait (<a href="https://github.com/rust-lang/rust/issues/13231">rust-lang/rust #13231</a>). <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
<li id="fn:2">
<p>Actually, <code class="highlighter-rouge">read</code> and <code class="highlighter-rouge">write</code> will return <code class="highlighter-rouge">Result<RwLockReadGuard></code> or <code class="highlighter-rouge">Result<RwLockWriteGuard></code>, respectively. <code class="highlighter-rouge">RwLockReadGuard</code> implements <code class="highlighter-rouge">Deref</code>, and <code class="highlighter-rouge">RwLockWriteGuard</code> implements both <code class="highlighter-rouge">Deref</code> and <code class="highlighter-rouge">DerefMut</code>, which are transparently coerced into mutable (<code class="highlighter-rouge">&T</code>) and immutable (<code class="highlighter-rouge">&mut T</code>) references, respectively. <a href="#fnref:2" class="reversefootnote">↩</a></p>
</li>
<li id="fn:3">
<p>Similarly to <code class="highlighter-rouge">RwLock</code>, calling <code class="highlighter-rouge">lock</code> on a <code class="highlighter-rouge">Mutex</code> will return a <code class="highlighter-rouge">Result<MutexGuard></code>, which implements both <code class="highlighter-rouge">Deref</code> and <code class="highlighter-rouge">DerefMut</code>, and can be coerced into <code class="highlighter-rouge">&T</code> or <code class="highlighter-rouge">&mut T</code>, respectively. <a href="#fnref:3" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
http://ricardomartins.cc/2016/06/08/interior-mutabilityInterior mutability in Rust: what, why, how?2016-06-08T00:00:00-05:002016-06-08T00:00:00-05:00Ricardo Martinshttp://ricardomartins.cc/<blockquote>
<p>Rust is like doing parkour while suspended on strings & wearing protective gear. Yes, it will sometimes look a little ridiculous, but you’ll be able to do all sorts of cool moves without hurting yourself. – <a href="https://www.reddit.com/r/rust/comments/4l44z3/why_should_i_use_rust/d3k7ayi">llogiq on reddit</a></p>
</blockquote>
<div class="summary">
<h1 id="key-takeaways">Key takeaways</h1>
<ul>
<li>Interior mutability is when you have an immutable reference (<code class="highlighter-rouge">&T</code>) but you can mutate the destination</li>
<li>It’s useful when you need mutable fields inside immutable data structures</li>
<li><code class="highlighter-rouge">std::cell::Cell<T></code> and <code class="highlighter-rouge">std::cell::RefCell<T></code> can be used to achieve interior mutability</li>
<li><code class="highlighter-rouge">Cell</code> wraps <code class="highlighter-rouge">Copy</code> values and doesn’t have borrow checking</li>
<li><code class="highlighter-rouge">RefCell</code> wraps any kind of value, has run-time borrow checking, and requires “locking” with <code class="highlighter-rouge">borrow</code> or <code class="highlighter-rouge">borrow_mut</code>, which give you an immutable or mutable reference, respectively</li>
<li>Neither are safe to be directly used in a multi-threaded environment. Use <code class="highlighter-rouge">Mutex</code> or <code class="highlighter-rouge">RwLock</code> instead.</li>
</ul>
</div>
<p><em>This article is part of a series about interior mutability in Rust. You can read <a href="/2016/06/25/interior-mutability-thread-safety">part 2 here</a> and <a href="/2016/07/11/interior-mutability-behind-the-curtain">part 3 here</a>.</em></p>
<h1 id="introduction">Introduction</h1>
<p>Sometimes data structures need to mutate one or more of their fields even when they are declared immutable. This may sound surprising at first, but you’ve probably relied on this behavior before, like when you clone a reference counted wrapper such as <code class="highlighter-rouge">Rc</code>, or when you lock a <code class="highlighter-rouge">Mutex</code>. However, in Rust, mutability is an all-or-nothing attribute: either a variable is declared as mutable and all of its fields are also mutable (if it is a <code class="highlighter-rouge">struct</code>), or it’s declared immutable and so are all of its fields. How do we get selective field mutability? Something mysterious is afoot.</p>
<p>Have you ever wondered how <code class="highlighter-rouge">Rc</code> is implemented? Let’s have a try! A naive first solution would be something like the following:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="n">NaiveRc</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="n">reference_count</span><span class="p">:</span> <span class="nb">usize</span><span class="p">,</span>
<span class="n">inner_value</span><span class="p">:</span> <span class="n">T</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span> <span class="n">Clone</span> <span class="k">for</span> <span class="n">NaiveRc</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">clone</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="k">self</span><span class="py">.reference_count</span> <span class="o">+=</span> <span class="mi">1</span><span class="p">;</span>
<span class="c">// ...</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>You probably spotted the problem right away: <code class="highlighter-rouge">clone</code> takes a read-only reference to <code class="highlighter-rouge">self</code>, so the reference count can’t be updated! Dang it.</p>
<p>We could implement a special, differently-named cloning function that takes <code class="highlighter-rouge">&mut self</code>, but that is awful for usability (because it defies the convention of simply checking if a type implements <code class="highlighter-rouge">Clone</code>), and forces the user of our API to always declare mutable instances of that type. We also know that the reference counted wrappers in the standard library (<a href="https://doc.rust-lang.org/std/rc/struct.Rc.html"><code class="highlighter-rouge">std::rc::Rc</code></a> and <a href="https://doc.rust-lang.org/std/sync/struct.Arc.html"><code class="highlighter-rouge">std::sync::Arc</code></a>) don’t rely on that solution, which suggests there’s another way.</p>
<p>So, how did they solve this problem in <code class="highlighter-rouge">Rc</code> and <code class="highlighter-rouge">Arc</code>? Does the standard library rely on some sort of specially nasty magic? Not at all!</p>
<p>This is an instance of <strong>interior mutability</strong>, and the Rust language provides you with tools to easily and cleanly solve this sort of situation.</p>
<h1 id="what">What?</h1>
<p>Interior mutability is a concept that many programmers new to Rust have never come across before or haven’t had to explicitly think about. It’s also more visible in Rust than in most other programming languages because we have to think about the mutability or not of variables and function arguments. This is even more evident when we look at immutable vs. mutable as shared vs. exclusive access to objects. The unspoken heuristic is that avoiding mutability when possible is good.</p>
<p>And yet, in some cases you need a few mutable fields in data structures, whether they’re mutable or not. Interior mutability gives you that additional flexibility and allows you to hide implementation details from the user of your API, while preserving (some measure of) access safety.</p>
<p>To explain what interior mutability is, it’s better to step back a little and start with something you’re familiar about: exterior mutability.</p>
<p><strong>Exterior mutability</strong> is the sort of mutability you get from mutable references (<code class="highlighter-rouge">&mut T</code>). The type of declaration, <code class="highlighter-rouge">&T</code> or <code class="highlighter-rouge">&mut T</code>, makes it clear if you’re free to update a variable or call mutating methods on objects. Exterior mutability is checked and enforced at compile-time, as you know:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="n">Foo</span> <span class="p">{</span> <span class="n">x</span><span class="p">:</span> <span class="nb">u32</span> <span class="p">};</span>
<span class="k">let</span> <span class="n">foo</span> <span class="o">=</span> <span class="n">Foo</span> <span class="p">{</span> <span class="n">x</span><span class="p">:</span> <span class="mi">1</span> <span class="p">};</span>
<span class="c">// The borrow checker will complain about this and abort compilation</span>
<span class="n">foo</span><span class="py">.x</span> <span class="o">=</span> <span class="mi">2</span><span class="p">;</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">bar</span> <span class="o">=</span> <span class="n">Foo</span> <span class="p">{</span> <span class="n">x</span><span class="p">:</span> <span class="mi">1</span> <span class="p">};</span>
<span class="c">// 'bar' is mutable, so you can change the content of any of its fields</span>
<span class="n">bar</span><span class="py">.x</span> <span class="o">=</span> <span class="mi">2</span><span class="p">;</span>
</code></pre></div></div>
<p>If you have an immutable reference, you can’t change the value. Conversely, because there is no field-level mutability in Rust, if you want to mutate a single field, you need to make the entire structure mutable, and that’s it. Or is it? Not so fast.</p>
<p><strong>Interior mutability</strong>, in contrast, is when you have an immutable reference (i.e., <code class="highlighter-rouge">&T</code>) but you can mutate the data structure. As I mentioned before, that’s what happens when you clone an <code class="highlighter-rouge">Rc</code> or lock a <code class="highlighter-rouge">Mutex</code> (both <code class="highlighter-rouge">Mutex::lock</code> and <code class="highlighter-rouge">Mutex::try_lock</code> work in immutable instances).</p>
<p>A simple example will make the difference clearer. Suppose we have a simple structure like the following:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="n">Point</span> <span class="p">{</span> <span class="n">x</span><span class="p">:</span> <span class="nb">i32</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="nb">i32</span> <span class="p">}</span>
</code></pre></div></div>
<p>An immutable <code class="highlighter-rouge">Point</code> can be seen as an immutable memory chunk, whose fields (sections of the memory chunk) can’t have their content changed at all. When you declare an immutable <code class="highlighter-rouge">Point</code> your hands are tied.</p>
<p>Consider now a slightly different, magically-enhanced <code class="highlighter-rouge">MagicPoint</code>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="n">MagicPoint</span> <span class="p">{</span> <span class="n">x</span><span class="p">:</span> <span class="nb">i32</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">Magic</span><span class="o"><</span><span class="nb">i32</span><span class="o">></span> <span class="p">}</span>
</code></pre></div></div>
<picture>
<source media="(min-width: 600px)" srcset="diagram.png 1x, diagram-hd.png 2x" />
<source srcset="diagram-small.png 1x, diagram-small-hd.png 2x" />
<img src="diagram.png" alt="Representation of Point and MagicPoint" />
</picture>
<p>For now, ignore how <code class="highlighter-rouge">Magic</code> works, and think of it as a pointer to a mutable memory address, a new layer of indirection. Like previously, if you have an immutable <code class="highlighter-rouge">MagicPoint</code>, you can’t assign new values to any of its fields. However, in this case you don’t need to change the content of <code class="highlighter-rouge">y</code>, only the destination of that magical pointer, i.e., the other memory chunk, and that one <em>is</em> mutable!<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup></p>
<p>To be clear, even though the API for <code class="highlighter-rouge">Magic</code> will make it seem as if you’re relying on indirection to access and update the wrapped value, the memory representation of <code class="highlighter-rouge">MagicPoint</code> will actually be flat.</p>
<p>Note that when you rely on interior mutability, you are giving up the compile-time safety guarantees that exterior mutability gives you. As we’ll see next, it’s not that bad, provided you’re careful.</p>
<h1 id="how">How?</h1>
<p>So, how can we get magical mutable pointers? Fortunately for us, the Rust standard library provides two wrappers, <a href="https://doc.rust-lang.org/std/cell/struct.Cell.html"><code class="highlighter-rouge">std::cell::Cell</code></a> and <a href="https://doc.rust-lang.org/std/cell/struct.RefCell.html"><code class="highlighter-rouge">std::cell::RefCell</code></a>, that allow us to introduce interior mutability in externally immutable instances of data structures. With <code class="highlighter-rouge">Cell<T></code> and <code class="highlighter-rouge">RefCell<T></code> in our collective toolbelts, we can harness the power of interior mutability.</p>
<p>Both wrappers provide interior mutability and give up compile-time borrow checking on the inner value, but give different safety guarantees and serve different purposes. The most obvious difference between them is that <code class="highlighter-rouge">RefCell</code> makes run-time borrow checks, while <code class="highlighter-rouge">Cell</code> does not.</p>
<p><code class="highlighter-rouge">Cell</code> is quite simple to use: you can read and write a <code class="highlighter-rouge">Cell</code>’s inner value by calling <code class="highlighter-rouge">get</code> or <code class="highlighter-rouge">set</code> on it. Since there are no compile-time or run-time checks, you do have to be careful to avoid some bugs the borrow checker would stop you from writing, such as accidentally overwriting the wrapped value:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">cell</span><span class="p">::</span><span class="n">Cell</span><span class="p">;</span>
<span class="k">fn</span> <span class="nf">foo</span><span class="p">(</span><span class="n">cell</span><span class="p">:</span> <span class="o">&</span><span class="n">Cell</span><span class="o"><</span><span class="nb">u32</span><span class="o">></span><span class="p">)</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">value</span> <span class="o">=</span> <span class="n">cell</span><span class="nf">.get</span><span class="p">();</span>
<span class="n">cell</span><span class="nf">.set</span><span class="p">(</span><span class="n">value</span> <span class="o">*</span> <span class="mi">2</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">cell</span> <span class="o">=</span> <span class="nn">Cell</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="k">let</span> <span class="n">value</span> <span class="o">=</span> <span class="n">cell</span><span class="nf">.get</span><span class="p">();</span>
<span class="k">let</span> <span class="n">new_value</span> <span class="o">=</span> <span class="n">cell</span><span class="nf">.get</span><span class="p">()</span> <span class="o">+</span> <span class="mi">1</span><span class="p">;</span>
<span class="nf">foo</span><span class="p">(</span><span class="o">&</span><span class="n">cell</span><span class="p">);</span>
<span class="n">cell</span><span class="nf">.set</span><span class="p">(</span><span class="n">new_value</span><span class="p">);</span> <span class="c">// oops, we clobbered the work done by foo</span>
<span class="p">}</span>
</code></pre></div></div>
<p>In contrast, a <code class="highlighter-rouge">RefCell</code> requires you to call <code class="highlighter-rouge">borrow</code> or <code class="highlighter-rouge">borrow_mut</code> (immutable and mutable borrows) before using it, yielding a pointer to the value. Its borrow semantics are identical to externally mutable variables: you can have either a mutable borrow on the inner value or several immutable borrows, so the kind of bug I mentioned earlier is detected in run-time.</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">cell</span><span class="p">::</span><span class="n">Cell</span><span class="p">;</span>
<span class="k">struct</span> <span class="n">NaiveRc</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="n">inner_value</span><span class="p">:</span> <span class="n">T</span><span class="p">,</span>
<span class="n">references</span><span class="p">:</span> <span class="n">Cell</span><span class="o"><</span><span class="nb">usize</span><span class="o">></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">NaiveRc</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">new</span><span class="p">(</span><span class="n">inner</span><span class="p">:</span> <span class="n">T</span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="n">NaiveRc</span> <span class="p">{</span>
<span class="n">inner_value</span><span class="p">:</span> <span class="n">inner</span><span class="p">,</span>
<span class="n">references</span><span class="p">:</span> <span class="nn">Cell</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">references</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="nb">usize</span> <span class="p">{</span>
<span class="k">self</span><span class="py">.references</span><span class="nf">.get</span><span class="p">()</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="p">:</span> <span class="n">Clone</span><span class="o">></span> <span class="n">Clone</span> <span class="k">for</span> <span class="n">NaiveRc</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">clone</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="k">self</span><span class="py">.references</span><span class="nf">.set</span><span class="p">(</span><span class="k">self</span><span class="py">.references</span><span class="nf">.get</span><span class="p">()</span> <span class="o">+</span> <span class="mi">1</span><span class="p">);</span>
<span class="n">NaiveRc</span> <span class="p">{</span>
<span class="n">inner_value</span><span class="p">:</span> <span class="k">self</span><span class="py">.inner_value</span><span class="nf">.clone</span><span class="p">(),</span>
<span class="n">references</span><span class="p">:</span> <span class="k">self</span><span class="py">.references</span><span class="nf">.clone</span><span class="p">(),</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">wrapped</span> <span class="o">=</span> <span class="nn">NaiveRc</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="s">"Hello!"</span><span class="p">);</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"references before cloning: {:?}"</span><span class="p">,</span> <span class="n">wrapped</span><span class="nf">.references</span><span class="p">());</span>
<span class="k">let</span> <span class="n">wrapped_clone</span> <span class="o">=</span> <span class="n">wrapped</span><span class="nf">.clone</span><span class="p">();</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"references after cloning: {:?}"</span><span class="p">,</span> <span class="n">wrapped</span><span class="nf">.references</span><span class="p">());</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"clone references: {:?}"</span><span class="p">,</span> <span class="n">wrapped_clone</span><span class="nf">.references</span><span class="p">());</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Calling <code class="highlighter-rouge">borrow</code> or <code class="highlighter-rouge">borrow_mut</code> on a mutably borrowed <code class="highlighter-rouge">RefCell</code> will cause a panic, as will calling <code class="highlighter-rouge">borrow_mut</code> on a immutably borrowed value. This aspect makes <code class="highlighter-rouge">RefCell</code> unsuitable to be used in a parallel scenario; you should use a thread-safe type (like a <code class="highlighter-rouge">Mutex</code> or a <code class="highlighter-rouge">RwLock</code>, for example) instead.</p>
<p>A <code class="highlighter-rouge">RefCell</code> will stay “locked” until the pointer you received falls out of scope, so you might want to declare a new block scope (ie., <code class="highlighter-rouge">{ ... }</code>) while working with the borrowed value, or even explicitly <code class="highlighter-rouge">drop</code> the borrowed value when you’re done with it, to avoid unpleasant surprises.</p>
<p>Another significant difference between <code class="highlighter-rouge">Cell</code> and <code class="highlighter-rouge">RefCell</code> is that <code class="highlighter-rouge">Cell<T></code> requires that the inner value <code class="highlighter-rouge">T</code> implements <code class="highlighter-rouge">Copy</code>, while <code class="highlighter-rouge">RefCell<T></code> has no such restriction. Often, you won’t want copy semantics on your wrapped types, so you’ll have to use <code class="highlighter-rouge">RefCell</code>.</p>
<p>Put succinctly, <code class="highlighter-rouge">Cell</code> has <code class="highlighter-rouge">Copy</code> semantics and provides <em>values</em>, while <code class="highlighter-rouge">RefCell</code> has <code class="highlighter-rouge">move</code> semantics and provides <em>references</em>.</p>
<h1 id="why">Why?</h1>
<p>There are a few general cases that call for interior mutability, such as:</p>
<ol>
<li>Introducing mutability inside of something immutable</li>
<li>Mutating implementations of <code class="highlighter-rouge">Clone</code></li>
<li>Implementation details of logically immutable methods</li>
<li>Mutating reference-counted variables</li>
</ol>
<h2 id="introducing-mutability-inside-of-something-immutable">Introducing mutability inside of something immutable</h2>
<p>Returning to the <code class="highlighter-rouge">NaiveRc</code> example in the introduction, reference-counting pointers, like <code class="highlighter-rouge">Rc</code> and <code class="highlighter-rouge">Arc</code>, need internal mutability. When you clone those pointers, the reference counter inside them has to be updated, whether they’re mutable or not. Without interior mutability, you would be forced to always use mutable pointers, which would allow mutation of the inner value and may be undesired.</p>
<p>For instance, consider the following naive reference counted wrapper:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">cell</span><span class="p">::</span><span class="n">Cell</span><span class="p">;</span>
<span class="k">struct</span> <span class="n">NaiveRc</span><span class="o"><</span><span class="nv">'a</span><span class="p">,</span> <span class="n">T</span><span class="p">:</span> <span class="nv">'a</span><span class="o">></span> <span class="p">{</span>
<span class="n">inner_value</span><span class="p">:</span> <span class="o">&</span><span class="nv">'a</span> <span class="n">T</span><span class="p">,</span>
<span class="n">references</span><span class="p">:</span> <span class="n">Cell</span><span class="o"><</span><span class="nb">usize</span><span class="o">></span>
<span class="p">}</span>
<span class="k">let</span> <span class="n">x</span> <span class="o">=</span> <span class="n">NaiveRc</span> <span class="p">{</span> <span class="n">inner_value</span><span class="p">:</span> <span class="o">&</span><span class="mi">1</span><span class="p">,</span> <span class="n">references</span><span class="p">:</span> <span class="nn">Cell</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">};</span>
<span class="n">x</span><span class="py">.references</span><span class="nf">.set</span><span class="p">(</span><span class="mi">2</span><span class="p">);</span> <span class="c">// it works!</span>
<span class="n">x</span><span class="py">.inner_value</span> <span class="o">=</span> <span class="o">&</span><span class="mi">2</span><span class="p">;</span> <span class="c">// beep boop, x is immutable,</span>
<span class="c">// you can't assign a new value to any of its fields!</span>
</code></pre></div></div>
<h2 id="mutating-implementations-of-clone">Mutating implementations of <code class="highlighter-rouge">Clone</code></h2>
<p>Way back in the introduction, we noticed that cloning a reference-counted value (<code class="highlighter-rouge">Rc<T></code>) needs to increment the reference counter. This is simply a special case of the previous point, but it deserves reiterating.</p>
<p>On the other hand, dropping such a value requires decrementing the reference counter, but <code class="highlighter-rouge">drop</code> works with mutable references (<code class="highlighter-rouge">fn drop(&mut self)</code>), so there’s no problem there.</p>
<h2 id="implementation-details-of-logically-immutable-methods">Implementation details of logically immutable methods</h2>
<p>For instance, you might want to amortize the running time of an expensive algorithm operating on your data structure by using a cache inside it. The cache must be able to be updated even when the data structure itself is immutable.</p>
<h2 id="mutating-reference-counted-variables">Mutating reference-counted variables</h2>
<p>Suppose we need multiple references to some objects. For example, when connecting nodes in a graph. “<em>Oh, that’s easy</em>”, you think. “<em>I’ll just wrap my nodes in <code class="highlighter-rouge">Rc</code> or <code class="highlighter-rouge">Arc</code> and call it a day</em>”. That a perfectly reasonable line of though, and it would work… if you never, ever needed to mutate nodes. Once you try building the graph by incrementally adding and connecting nodes, the compiler will give you grief. Oh no, what is going on? Unfortunately for us, <code class="highlighter-rouge">Rc</code> preserves safety by only giving you shared (i.e., immutable) references when you call <code class="highlighter-rouge">clone</code>. Quoth the <a href="https://doc.rust-lang.org/stable/std/rc/"><code class="highlighter-rouge">std::rc</code> module documentation</a>:</p>
<blockquote>
<p>The Rc<T> type provides shared ownership of an immutable value. Destruction is deterministic, and will occur as soon as the last owner is gone.</p>
</blockquote>
<p>You could call <code class="highlighter-rouge">get_mut</code> to receive an <code class="highlighter-rouge">Option<&mut T></code>, but that would work only once: <code class="highlighter-rouge">get_mut</code> only returns a mutable reference as if there is only one “strong” reference to the value.<sup id="fnref:2"><a href="#fn:2" class="footnote">2</a></sup> Foiled again!</p>
<p>Fortunately, you can use interior mutability here: use <code class="highlighter-rouge">Rc<Cell<T>></code> or <code class="highlighter-rouge">Rc<RefCell<T>></code>. That way you can <code class="highlighter-rouge">clone</code> the reference-counted wrapper as much as you want and still modify the innermost value wrapped by <code class="highlighter-rouge">Cell</code> or <code class="highlighter-rouge">RefCell</code>.</p>
<p>You can see a first try at a solution <a href="https://play.rust-lang.org/?gist=bd01037fd9b9bb3d1a15bde61f580c6f">in this example in the Rust Playground</a>. As you can see, the problem is solved, but the solution is verbose and ugly. Not only that, the user of our API is aware of implementation details! What gives? Where’s the elegant abstraction I promised a few paragraphs above?</p>
<p>Now that you’ve seen and understood how this works, I can show you a cleaner version:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">cell</span><span class="p">::</span><span class="n">RefCell</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">rc</span><span class="p">::</span><span class="nb">Rc</span><span class="p">;</span>
<span class="c">// A graph can be represented in several ways. For the sake of illustrating how</span>
<span class="c">// interior mutability works in practice, let's go with the simplest</span>
<span class="c">// representation: a list of nodes.</span>
<span class="c">// Each node has an inner value and a list of adjacent nodes it is connected to</span>
<span class="c">// (through a directed edge).</span>
<span class="c">// That list of adjacent nodes cannot be the exclusive owner of those nodes, or</span>
<span class="c">// else each node would have at most one edge to another node and the graph</span>
<span class="c">// couldn't also own these nodes.</span>
<span class="c">// We need to wrap Node with a reference-counted box, such as Rc or Arc. We'll</span>
<span class="c">// go with Rc, because this is a toy example.</span>
<span class="c">// However, Rc<T> and Arc<T> enforce memory safety by only giving out shared</span>
<span class="c">// (i.e., immutable) references to the wrapped object, and we need mutability to</span>
<span class="c">// be able to connect nodes together.</span>
<span class="c">// The solution for this problem is wrapping Node in either Cell or RefCell, to</span>
<span class="c">// restore mutability. We're going to use RefCell because Node<T> doesn't</span>
<span class="c">// implement Copy (we don't want to have independent copies of nodes!).</span>
<span class="c">// Represents a reference to a node.</span>
<span class="c">// This makes the code less repetitive to write and easier to read.</span>
<span class="k">type</span> <span class="n">NodeRef</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="o">=</span> <span class="nb">Rc</span><span class="o"><</span><span class="n">RefCell</span><span class="o"><</span><span class="n">_Node</span><span class="o"><</span><span class="n">T</span><span class="o">>>></span><span class="p">;</span>
<span class="c">// The private representation of a node.</span>
<span class="k">struct</span> <span class="n">_Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="n">inner_value</span><span class="p">:</span> <span class="n">T</span><span class="p">,</span>
<span class="n">adjacent</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="n">NodeRef</span><span class="o"><</span><span class="n">T</span><span class="o">>></span><span class="p">,</span>
<span class="p">}</span>
<span class="c">// The public representation of a node, with some syntactic sugar.</span>
<span class="k">struct</span> <span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">(</span><span class="n">NodeRef</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">);</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="c">// Creates a new node with no edges.</span>
<span class="k">fn</span> <span class="nf">new</span><span class="p">(</span><span class="n">inner</span><span class="p">:</span> <span class="n">T</span><span class="p">)</span> <span class="k">-></span> <span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">let</span> <span class="n">node</span> <span class="o">=</span> <span class="n">_Node</span> <span class="p">{</span> <span class="n">inner_value</span><span class="p">:</span> <span class="n">inner</span><span class="p">,</span> <span class="n">adjacent</span><span class="p">:</span> <span class="nd">vec!</span><span class="p">[]</span> <span class="p">};</span>
<span class="nf">Node</span><span class="p">(</span><span class="nn">Rc</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="nn">RefCell</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="n">node</span><span class="p">)))</span>
<span class="p">}</span>
<span class="c">// Adds a directed edge from this node to other node.</span>
<span class="k">fn</span> <span class="nf">add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">,</span> <span class="n">other</span><span class="p">:</span> <span class="o">&</span><span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">></span><span class="p">)</span> <span class="p">{</span>
<span class="p">(</span><span class="k">self</span><span class="err">.</span><span class="mi">0</span><span class="nf">.borrow_mut</span><span class="p">())</span><span class="py">.adjacent</span><span class="nf">.push</span><span class="p">(</span><span class="n">other</span><span class="err">.</span><span class="mi">0</span><span class="nf">.clone</span><span class="p">());</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">struct</span> <span class="n">Graph</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="n">nodes</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">>></span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">Graph</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">fn</span> <span class="nf">with_nodes</span><span class="p">(</span><span class="n">nodes</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="n">Node</span><span class="o"><</span><span class="n">T</span><span class="o">>></span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="n">Graph</span> <span class="p">{</span> <span class="n">nodes</span><span class="p">:</span> <span class="n">nodes</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="c">// Create some nodes</span>
<span class="k">let</span> <span class="n">node_1</span> <span class="o">=</span> <span class="nn">Node</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="k">let</span> <span class="n">node_2</span> <span class="o">=</span> <span class="nn">Node</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">2</span><span class="p">);</span>
<span class="k">let</span> <span class="n">node_3</span> <span class="o">=</span> <span class="nn">Node</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">3</span><span class="p">);</span>
<span class="c">// Connect some of the nodes (with directed edges)</span>
<span class="n">node_1</span><span class="nf">.add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="n">node_2</span><span class="p">);</span>
<span class="n">node_1</span><span class="nf">.add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="n">node_3</span><span class="p">);</span>
<span class="n">node_2</span><span class="nf">.add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="n">node_1</span><span class="p">);</span>
<span class="n">node_3</span><span class="nf">.add_adjacent</span><span class="p">(</span><span class="o">&</span><span class="n">node_1</span><span class="p">);</span>
<span class="c">// Add nodes to graph</span>
<span class="k">let</span> <span class="n">graph</span> <span class="o">=</span> <span class="nn">Graph</span><span class="p">::</span><span class="nf">with_nodes</span><span class="p">(</span><span class="nd">vec!</span><span class="p">[</span><span class="n">node_1</span><span class="p">,</span> <span class="n">node_2</span><span class="p">,</span> <span class="n">node_3</span><span class="p">]);</span>
<span class="c">// Show every node in the graph and list their neighbors</span>
<span class="k">for</span> <span class="n">node</span> <span class="n">in</span> <span class="n">graph</span><span class="py">.nodes</span><span class="nf">.iter</span><span class="p">()</span><span class="nf">.map</span><span class="p">(|</span><span class="n">n</span><span class="p">|</span> <span class="n">n</span><span class="err">.</span><span class="mi">0</span><span class="nf">.borrow</span><span class="p">())</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">value</span> <span class="o">=</span> <span class="n">node</span><span class="py">.inner_value</span><span class="p">;</span>
<span class="k">let</span> <span class="n">neighbours</span> <span class="o">=</span> <span class="n">node</span><span class="py">.adjacent</span><span class="nf">.iter</span><span class="p">()</span>
<span class="nf">.map</span><span class="p">(|</span><span class="n">n</span><span class="p">|</span> <span class="n">n</span><span class="nf">.borrow</span><span class="p">()</span><span class="py">.inner_value</span><span class="p">)</span>
<span class="py">.collect</span><span class="p">::</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="n">_</span><span class="o">>></span><span class="p">();</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"node ({}) is connected to: {:?}"</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">neighbours</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>If you ignore the loop that prints out the graph’s information, now the user doesn’t know how a <code class="highlighter-rouge">Node</code> is implemented. This version’s usability can still be improved by implementing the <code class="highlighter-rouge">std::fmt::Debug</code> trait for <code class="highlighter-rouge">Node</code> and <code class="highlighter-rouge">Graph</code>, for instance.</p>
<p>You can <a href="https://play.rust-lang.org/?gist=9ccf40fae2347519fcae7dd42ddf5ed6">play with this example</a> in the Rust Playground. Try changing some things yourself! I find breaking things helps me consolidate new knowledge. I suggest:</p>
<ol>
<li>Replacing <code class="highlighter-rouge">RefCell</code> with <code class="highlighter-rouge">Cell</code></li>
<li>Removing <code class="highlighter-rouge">RefCell</code> and using <code class="highlighter-rouge">Rc<Node<T>></code></li>
<li>Removing <code class="highlighter-rouge">Rc</code> and using <code class="highlighter-rouge">RefCell<Node<T>></code></li>
</ol>
<p>You could also try replacing <code class="highlighter-rouge">Rc</code> with <code class="highlighter-rouge">Arc</code>, but you wouldn’t notice anything different. <code class="highlighter-rouge">Arc</code> is a thread-safe version of <code class="highlighter-rouge">Rc</code>, which comes with a performance cost and doesn’t really make sense in single-threaded programs.</p>
<p><a href="https://play.rust-lang.org/?gist=9ce1029d16433114f6bdda32b2e9fc03">An alternative solution</a> could involve wrapping the adjacent node vector in a <code class="highlighter-rouge">RefCell</code> instead of wrapping the node itself. That can also work, depending on what you intend to do, but it is semantically different from the previous solution, as you would be unable to mutate a node’s inner value in addition to its list of adjacent nodes.</p>
<h1 id="which-to-pick">Which to pick?</h1>
<p>If <code class="highlighter-rouge">RefCell</code> can explode in your face and shouldn’t be used “raw” in a multi-threaded program, why bother using it?</p>
<p>While <code class="highlighter-rouge">Cell</code> is a good choice for many cases, there are a few reasons you might want to use <code class="highlighter-rouge">RefCell</code>:</p>
<ol>
<li>The wrapped value doesn’t implement <code class="highlighter-rouge">Copy</code>.</li>
<li>Only <code class="highlighter-rouge">RefCell</code> has run-time checks. In some scenarios you’d rather kill the program than risk corrupting data.</li>
<li><code class="highlighter-rouge">RefCell</code> exposes pointers to the stored value, <code class="highlighter-rouge">Cell</code> doesn’t.</li>
</ol>
<p>As a rule of thumb, choose <code class="highlighter-rouge">Cell</code> if your wrapped value implements <code class="highlighter-rouge">Copy</code> (such as primitive values, like integers and floats). If the wrapped value is a <code class="highlighter-rouge">struct</code>, doesn’t implement <code class="highlighter-rouge">Copy</code> <strong>or</strong> you need dynamically checked borrows, use <code class="highlighter-rouge">RefCell</code> instead.</p>
<h1 id="wrapping-up">Wrapping up</h1>
<div class="center">
<table>
<thead>
<tr>
<th> </th>
<th>Cell</th>
<th>RefCell</th>
</tr>
</thead>
<tbody>
<tr>
<td>Semantics</td>
<td>Copy</td>
<td>Move</td>
</tr>
<tr>
<td>Provides</td>
<td>Values</td>
<td>References</td>
</tr>
<tr>
<td>Panics?</td>
<td>Never</td>
<td>Mixed borrows or more than one mutable borrow</td>
</tr>
<tr>
<td>Use with</td>
<td>Primitive types</td>
<td>Structures or non-Copy types</td>
</tr>
</tbody>
</table>
</div>
<p>The table above summarizes what you learned in this blog post.</p>
<p>I hope you found this article useful and/or interesting. As always, if you found a mistake or have any questions, please ping me on Twitter (<a href="https://twitter.com/meqif">@meqif</a>) or send me an email (<a href="mailto:words@ricardomartins.cc">words@ricardomartins.cc</a>). You can also join the discussion on <a href="https://www.reddit.com/r/rust/comments/4na9p6/interior_mutability_in_rust_what_why_how/">reddit</a>.</p>
<p>As <a href="https://www.reddit.com/r/rust/comments/4na9p6/interior_mutability_in_rust_what_why_how/d425bxq">Steve Klabnik</a>, <a href="https://www.reddit.com/r/rust/comments/4na9p6/interior_mutability_in_rust_what_why_how/d42dtcz">/u/critiqjo</a> and <a href="https://www.reddit.com/r/rust/comments/4na9p6/interior_mutability_in_rust_what_why_how/d42gsdm">/u/birkenfield</a> kindly pointed out, <code class="highlighter-rouge">Mutex</code> and <code class="highlighter-rouge">RwLock</code> already have interior mutability, so there’s no need to put a <code class="highlighter-rouge">Cell</code> inside them. In multi-threaded scenarios you should use <code class="highlighter-rouge">Mutex</code> and <code class="highlighter-rouge">RwLock</code> without an additional <code class="highlighter-rouge">Cell</code> or <code class="highlighter-rouge">RefCell</code>.</p>
<p><a href="https://www.reddit.com/r/rust/comments/4na9p6/interior_mutability_in_rust_what_why_how/d42sn3z">/u/krdln</a> suggested the alternative graph implementation above.</p>
<div class="footnotes">
<ol>
<li id="fn:1">
<p>If you are familiar with C and this reminds you of <code class="highlighter-rouge">const</code> pointers (whose value also can’t change but the content at the destination memory address can), you are in the right track. <code class="highlighter-rouge">y</code> would be something like a <code class="highlighter-rouge">int *const</code>. <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
<li id="fn:2">
<p>I really don’t want to get into the strong and weak reference thing now. Suffice it to say that strong references stop objects from being destroyed, while weak references don’t. <a href="#fnref:2" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
http://ricardomartins.cc/2015/10/12/practical_differences_between_rust_closures_and_functionsPractical differences between Rust closures and functions2015-10-12T00:00:00-05:002015-10-12T00:00:00-05:00Ricardo Martinshttp://ricardomartins.cc/<p>Elizabeth <a href="https://users.rust-lang.org/t/taking-suggestions-for-the-new-rust-faq/2813/27">asks an interesting question</a>: “What’s the difference between a function and a closure that doesn’t enclose any variable?”</p>
<p>When I read that question, I was intrigued. Using closures instead of functions tends to be more costly because closures capture some of the environment, and that has some overhead. In practice, we usually don’t think about the difference when programming, and the choice of one over the other comes down to personal preference and what feels right.</p>
<p>However, Rust has a reasonably smart compiler that can take advantage from a wide array of optimisations, and <a href="http://blog.rust-lang.org/2015/05/11/traits.html">zero cost abstractions</a> is one of Rust’s strengths, so it’s also reasonable to expect little difference between a function and a closure with no free variables (i.e., no enclosed variables) after optimisation.</p>
<p>This matter can be unfolded into several smaller questions, but for the sake of pragmatism I will be focusing on the differences in the code generated by the <strike>current stable</strike> nightly release of the compiler, <code class="highlighter-rouge">rustc 1.5.0-nightly (7bf4c885f 2015-09-26)</code>. The easiest way to compare the generated assembly code is to compile the two different versions with <code class="highlighter-rouge">rustc -C opt-level=2 --emit asm</code> and check the differences with <code class="highlighter-rouge">diff</code>. Comparing only the optimised code will help us avoid drowning in inconsequential differences, since the debug builds are only useful for development and aren’t supposed to be used in production. 😉</p>
<p>Alright, let’s go ahead and compare the following two examples (adapted from Elizabeth’s question):</p>
<h3 id="function">Function</h3>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">double</span><span class="p">(</span><span class="n">x</span><span class="p">:</span> <span class="nb">i32</span><span class="p">)</span> <span class="k">-></span> <span class="nb">i32</span> <span class="p">{</span>
<span class="mi">2</span> <span class="o">*</span> <span class="n">x</span>
<span class="p">}</span>
<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">v</span> <span class="o">=</span> <span class="nd">vec!</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">);</span>
<span class="k">let</span> <span class="n">w</span> <span class="o">=</span> <span class="n">v</span><span class="nf">.into_iter</span><span class="p">()</span><span class="nf">.map</span><span class="p">(</span><span class="n">double</span><span class="p">);</span>
<span class="c">// Make sure the result isn't optimised away</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"{:?}"</span><span class="p">,</span> <span class="n">w</span><span class="py">.collect</span><span class="p">::</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="nb">i32</span><span class="o">>></span><span class="p">());</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="closure">Closure</h3>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">v</span> <span class="o">=</span> <span class="nd">vec!</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">);</span>
<span class="k">let</span> <span class="n">w</span> <span class="o">=</span> <span class="n">v</span><span class="nf">.into_iter</span><span class="p">()</span><span class="nf">.map</span><span class="p">(|</span><span class="n">x</span><span class="p">|</span> <span class="mi">2</span> <span class="o">*</span> <span class="n">x</span><span class="p">);</span>
<span class="c">// Make sure the result isn't optimised away</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"{:?}"</span><span class="p">,</span> <span class="n">w</span><span class="py">.collect</span><span class="p">::</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="nb">i32</span><span class="o">>></span><span class="p">());</span>
<span class="p">}</span>
</code></pre></div></div>
<p>I made both versions available on the Rust playground: <a href="https://play.rust-lang.org/?gist=344b0fb2cca9d4618951&version=stable">function</a> and <a href="https://play.rust-lang.org/?gist=9ad2695b4083e0a4b7ee&version=stable">closure</a>. If you want to see for yourself the output of running <code class="highlighter-rouge">diff</code> on the resulting files, <a href="https://gist.github.com/meqif/6c9e451a4a56c8d23f37">here you go</a>. Be warned: the diff is “noisy” because some CPU registers are used in one version but not in the other, and there is also <a href="https://en.wikipedia.org/wiki/Name_mangling">name mangling</a>.</p>
<p>Looking at the generated assembly code, the only significant difference is that the code corresponding to the body of the function/closure, <code class="highlighter-rouge">2 * x</code>, is preceded by a call to <code class="highlighter-rouge">__rust_allocate</code> (<a href="https://doc.rust-lang.org/std/rt/heap/fn.allocate.html"><code class="highlighter-rouge">std::rt::heap::allocate</code></a>), and followed by a call to <code class="highlighter-rouge">__rust_deallocate</code> (<a href="https://doc.rust-lang.org/std/rt/heap/fn.deallocate.html"><code class="highlighter-rouge">std::rt::heap::deallocate</code></a>), but only in the version where a closure is used.</p>
<strike>That makes sense: the closure is actually an instance of `FnOnce`, which we can see being created and destroyed with `allocate` and `deallocate`, respectively.</strike>
<p><strong>EDIT</strong>: After I published this post, <a href="https://twitter.com/huon_w/status/654111077425266688">Huon Wilson</a> and <a href="https://twitter.com/dot_dash/status/654218821906092032">Björn Steinbrink</a> took a look to the generated assembly code and concluded that the extra allocation is actually an instance of the vector <code class="highlighter-rouge">v</code> that isn’t being optimized away in the closure version, which is a little surprising. Björn suspects [<a href="https://twitter.com/dot_dash/status/654219152991879168">1</a>, <a href="https://twitter.com/dot_dash/status/654219428880629760">2</a>] this is due to a different set of optimizations being applied in each version. Huon also pointed out that closures are never implicitly on the heap, as he explains in an <a href="http://huonw.github.io/blog/2015/05/finding-closure-in-rust/">excellent blog post</a> about closures.
Thanks for the precious feedback, Huon and Björn!</p>
<p>For more details about how closures are implemented in Rust, check the <a href="https://doc.rust-lang.org/stable/book/closures.html#closure-implementation">Closures chapter</a> in the official book, as well as Huon Wilson’s <a href="http://huonw.github.io/blog/2015/05/finding-closure-in-rust/">blog post</a>.</p>
<p>I also ran a small benchmark pitting both versions against each other, but there is no measurable difference between both versions.</p>
<p>The bottom line is, <strong>for most users there is no practical difference between functions and closures without captured variables</strong>. Use whatever seems right. Personally, I prefer using closures when they’re small and used in one place (like in the previous examples), and functions otherwise. Remember Harold Abelson’s advice:</p>
<blockquote>
<p>“Programs must be written for people to read, and only incidentally for machines to execute.”</p>
</blockquote>
<h2 id="bonus-round-named-closure">Bonus round: named closure</h2>
<p>Before hitting “Publish”, I discussed that observation with a friend unfamiliar with Rust, and he thought it might be interesting to check if there was any difference between using a closure directly in the argument, as previously, and using a named closure, i.e., assigning the closure to a variable and passing it to <code class="highlighter-rouge">map</code>, like this:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">double</span> <span class="o">=</span> <span class="p">|</span><span class="n">x</span><span class="p">|</span> <span class="mi">2</span> <span class="o">*</span> <span class="n">x</span><span class="p">;</span>
<span class="k">let</span> <span class="n">v</span> <span class="o">=</span> <span class="nd">vec!</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">);</span>
<span class="k">let</span> <span class="n">w</span> <span class="o">=</span> <span class="n">v</span><span class="nf">.into_iter</span><span class="p">()</span><span class="nf">.map</span><span class="p">(</span><span class="n">double</span><span class="p">);</span>
<span class="c">// Make sure the result isn't optimised away</span>
<span class="nd">println!</span><span class="p">(</span><span class="s">"{:?}"</span><span class="p">,</span> <span class="n">w</span><span class="py">.collect</span><span class="p">::</span><span class="o"><</span><span class="nb">Vec</span><span class="o"><</span><span class="nb">i32</span><span class="o">>></span><span class="p">());</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Since both the Rust compiler and LLVM (the compiler infrastructure that powers the compiler) are smart, the intermediate variable is optimises away, generating exactly the same code for this version as it did for the one with an anonymous closure.</p>
<h2 id="update">Update</h2>
<p><a href="https://twitter.com/JakeGoulding/status/654486706935230464">Jake Goulding</a> noticed I goofed up and used the nightly release of the compiler instead of the stable release I meant to use. I recompiled the examples with the current stable <code class="highlighter-rouge">rustc 1.3.0 (9a92aaf19 2015-09-15)</code> and generated <a href="https://gist.github.com/meqif/ee9459f6440ede652665">a new diff</a>. As you can see in this diff, the extra allocation <em>is</em> optimised away, just like it should. I apologize for the mistake. Thank you, Jake!</p>
http://ricardomartins.cc/2014/04/09/bleeding-hearts-and-sharp-bitsBleeding hearts and sharp bits2014-04-09T00:00:00-05:002014-04-09T00:00:00-05:00Ricardo Martinshttp://ricardomartins.cc/<blockquote>
<p>If you can’t dance, don’t blame the dance floor.</p>
</blockquote>
<p><a href="http://heartbleed.com">Heartbleed</a>, as you’ve probably read by now, is a serious security issue resulting from a bug in <a href="http://openssl.org">OpenSSL</a> which allows an attacker to freely read the memory of the affected servers. Serious 💩, indeed.</p>
<p>Some people were quick to point their fingers at the language used for implementation of the SSL standard in the OpenSSL project—that is to say, C—and are clamoring for a new implementation in what they deem a safer language, such as <a href="http://www.rust-lang.org/">Rust</a> and <a href="http://golang.org/">Go</a>.</p>
<p>Now, C is an old language and in many aspects it’s little more than assembly language dressed up. It has sharp edges and people cut themselves on those very often — dealing explicitly with memory allocation and pointers is prone to error, even for experienced programmers. I agree that having to deal with all the minutiae is cumbersome and usually unnecessary in the modern day. However, I believe the problem isn’t so much that the language allows you to shoot yourself in the foot but that many programmers are careless.</p>
<p>Let’s have a look at the <a href="http://git.openssl.org/gitweb/?p=openssl.git;a=blob;f=ssl/t1_lib.c;h=b82fadace66e764b47ab2d854621ad89b804e8d2#l2582">code</a> before the <a href="http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=96db9023b881d7cd9f379b0c154650d6c108e9a3">fix</a>:</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">int</span>
<span class="nf">dtls1_process_heartbeat</span><span class="p">(</span><span class="n">SSL</span> <span class="o">*</span><span class="n">s</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">unsigned</span> <span class="kt">char</span> <span class="o">*</span><span class="n">p</span> <span class="o">=</span> <span class="o">&</span><span class="n">s</span><span class="o">-></span><span class="n">s3</span><span class="o">-></span><span class="n">rrec</span><span class="p">.</span><span class="n">data</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="o">*</span><span class="n">pl</span><span class="p">;</span>
<span class="kt">unsigned</span> <span class="kt">short</span> <span class="n">hbtype</span><span class="p">;</span>
<span class="kt">unsigned</span> <span class="kt">int</span> <span class="n">payload</span><span class="p">;</span>
<span class="kt">unsigned</span> <span class="kt">int</span> <span class="n">padding</span> <span class="o">=</span> <span class="mi">16</span><span class="p">;</span> <span class="cm">/* Use minimum padding */</span>
</code></pre></div></div>
<p>Rather boring variable declaration, although the names could be better (what kind of mischievous monster names a length variable <code class="highlighter-rouge">payload</code>?).</p>
<p>That SSL type is rather opaque, but following the trail, one ends up with this structure:</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">typedef</span> <span class="k">struct</span> <span class="n">ssl3_record_st</span>
<span class="p">{</span>
<span class="kt">int</span> <span class="n">type</span><span class="p">;</span> <span class="cm">/* type of record */</span>
<span class="kt">unsigned</span> <span class="kt">int</span> <span class="n">length</span><span class="p">;</span> <span class="cm">/* How many bytes available */</span>
<span class="kt">unsigned</span> <span class="kt">int</span> <span class="n">off</span><span class="p">;</span> <span class="cm">/* read/write offset into 'buf' */</span>
<span class="kt">unsigned</span> <span class="kt">char</span> <span class="o">*</span><span class="n">data</span><span class="p">;</span> <span class="cm">/* pointer to the record data */</span>
<span class="kt">unsigned</span> <span class="kt">char</span> <span class="o">*</span><span class="n">input</span><span class="p">;</span> <span class="cm">/* where the decode bytes are */</span>
<span class="kt">unsigned</span> <span class="kt">char</span> <span class="o">*</span><span class="n">comp</span><span class="p">;</span> <span class="cm">/* only used with decompression - malloc()ed */</span>
<span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">epoch</span><span class="p">;</span> <span class="cm">/* epoch number, needed by DTLS1 */</span>
<span class="kt">unsigned</span> <span class="kt">char</span> <span class="n">seq_num</span><span class="p">[</span><span class="mi">8</span><span class="p">];</span> <span class="cm">/* sequence number, needed by DTLS1 */</span>
<span class="p">}</span> <span class="n">SSL3_RECORD</span><span class="p">;</span>
</code></pre></div></div>
<p>Anyway, the argument <code class="highlighter-rouge">s</code> is read as a SSL3 record and <code class="highlighter-rouge">char p</code> is left pointing to <code class="highlighter-rouge">s</code>’s data. So far, nothing unusual or interesting. Onwards!</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/* Read type and payload length first */</span>
<span class="n">hbtype</span> <span class="o">=</span> <span class="o">*</span><span class="n">p</span><span class="o">++</span><span class="p">;</span>
<span class="n">n2s</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">payload</span><span class="p">);</span>
<span class="n">pl</span> <span class="o">=</span> <span class="n">p</span><span class="p">;</span>
</code></pre></div></div>
<p><img src="/images/zHtjSoU.jpg" alt="" /></p>
<p>Ok, let’s go through this slowly.</p>
<p><code class="highlighter-rouge">p</code> is of type <code class="highlighter-rouge">char</code>, so they’re jumping over the first byte and copying the first two bytes to <code class="highlighter-rouge">hbtype</code> (which is a <code class="highlighter-rouge">short</code>). This is because the heartbeat extension sticks this stuff into the data portion of <code class="highlighter-rouge">SSL3_RECORD</code>, so some gymnastics is needed to access it.</p>
<p><code class="highlighter-rouge">n2s</code> and <code class="highlighter-rouge">s2n</code> are defined in <code class="highlighter-rouge">ssl/ssl_locl.h</code> as follows:</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#define n2s(c,s) ((s=(((unsigned int)(c[0]))<< 8)| \
(((unsigned int)(c[1])) )),c+=2)
#define s2n(s,c) ((c[0]=(unsigned char)(((s)>> 8)&0xff), \
c[1]=(unsigned char)(((s) )&0xff)),c+=2)
</span></code></pre></div></div>
<p><code class="highlighter-rouge">n2s</code> is supposed to read two <code class="highlighter-rouge">chars</code> and return them inside an integer (or anything with at least 2 bytes). What it actually does is read two of whatever the <code class="highlighter-rouge">c</code> pointer says its size is and return the result of arithmetical OR between them. A pedantic detail but dangerous if misused. I suppose strong type checking could be useful here to prevent mistakes.</p>
<p><code class="highlighter-rouge">s2n</code> is similar but does the opposite—dump two <code class="highlighter-rouge">chars</code> into an integer.</p>
<p>So, we read a couple of bytes, convert them to a <code class="highlighter-rouge">short</code> and assign that to <code class="highlighter-rouge">payload</code>, which is the payload length.</p>
<p>Then <code class="highlighter-rouge">pl</code> is changed to point to the payload data, including the two bytes indicating its supposed length.</p>
<p>What’s becomes apparent if you’re following the code carefully is that the heartbeat payload length hasn’t been checked for validity yet and, as we’ll later see, isn’t before it’s used.
This is probably the most common kind of error in C, which makes it somewhat more appalling because it should be at the forefront of the programmer’s mental checks.</p>
<p>Later on, there’s this:</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="p">(</span><span class="n">hbtype</span> <span class="o">==</span> <span class="n">TLS1_HB_REQUEST</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">unsigned</span> <span class="kt">char</span> <span class="o">*</span><span class="n">buffer</span><span class="p">,</span> <span class="o">*</span><span class="n">bp</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">r</span><span class="p">;</span>
<span class="cm">/* Allocate memory for the response, size is 1 bytes
* message type, plus 2 bytes payload length, plus
* payload, plus padding
*/</span>
<span class="n">buffer</span> <span class="o">=</span> <span class="n">OPENSSL_malloc</span><span class="p">(</span><span class="mi">1</span> <span class="o">+</span> <span class="mi">2</span> <span class="o">+</span> <span class="n">payload</span> <span class="o">+</span> <span class="n">padding</span><span class="p">);</span>
<span class="n">bp</span> <span class="o">=</span> <span class="n">buffer</span><span class="p">;</span>
<span class="cm">/* Enter response type, length and copy payload */</span>
<span class="o">*</span><span class="n">bp</span><span class="o">++</span> <span class="o">=</span> <span class="n">TLS1_HB_RESPONSE</span><span class="p">;</span>
<span class="n">s2n</span><span class="p">(</span><span class="n">payload</span><span class="p">,</span> <span class="n">bp</span><span class="p">);</span>
<span class="n">memcpy</span><span class="p">(</span><span class="n">bp</span><span class="p">,</span> <span class="n">pl</span><span class="p">,</span> <span class="n">payload</span><span class="p">);</span>
<span class="n">bp</span> <span class="o">+=</span> <span class="n">payload</span><span class="p">;</span>
</code></pre></div></div>
<p>First a buffer for the response is allocated, then a region of memory is copied. Both their sizes depend on the payload length received earlier and it wasn’t bound-checked!</p>
<p>This allows an attacker to send a heartbeat with an arbitrary payload length (up to 64KB (2<sup>16</sup> bytes) of memory) and a much smaller payload body, leading to reads beyond the SSL record.
64KB seems small but it’s enough to get SSL certificates, passwords, cookies, whatever is reachable.</p>
<p>I’m not entirely sure that this mistake is of the sort that a cleverer language would catch, since it’s likely that the <code class="highlighter-rouge">memcpy</code> will happen within the application’s memory. If it tried to reach outside, the kernel would take notice and kill it with a Segmentation Fault, turning this issue into a less threatening program termination bug.</p>
<p>I know I shouldn’t be surprised at how trusting most programmers are about user input, but these people are programming a <em>cryptography library</em>. They should know better. Scratch that, everyone <em>trusts</em> them to know better and to be paranoid. To say I’m disappointed is putting it lightly.</p>
<p>While I agree that it would benefit everyone if a safer language was used for this sort of software, I do not agree that the blame lies entirely with C.
Careless mistakes can happen with any language. The language used may reduce the likelihood that a mistake comes up in places with serious consequences but vigilance is absolutely essential.</p>
http://ricardomartins.cc/2014/01/14/rebootReboot2014-01-14T00:00:00-06:002014-01-14T00:00:00-06:00Ricardo Martinshttp://ricardomartins.cc/<p>New layout, new almost everything.</p>
<aside>
<p>Yes, I know it's hard to tell.</p>
</aside>
<p>This was mostly an excuse to start over and play with HTML5 and CSS. Nothing too fancy and somewhat responsive.</p>
<p>Since I tend to make parenthetical comments, I played a little with the <code class="highlighter-rouge">aside</code> tag, mostly in conjunction with CSS flexboxes for ease of positioning and responsive design. While they worked well in Chrome (both desktop and mobile), I ran into some issues in other browsers.</p>
<p>At the time of writing, the state of flexbox support across browsers isn’t very encouraging: <a href="http://stackoverflow.com/tags/flexbox/info">most browsers support some version of flexbox</a> but Chrome is the only one whose current (and most popular) version has support for the latest working draft, while Firefox has a sadly incomplete implementation even in the latest beta (27.0 at the time of writing). I guess I will have to wait some more time or come up with alternative solutions.</p>
<p>The current version of this blog makes use of the following code:</p>
<ul>
<li><a href="https://davecoyle.com/tech-notes/jekyll-templates-for-atom-rss/">RSS and Atom templates</a> courtesy of <a href="https://davecoyle.com/">Dave Coyle</a>.</li>
<li><a href="https://github.com/havvg/havvg.github.com/blob/master/sitemap.xml">Sitemap</a> by <a href="http://toni.uebernickel.info/">Toni Uebernickel</a></li>
</ul>