-
Notifications
You must be signed in to change notification settings - Fork 5
/
Copy pathlessons_learned_from_a_successful_rust_rewrite.html
324 lines (294 loc) · 30.6 KB
/
lessons_learned_from_a_successful_rust_rewrite.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
<!DOCTYPE html>
<html>
<head>
<title>Lessons learned from a successful Rust rewrite</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link type="application/atom+xml" href="/blog/feed.xml" rel="self">
<link rel="shortcut icon" type="image/ico" href="/blog/favicon.ico">
<link rel="stylesheet" type="text/css" href="main.css">
<link rel="stylesheet" href="https://unpkg.com/@highlightjs/[email protected]/styles/default.min.css">
<script src="highlight.min.js"></script>
<!-- From https://github.com/odin-lang/odin-lang.org/blob/6f48c2cfb094a42dffd34143884fa958bd9c0ba2/themes/odin/layouts/partials/head.html#L71 -->
<script src="x86asm.min.js"></script>
<script src="odin_syntax.js"></script>
<script type="module" src="search_index.js"></script>
<script type="module" src="search.js"></script>
</head>
<body>
<div id="banner">
<div id="name">
<img id="me" src="me.jpeg">
<span>Philippe Gaultier</span>
</div>
<input id="search" placeholder="🔎 Search" autocomplete=off>
<ul>
<li> <a href="/blog/body_of_work.html">Body of work</a> </li>
<li> <a href="/blog/articles-by-tag.html">Tags</a> </li>
<li> <a href="https://github.com/gaultier/resume/raw/master/Philippe_Gaultier_resume_en.pdf">
Resume
</a> </li>
<li> <a href="/blog/feed.xml">
<svg viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M5.5 3.5C4.39543 3.5 3.5 4.39543 3.5 5.5V18.5C3.5 19.6046 4.39543 20.5 5.5 20.5H18.5C19.6046 20.5 20.5 19.6046 20.5 18.5V5.5C20.5 4.39543 19.6046 3.5 18.5 3.5H5.5ZM7 19C8.10457 19 9 18.1046 9 17C9 15.8954 8.10457 15 7 15C5.89543 15 5 15.8954 5 17C5 18.1046 5.89543 19 7 19ZM6.14863 10.5052C6.14863 10.0379 6.52746 9.65906 6.99478 9.65906C7.95949 9.65906 8.91476 9.84908 9.80603 10.2183C10.6973 10.5874 11.5071 11.1285 12.1893 11.8107C12.8715 12.4929 13.4126 13.3027 13.7817 14.194C14.1509 15.0852 14.3409 16.0405 14.3409 17.0052C14.3409 17.4725 13.9621 17.8514 13.4948 17.8514C13.0275 17.8514 12.6486 17.4725 12.6486 17.0052C12.6486 16.2627 12.5024 15.5275 12.2183 14.8416C11.9341 14.1556 11.5177 13.5324 10.9927 13.0073C10.4676 12.4823 9.84437 12.0659 9.15842 11.7817C8.47246 11.4976 7.73726 11.3514 6.99478 11.3514C6.52746 11.3514 6.14863 10.9725 6.14863 10.5052ZM7 5.15385C6.53268 5.15385 6.15385 5.53268 6.15385 6C6.15385 6.46732 6.53268 6.84615 7 6.84615C8.33342 6.84615 9.65379 7.10879 10.8857 7.61907C12.1176 8.12935 13.237 8.87728 14.1799 9.82015C15.1227 10.763 15.8707 11.8824 16.3809 13.1143C16.8912 14.3462 17.1538 15.6666 17.1538 17C17.1538 17.4673 17.5327 17.8462 18 17.8462C18.4673 17.8462 18.8462 17.4673 18.8462 17C18.8462 15.4443 18.5397 13.9039 17.9444 12.4667C17.3491 11.0294 16.4765 9.72352 15.3765 8.6235C14.2765 7.52349 12.9706 6.65091 11.5333 6.05558C10.0961 5.46026 8.55566 5.15385 7 5.15385Z" fill="#000000"/>
</svg>
</a> </li>
<li> <a href="https://www.linkedin.com/in/philippegaultier/">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" data-supported-dps="24x24" fill="currentColor" class="mercado-match" width="24" height="24" focusable="false">
<path d="M20.5 2h-17A1.5 1.5 0 002 3.5v17A1.5 1.5 0 003.5 22h17a1.5 1.5 0 001.5-1.5v-17A1.5 1.5 0 0020.5 2zM8 19H5v-9h3zM6.5 8.25A1.75 1.75 0 118.3 6.5a1.78 1.78 0 01-1.8 1.75zM19 19h-3v-4.74c0-1.42-.6-1.93-1.38-1.93A1.74 1.74 0 0013 14.19a.66.66 0 000 .14V19h-3v-9h2.9v1.3a3.11 3.11 0 012.7-1.4c1.55 0 3.36.86 3.36 3.66z"/>
</svg>
</a> </li>
<li> <a href="https://github.com/gaultier">
<svg height="32" aria-hidden="true" viewBox="0 0 24 24" version="1.1" width="32" data-view-component="true" class="octicon octicon-mark-github v-align-middle">
<path d="M12.5.75C6.146.75 1 5.896 1 12.25c0 5.089 3.292 9.387 7.863 10.91.575.101.79-.244.79-.546 0-.273-.014-1.178-.014-2.142-2.889.532-3.636-.704-3.866-1.35-.13-.331-.69-1.352-1.18-1.625-.402-.216-.977-.748-.014-.762.906-.014 1.553.834 1.769 1.179 1.035 1.74 2.688 1.25 3.349.948.1-.747.402-1.25.733-1.538-2.559-.287-5.232-1.279-5.232-5.678 0-1.25.445-2.285 1.178-3.09-.115-.288-.517-1.467.115-3.048 0 0 .963-.302 3.163 1.179.92-.259 1.897-.388 2.875-.388.977 0 1.955.13 2.875.388 2.2-1.495 3.162-1.179 3.162-1.179.633 1.581.23 2.76.115 3.048.733.805 1.179 1.825 1.179 3.09 0 4.413-2.688 5.39-5.247 5.678.417.36.776 1.05.776 2.128 0 1.538-.014 2.774-.014 3.162 0 .302.216.662.79.547C20.709 21.637 24 17.324 24 12.25 24 5.896 18.854.75 12.5.75Z"/>
</svg>
</a> </li>
<li> <a href="https://hachyderm.io/@pg">
<svg width="75" height="79" viewBox="0 0 75 79" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M73.8393 17.4898C72.6973 9.00165 65.2994 2.31235 56.5296 1.01614C55.05 0.797115 49.4441 0 36.4582 0H36.3612C23.3717 0 20.585 0.797115 19.1054 1.01614C10.5798 2.27644 2.79399 8.28712 0.904997 16.8758C-0.00358524 21.1056 -0.100549 25.7949 0.0682394 30.0965C0.308852 36.2651 0.355538 42.423 0.91577 48.5665C1.30307 52.6474 1.97872 56.6957 2.93763 60.6812C4.73325 68.042 12.0019 74.1676 19.1233 76.6666C26.7478 79.2728 34.9474 79.7055 42.8039 77.9162C43.6682 77.7151 44.5217 77.4817 45.3645 77.216C47.275 76.6092 49.5123 75.9305 51.1571 74.7385C51.1797 74.7217 51.1982 74.7001 51.2112 74.6753C51.2243 74.6504 51.2316 74.6229 51.2325 74.5948V68.6416C51.2321 68.6154 51.2259 68.5896 51.2142 68.5661C51.2025 68.5426 51.1858 68.522 51.1651 68.5058C51.1444 68.4896 51.1204 68.4783 51.0948 68.4726C51.0692 68.4669 51.0426 68.467 51.0171 68.4729C45.9835 69.675 40.8254 70.2777 35.6502 70.2682C26.7439 70.2682 24.3486 66.042 23.6626 64.2826C23.1113 62.762 22.7612 61.1759 22.6212 59.5646C22.6197 59.5375 22.6247 59.5105 22.6357 59.4857C22.6466 59.4609 22.6633 59.4391 22.6843 59.422C22.7053 59.4048 22.73 59.3929 22.7565 59.3871C22.783 59.3813 22.8104 59.3818 22.8367 59.3886C27.7864 60.5826 32.8604 61.1853 37.9522 61.1839C39.1768 61.1839 40.3978 61.1839 41.6224 61.1516C46.7435 61.008 52.1411 60.7459 57.1796 59.7621C57.3053 59.7369 57.431 59.7154 57.5387 59.6831C65.4861 58.157 73.0493 53.3672 73.8178 41.2381C73.8465 40.7606 73.9184 36.2364 73.9184 35.7409C73.9219 34.0569 74.4606 23.7949 73.8393 17.4898Z" fill="url(#paint0_linear_549_34)"/>
<path d="M61.2484 27.0263V48.114H52.8916V27.6475C52.8916 23.3388 51.096 21.1413 47.4437 21.1413C43.4287 21.1413 41.4177 23.7409 41.4177 28.8755V40.0782H33.1111V28.8755C33.1111 23.7409 31.0965 21.1413 27.0815 21.1413C23.4507 21.1413 21.6371 23.3388 21.6371 27.6475V48.114H13.2839V27.0263C13.2839 22.7176 14.384 19.2946 16.5843 16.7572C18.8539 14.2258 21.8311 12.926 25.5264 12.926C29.8036 12.926 33.0357 14.5705 35.1905 17.8559L37.2698 21.346L39.3527 17.8559C41.5074 14.5705 44.7395 12.926 49.0095 12.926C52.7013 12.926 55.6784 14.2258 57.9553 16.7572C60.1531 19.2922 61.2508 22.7152 61.2484 27.0263Z" fill="white"/>
<defs>
<linearGradient id="paint0_linear_549_34" x1="37.0692" y1="0" x2="37.0692" y2="79" gradientUnits="userSpaceOnUse">
<stop stop-color="#6364FF"/>
<stop offset="1" stop-color="#563ACC"/>
</linearGradient>
</defs>
</svg>
</a> </li>
<li> <a href="https://bsky.app/profile/pgaultier.bsky.social">
<svg fill="none" viewBox="0 0 64 57" width="32" style="width: 32px; height: 28.5px;"><path fill="#0085ff" d="M13.873 3.805C21.21 9.332 29.103 20.537 32 26.55v15.882c0-.338-.13.044-.41.867-1.512 4.456-7.418 21.847-20.923 7.944-7.111-7.32-3.819-14.64 9.125-16.85-7.405 1.264-15.73-.825-18.014-9.015C1.12 23.022 0 8.51 0 6.55 0-3.268 8.579-.182 13.873 3.805ZM50.127 3.805C42.79 9.332 34.897 20.537 32 26.55v15.882c0-.338.13.044.41.867 1.512 4.456 7.418 21.847 20.923 7.944 7.111-7.32 3.819-14.64-9.125-16.85 7.405 1.264 15.73-.825 18.014-9.015C62.88 23.022 64 8.51 64 6.55c0-9.818-8.578-6.732-13.873-2.745Z"/></svg>
</a> </li>
</ul>
</div>
<div id="search-matches" hidden>
</div>
<div id="pseudo-body">
<div class="article-prelude">
<p><a href="/blog"> ⏴ Back to all articles</a></p>
<p class="publication-date">Published on 2024-10-30</p>
</div>
<div class="article-title">
<h1>Lessons learned from a successful Rust rewrite</h1>
<div class="tags"> <a href="/blog/articles-by-tag.html#rust" class="tag">Rust</a> <a href="/blog/articles-by-tag.html#cplusplus" class="tag">C++</a> <a href="/blog/articles-by-tag.html#rewrite" class="tag">Rewrite</a></div>
</div>
<strong>Table of contents</strong>
<ul>
<li>
<a href="#1596753652-what-worked-well">What worked well</a>
</li>
<li>
<a href="#3041159231-what-did-not-work-so-well">What did not work so well</a>
<ul>
<li>
<a href="#2006554032-i-am-still-chasing-undefined-behavior">I am still chasing Undefined Behavior</a>
</li>
<li>
<a href="#2694732638-miri-does-not-always-work-and-i-still-have-to-use-valgrind">Miri does not always work and I still have to use Valgrind</a>
</li>
<li>
<a href="#2584141995-i-am-still-chasing-memory-leaks">I am still chasing memory leaks</a>
</li>
<li>
<a href="#1568320768-cross-compilation-does-not-always-work">Cross-compilation does not always work</a>
</li>
<li>
<a href="#929959896-cbindgen-does-not-always-work">Cbindgen does not always work</a>
</li>
<li>
<a href="#3393406502-unstable-abi">Unstable ABI</a>
</li>
<li>
<a href="#2235693709-no-support-for-custom-memory-allocators">No support for custom memory allocators</a>
</li>
<li>
<a href="#3763530832-complexity">Complexity</a>
</li>
</ul>
</li>
<li>
<a href="#3796851539-conclusion">Conclusion</a>
</li>
</ul>
<p><em>Discussions: <a href="https://old.reddit.com/r/rust/comments/1gflxxh/lessons_learned_from_a_successful_rust_rewrite/?">/r/rust</a>, <a href="https://old.reddit.com/r/programming/comments/1gfljj7/lessons_learned_from_a_successful_rust_rewrite/?">/r/programming</a>, <a href="https://news.ycombinator.com/item?id=41994189">HN</a>, <a href="https://lobste.rs/s/n6gciw/lessons_learned_from_successful_rust">lobsters</a></em></p>
<p>I have written about my on-going rewrite-it-to-Rust effort at work: <a href="/blog/you_inherited_a_legacy_cpp_codebase_now_what.md">1</a>, <a href="/blog/how_to_rewrite_a_cpp_codebase_successfully.md">2</a>, <a href="/blog/rust_c++_interop_trick.html">3</a>. And now it's finished, meaning it's 100% Rust and 0% C++ - the public C API has not changed, just the implementation, one function at time until the end. Let's have a look back at what worked, what didn't, and what can be done about it.</p>
<p>For context, I have written projects in pure Rust before, so I won't mention all of the usual Rust complaints, like "learning it is hard", they did not affect me during this project.</p>
<h2 id="1596753652-what-worked-well">
<a class="title" href="#1596753652-what-worked-well">What worked well</a>
<a class="hash-anchor" href="#1596753652-what-worked-well" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h2>
<p>The rewrite was done incrementally, in a stop-and-go fashion. At some point, as I expected, we had to add brand new features while the rewrite was on-going and that was very smooth with this approach. Contrast this with the (wrong) approach of starting a new codebase from scratch in parallel, and then the feature has to be implemented twice.</p>
<p>The new code is much, much simpler and easier to reason about. It is roughly the same number of lines of code as the old C++ codebase, or slightly more. Some people think that equivalent Rust code will be much shorter (I have heard ratios of 1/2 or 2/3), but in my experience, it's not really the case. C++ can be incredibly verbose in some instances, but Rust as well. And the C++ code will often ignore some errors that the Rust compiler forces the developer to handle, which is a good thing, but also makes the codebase slightly bigger.</p>
<p>Undergoing a rewrite, even a bug-for-bug one like ours, opens many new doors in terms of performance. For example, some fields in C++ were assumed to be of a dynamic size, but we realized that they were always 16 bytes according to business rules, so we stored them in an array of a fixed size, thus simplifying lots of code and reducing heap allocations. That's not strictly due to Rust, it's just that having this holistic view of the codebase yields many benefits.</p>
<p>Related to this: we delete lots and lots of dead code. I estimate that we removed perhaps a third or half of the whole C++ codebase because it was simply never used. Some of it were half-assed features some long-gone customer asked for, and some were simply never run or even worse, never even built (they were C++ files not even present in the CMake build system). I feel that modern programming languages such as Rust or Go are much more aggressive at flagging dead code and pestering the developer about it, which again, is a good thing.</p>
<p>We don't have to worry about out-of-bounds accesses and overflow/underflows with arithmetic. These were the main issues in the C++ code. Even if C++ containers have this <code>.at()</code> method to do bounds check, in my experience, most people do not use them. It's nice that this happens by default. And overflows/underflows checks are typically never addressed in C and C++ codebases.</p>
<p>Cross-compilation is pretty smooth, although not always, see next section.</p>
<p>The builtin test framework in Rust is very serviceable. All the ones I used in C++ were terrible and took so much time to even compile.</p>
<p>Rust is much more concerned with correctness than C++, so it sparked a lot of useful discussions. For example: oh, the Rust compiler is forcing me to check if this byte array is valid UTF8 when I try to convert it to a string. The old C++ code did no such check. Let's add this check.</p>
<p>It felt so good to remove all the CMake files. On all the C or C++ projects I worked on, I never felt that CMake was worth it and I always lost a lot of hours to coerce it into doing what I needed.</p>
<h2 id="3041159231-what-did-not-work-so-well">
<a class="title" href="#3041159231-what-did-not-work-so-well">What did not work so well</a>
<a class="hash-anchor" href="#3041159231-what-did-not-work-so-well" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h2>
<p>This section is surprisingly long and is the most interesting in my opinion. Did Rust hold its promises?</p>
<h3 id="2006554032-i-am-still-chasing-undefined-behavior">
<a class="title" href="#2006554032-i-am-still-chasing-undefined-behavior">I am still chasing Undefined Behavior</a>
<a class="hash-anchor" href="#2006554032-i-am-still-chasing-undefined-behavior" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h3>
<p>Doing an incremental rewrite from C/C++ to Rust, we had to use a lot of raw pointers and <code>unsafe{}</code> blocks. And even when segregating these to the entry point of the library, they proved to be a big pain in the neck.</p>
<p>All the stringent rules of Rust still apply inside these blocks but the compiler just stops checking them for you, so you are on your own. As such, it's so easy to introduce Undefined Behavior. I honestly think from this experience that it is easier to inadvertently introduce Undefined Behavior in Rust than in C++, and it turn, it's easier in C++ than in C.</p>
<p>The main rule in Rust is: <del>multiple read-only pointers XOR one mutable pointer</del> <code>multiple read-only reference XOR one mutable reference</code>. That's what the borrow checker is always pestering you about.</p>
<p>But when using raw pointers, it's so easy to silently break, especially when porting C or C++ code as-is, which is mutation and pointer heavy:</p>
<p><em>Note: Astute readers have pointed out that the issue in the snippet below is having multiple mutable references, not pointers, and that using the syntax <code>let a = &raw mut x;</code> in recent Rust versions, or <code>addr_of_mut</code> in older versions, avoids creating multiple mutable references.</em></p>
<pre><code class="language-rust">fn main() {
let mut x = 1;
unsafe {
let a: *mut usize = &mut x;
let b: *mut usize = &mut x;
*a = 2;
*b = 3;
}
}
</code></pre>
<p>You might think that this code is dumb and obviously wrong, but in a big real codebase, this is not so easy to spot, especially when these operations are hidden inside helper functions or layers and layers of abstraction, as Rust loves to do.</p>
<p><code>cargo run</code> is perfectly content with the code above. The Rust compiler can and will silently assume that there is only one mutable pointer to <code>x</code>, and make optimizations, and generate machine code, based on that assumption, which this code breaks.</p>
<p>The only savior here is <a href="https://github.com/rust-lang/miri">Miri</a>:</p>
<pre><code class="language-sh">$ cargo +nightly-2024-09-01 miri r
error: Undefined Behavior: attempting a write access using <2883> at alloc1335[0x0], but that tag does not exist in the borrow stack for this location
--> src/main.rs:7:9
|
7 | *a = 2;
| ^^^^^^
| |
| attempting a write access using <2883> at alloc1335[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of an access at alloc1335[0x0..0x8]
|
[...]
--> src/main.rs:4:29
|
4 | let a: *mut usize = &mut x;
| ^^^^^^
help: <2883> was later invalidated at offsets [0x0..0x8] by a Unique retag
--> src/main.rs:5:29
|
5 | let b: *mut usize = &mut x;
| ^^^^^^
[...]
</code></pre>
<p>So, what could have been a compile time error, is now a runtime error. Great. I hope you have 100% test coverage! Thank god there's Miri.</p>
<p>If you are writing <code>unsafe{}</code> code without Miri checking it, or if you do so without absolutely having to, I think this is foolish. It will blow up in your face.</p>
<p>Miri is awesome. But...</p>
<h3 id="2694732638-miri-does-not-always-work-and-i-still-have-to-use-valgrind">
<a class="title" href="#2694732638-miri-does-not-always-work-and-i-still-have-to-use-valgrind">Miri does not always work and I still have to use Valgrind</a>
<a class="hash-anchor" href="#2694732638-miri-does-not-always-work-and-i-still-have-to-use-valgrind" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h3>
<p>I am not talking about some parts of Miri that are experimental. Or the fact that running code under Miri is excruciatingly slow. Or the fact that Miri only works in <code>nightly</code>.</p>
<p>No, I am talking about code that Miri cannot run, period:</p>
<pre><code> |
471 | let pkey_ctx = LcPtr::new(unsafe { EVP_PKEY_CTX_new_id(EVP_PKEY_EC, null_mut()) })?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't call foreign function `␁aws_lc_0_16_0_EVP_PKEY_CTX_new_id` on OS `linux`
|
= help: if this is a basic API commonly used on this target, please report an issue with Miri
= help: however, note that Miri does not aim to support every FFI function out there; for instance, we will not support APIs for things such as GUIs, scripting languages, or databases
</code></pre>
<p>If you are using a library that has parts written in C or assembly, which is usual for cryptography libraries, or video compression, etc, you are out of luck.</p>
<p>So we resorted to add a feature flag to split the codebase between parts that use this problematic library and parts that don't. And Miri only runs tests with the feature disabled.</p>
<p>That means that there is a lot of <code>unsafe</code> code that is simply not being checked right now. Bummer.</p>
<p>Perhaps there could be a fallback implementation for these libraries that's entirely implemented in software (and in pure Rust). But that's not really feasible for most libraries to maintain two implementations just for Rust developers.</p>
<p>I resorted to run the problematic tests in <code>valgrind</code>, like I used to do with pure C/C++ code. It does not detect many things that Miri would, for example having more than one mutable pointer to the same value, which is perfectly fine in C/C++/Assembly, but not in Rust.</p>
<h3 id="2584141995-i-am-still-chasing-memory-leaks">
<a class="title" href="#2584141995-i-am-still-chasing-memory-leaks">I am still chasing memory leaks</a>
<a class="hash-anchor" href="#2584141995-i-am-still-chasing-memory-leaks" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h3>
<p>Our library offers a C API, something like this:</p>
<pre><code class="language-c">void* handle = MYLIB_init();
// Do some stuff with the handle...
MYLIB_release(handle);
</code></pre>
<p>Under the hood, <code>MYLIB_init</code> allocates some memory and <code>MYLIB_release()</code> frees it. This is a very usual pattern in C libraries, e.g. <code>curl_easy_init()/curl_easy_cleanup()</code>.</p>
<p>So immediately, you are thinking: well, it's easy to forget to call <code>MYLIB_release</code> in some code paths, and thus leak memory. And you'd be right. So let's implement them to illustrate. We are good principled developers so we write a Rust test:</p>
<pre><code class="language-rust">#[no_mangle]
pub extern "C" fn MYLIB_init() -> *mut std::ffi::c_void {
let alloc = Box::leak(Box::new(1usize));
alloc as *mut usize as *mut std::ffi::c_void
}
#[no_mangle]
pub extern "C" fn MYLIB_do_stuff(_handle: *mut std::ffi::c_void) {
// Do some stuff.
}
#[no_mangle]
pub extern "C" fn MYLIB_release(handle: *mut std::ffi::c_void) {
let _ = unsafe { Box::from_raw(handle as *mut usize) };
}
fn main() {}
#[cfg(test)]
mod test {
#[test]
fn test_init_release() {
let x = super::MYLIB_init();
super::MYLIB_do_stuff(x);
super::MYLIB_release(x);
}
}
</code></pre>
<p>A Rust developer first instinct would be to use RAII by creating a wrapper object which implements <code>Drop</code> and automatically calls the cleanup function.
However, we wanted to write our tests using the public C API of the library like a normal C application would, and it would not have access to this Rust feature.
Also, it can become unwieldy when there are tens of types that have an allocation/deallocation function. It's a lot of boilerplate!</p>
<p>And often, there is complicated logic with lots of code paths, and we need to ensure that the cleanup is always called. In C, this is typically done with <code>goto</code> to an <code>end:</code> label that always cleans up the resources. But Rust does not support this form of <code>goto</code>.</p>
<p>So we solved it with the <a href="https://docs.rs/scopeguard/latest/scopeguard/">defer</a> crate in Rust and implementing a <a href="https://www.gingerbill.org/article/2015/08/19/defer-in-cpp/">defer</a> statement in C++.</p>
<p>However, the Rust borrow checker really does not like the <code>defer</code> pattern. Typically, a cleanup function will take as its argument as <code>&mut</code> reference and that precludes the rest of the code to also store and use a second <code>&mut</code> reference to the same value. So we could not always use <code>defer</code> on the Rust side.</p>
<h3 id="1568320768-cross-compilation-does-not-always-work">
<a class="title" href="#1568320768-cross-compilation-does-not-always-work">Cross-compilation does not always work</a>
<a class="hash-anchor" href="#1568320768-cross-compilation-does-not-always-work" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h3>
<p>Same issue as with Miri, using libraries with a Rust API but with parts implemented in C or Assembly will make <code>cargo build --target=...</code> not work out of the box. It won't affect everyone out there, and perhaps it can be worked around by providing a sysroot like in C or C++. But that's a bummer still. For example, I think Zig manages this situation smoothly for most targets, since it ships with a C compiler and standard library, whereas <code>cargo</code> does not.</p>
<h3 id="929959896-cbindgen-does-not-always-work">
<a class="title" href="#929959896-cbindgen-does-not-always-work">Cbindgen does not always work</a>
<a class="hash-anchor" href="#929959896-cbindgen-does-not-always-work" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h3>
<p><a href="https://github.com/mozilla/cbindgen">cbindgen</a> is a conventionally used tool to generate a C header from a Rust codebase. It mostly works, until it does not. I hit quite a number of limitations or bugs. I thought of contributing PRs, but I found for most of these issues, a stale open PR, so I didn't. Every time, I thought of dumping <code>cbindgen</code> and writing all of the C prototypes by hand. I think it would have been simpler in the end.</p>
<p>Again, as a comparison, I believe Zig has a builtin C header generation tool.</p>
<h3 id="3393406502-unstable-abi">
<a class="title" href="#3393406502-unstable-abi">Unstable ABI</a>
<a class="hash-anchor" href="#3393406502-unstable-abi" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h3>
<p>I talked about this point in my previous articles so I won't be too long. Basically, all the useful standard library types such as <code>Option</code> have no stable ABI, so they have to be replicated manually with the <code>repr(C)</code> annotation, so that they can be used from C or C++. This again is a bummer and creates friction. Note that I am equally annoyed at C++ ABI issues for the same reason.</p>
<p>Many, many hours of hair pulling would be avoided if Rust and C++ adopted, like C, a <a href="https://daniel.haxx.se/blog/2024/10/30/eighteen-years-of-abi-stability/">stable ABI</a>.</p>
<h3 id="2235693709-no-support-for-custom-memory-allocators">
<a class="title" href="#2235693709-no-support-for-custom-memory-allocators">No support for custom memory allocators</a>
<a class="hash-anchor" href="#2235693709-no-support-for-custom-memory-allocators" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h3>
<p>With lots of C libraries, the user can provide its own allocator at runtime, which is often very useful. In Rust, the developer can only pick the global allocator at compile time. So we did not attempt to offer this feature in the library API.</p>
<p>Additionally, all of the aforementioned issues about cleaning up resources would have been instantly fixed by using an <a href="/blog/tip_of_the_day_2.html">arena allocator</a>, which is not at all idiomatic in Rust and does not integrate with the standard library (even though there are crates for it). Again, Zig and Odin all support arenas natively, and it's trivial to implement and use them in C. I really longed for an arena while chasing subtle memory leaks.</p>
<h3 id="3763530832-complexity">
<a class="title" href="#3763530832-complexity">Complexity</a>
<a class="hash-anchor" href="#3763530832-complexity" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h3>
<p>From the start, I decided I would not touch async Rust with a ten-foot pole, and I did not miss it at all, for this project.</p>
<p>Whilst reading the docs for <code>UnsafeCell</code> for the fourth time, and pondering whether I should use that or <code>RefCell</code>, while just having been burnt by the pitfalls of <code>MaybeUninit</code>, and asking myself if I need <code>Pin</code>, I really asked myself what life choices had led me to this.</p>
<p>Pure Rust is already very complex, but add to it the whole layer that is mainly there to deal with FFI, and it really becomes a beast. Especially for new Rust learners.</p>
<p>Some developers in our team straight declined to work on this codebase, mentioning the real or perceived Rust complexity.
Now, I think that Rust is still mostly easier to learn than C++, but admittedly not by much, especially in this FFI heavy context.</p>
<h2 id="3796851539-conclusion">
<a class="title" href="#3796851539-conclusion">Conclusion</a>
<a class="hash-anchor" href="#3796851539-conclusion" aria-hidden="true" onclick="navigator.clipboard.writeText(this.href);"></a>
</h2>
<p>I am mostly satisfied with this Rust rewrite, but I was disappointed in some areas, and it overall took much more effort than I anticipated. Using Rust with a lot of C interop feels like using a completely different language than using pure Rust. There is much friction, many pitfalls, and many issues in C++, that Rust claims to have solved, that are in fact not really solved at all.</p>
<p>I am deeply grateful to the developers of Rust, Miri, cbindgen, etc. They have done tremendous work. Still, the language and tooling, when doing lots of C FFI, feel immature, almost pre v1.0. If the ergonomics of <code>unsafe</code> (which are being worked and slightly improved in the recent versions), the standard library, the docs, the tooling, and the unstable ABI, all improve in the future, it could become a more pleasant experience.</p>
<p>I think that all of these points have been felt by Microsoft and Google, and that's why they are investing real money in this area to improve things.</p>
<p>If you do not yet know Rust, I recommend for your first project to use pure Rust, and stay far away from the whole FFI topic.</p>
<p>I initially considered using Zig or Odin for this rewrite, but I really did not want to use a pre v1.0 language for an enterprise production codebase (and I anticipated that it would be hard to convince other engineers and managers). Now, I am wondering if the experience would have really been worse than with Rust. Perhaps the Rust model is really at odds with the C model (or with the C++ model for that matter) and there is simply too much friction when using both together.</p>
<p>If I have to undertake a similar effort in the future, I think I would strongly consider going with Zig instead. We'll see. In any case, the next time someone say 'just rewrite it in Rust', point them to this article, and ask them if that changed their mind ;)</p>
<p><a href="/blog"> ⏴ Back to all articles</a></p>
<blockquote id="donate">
<p>If you enjoy what you're reading, you want to support me, and can afford it: <a href="https://paypal.me/philigaultier?country.x=DE&locale.x=en_US">Support me</a>. That allows me to write more cool articles!</p>
</blockquote>
<blockquote>
<p>
This blog is <a href="https://github.com/gaultier/blog">open-source</a>!
If you find a problem, please open a Github issue.
The content of this blog as well as the code snippets are under the <a href="https://en.wikipedia.org/wiki/BSD_licenses#3-clause_license_(%22BSD_License_2.0%22,_%22Revised_BSD_License%22,_%22New_BSD_License%22,_or_%22Modified_BSD_License%22)">BSD-3 License</a> which I also usually use for all my personal projects. It's basically free for every use but you have to mention me as the original author.
</p>
</blockquote>
</div>
</body>
</html>