Dirkjan Ochtman/writing/2019-01-01T00:00:00+01:00writingRust 2019: Ownership without fear2019-01-01T00:00:00+01:002019-01-01T00:00:00+01:00Dirkjan Ochtmantag:None,2019-01-01:/writing/2019/01/01/rust-2019-ownership-without-fear.html<p>Like <a class="reference external" href="https://dirkjan.ochtman.nl/writing/2018/01/14/rust-in-2018.html">last year</a>, the Rust community team <a class="reference external" href="https://blog.rust-lang.org/2018/12/06/call-for-rust-2019-roadmap-blogposts.html">asked</a> community members to write blog
posts "proposing goals and directions for 2019"; this is mine. Like <a class="reference external" href="https://boats.gitlab.io/blog/post/rust-2019/">withoutboats'
post</a> and <a class="reference external" href="http://smallcultfollowing.com/babysteps/blog/2019/01/07/rust-in-2019-focus-on-sustainability/">Niko Matsakis' take</a>, this post will focus on organizational issues.
As such, the "ownership" from the title refers to a leadership principle, not …</p><p>Like <a class="reference external" href="https://dirkjan.ochtman.nl/writing/2018/01/14/rust-in-2018.html">last year</a>, the Rust community team <a class="reference external" href="https://blog.rust-lang.org/2018/12/06/call-for-rust-2019-roadmap-blogposts.html">asked</a> community members to write blog
posts "proposing goals and directions for 2019"; this is mine. Like <a class="reference external" href="https://boats.gitlab.io/blog/post/rust-2019/">withoutboats'
post</a> and <a class="reference external" href="http://smallcultfollowing.com/babysteps/blog/2019/01/07/rust-in-2019-focus-on-sustainability/">Niko Matsakis' take</a>, this post will focus on organizational issues.
As such, the "ownership" from the title refers to a leadership principle, not the
technical type system concept we all know and love.</p>
<div class="section" id="finance-without-fear">
<h2>Finance without fear</h2>
<p>I believe the Rust project must figure out how accept donations for the benefit
of the Rust language, compiler and ecosystem. Whether this involves a new legal
entity or not, we should use the corporate money on the table where it could enable
the community to move the ecosystem faster than otherwise possible.</p>
<p>Even if other communities have run into trouble with this, the focus of the Rust
project on positive-sum outcomes and careful involvement with the community
should make it possible to come up with proposals that are widely supported.
Things like <a class="reference external" href="https://internals.rust-lang.org/t/homu-queue-woes-and-suggestions-on-how-to-fix-them/8954">improving the state or scope of CI</a>, having <a class="reference external" href="https://internals.rust-lang.org/t/procedural-macros-book/9113">a procedural macro
book written</a> or even building custom tooling might all be good ideas.</p>
<p>So far, the Rust project has preferred existing tools over building our own.
However, I think we run into the limitations of this strategy both in the
example of CI and in the limitations of GitHub issues for some of the more
involved discussions. This is a classic build vs buy discussion -- if we
consider CI and RFC discussions to be a "core business" of the Rust project,
then it might make sense to invest in improving these parts of the project's
process.</p>
<p>(Is "tooling without terror" too terrible a pun?)</p>
</div>
<div class="section" id="foundation-without-fear">
<h2>Foundation without fear</h2>
<p>Setting up a legal entity and governance structure for the Rust project
appears to be a controversial topic in the Rust community. Of course,
it could be a home within an existing foundation: joining Thunderbird
under the umbrella of the Mozilla Foundation, or another entity like
the Software Freedom Conservancy. I would like the increased
transparency and accountability that might come with a more explicit setup
where it concerns how the leadership of the project evolves.</p>
<p>Since it keeps coming up, it would be good to have an RFC about the
foundation idea; to have a thorough conversation on the benefits and
downsides. If the latter end up outweighing the former, the RFC can serve as
negative space documentation (<a class="reference external" href="https://graydon2.dreamwidth.org/263429.html">per Graydon</a>) for everyone thinking about it.</p>
<p>In a different vein, I think <a class="reference external" href="https://www.python.org/dev/peps/pep-8016/">PEP 8016</a> is worth exploring as a carefully
considered alternative we could crib ideas from. As an experiment, we
could have part of the core team be elected by a (large cross-section
of) the community.</p>
</div>
<div class="section" id="asynchronous-alignment">
<h2>Asynchronous alignment</h2>
<p>I think the Rust project could benefit from an increased sense of the direction
of the project. While the language design and library evolution processes
"just" some adjustments to the increased scale at which we're operating,
other parts of how the project executes are not so well-defined, or happen in
synchronous meetings that are opaque to the community. I'm used to open source
projects where decision making happens exclusively in asynchronous venues, and
the current way Rust teams and working groups function is quite different -- with
a notable exception for the way the compiler team <a class="reference external" href="https://internals.rust-lang.org/t/compiler-steering-meeting/8588">documents their work</a>.</p>
<p>The yearly roadmap building effort is one great way of doing this, but the
yearly cycle makes it a very coarse-grained check-in. One suggestion I've been
thinking about is to have a community event (maybe with video?) every release
cycle or two where a group of core team members answers crowd-sourced questions
from the community (along the lines of an AMA or Google's TGIF event).</p>
<p>This could also help cut down on the "Why don't you just" and "Be Heard"
sentiments that I've seen core project team members lament over the past year.
I think those sentiments come from a feeling of powerlessness and being
underinformed. In my mind, more deliberate and continuous communication about
the state of the project could go a long way towards improving on this.</p>
</div>
<div class="section" id="careful-conduct">
<h2>Careful conduct</h2>
<p>Although I wholly support the code of conduct and appreciate many aspects of
the way we interact online, there's some things that I think could be
improved. While I've seen people expressing their personal frustration with
a project decision get quickly moderated, I've also seen users who keep
posting vague forum threads or exert significant stop energy on others'
proposals. I wish we could be more openly critical of decisions (not people)
while at the same time more strict towards net-negative participants.</p>
<p>On a more subtle note, I've felt that core project team members cast the
community as an angry mob at times. Although I realize it is a big job,
I think it's fair to ask project leaders to bring more empathy to the table in
these cases -- to me, that is an important part of leadership. If a significant
part of the community is up in arms about something, there likely is a kernel
of truth to their argument, and the leadership should respond to the sentiment
to prevent it from festering.</p>
<p><strong>Update</strong>: <a class="reference external" href="https://www.reddit.com/r/rust/comments/agdkwm/rust_2019_ownership_without_fear/">interesting discussion on these topics</a> on Reddit.</p>
</div>
<div class="section" id="taking-the-long-view">
<h2>Taking the long view</h2>
<p>To my taste, too much focus in 2018 was spent on releasing the edition.
We could have spent more time addressing technical and organizational debt
and released the edition 3 or 6 months later to little consequence. The Rust
release process has been carefully engineered to optimize for cautious
movement towards long-term goals -- and the edition process, in setting
a fixed scope and an arbitrary hard deadline, subverted a number of these
goals. This caused issues around the release of the website as well as
problems with tools when 1.31 was released.</p>
<p>On the one hand, the cautious, deliberate approach to language design and
compiler development have served us well, and will likely continue to do so.
However, we should not forget that this process functions well in part due
to an imposed feedback cycle that gives enough time and space for people
to experiment before casting the new feature in (stable) stone per
<a class="reference external" href="http://www.hyrumslaw.com/">Hyrum's law</a>. In a similar vein, we cannot expect to get our organizational
changes right on the first try, and so we should not be afraid to experiment
and iterate with the way the community works on the Rust project, rather than
taking a long time to get it just right. This is especially true because the
experiment period allows a wider community to participate and interact with
the proposed change, thus reinforcing community alignment and improving feedback.</p>
<p>In this sense, we might learn from Jeff Bezos when he talks about a bias to action,
saying that we should not spend too much time discussing decisions that
are easy to change after the fact. For Rust, too, we want to remain at
Day 1 for as long as we can -- let's remember the "without stagnation" part.</p>
</div>
<div class="section" id="contemplating-cargo">
<h2>Contemplating Cargo</h2>
<p>On a somewhat more technical note, I hope for more attention given to Cargo.
While Cargo is a great tool in many ways, it also suffers from a number of
usability cliffs where the behavior is hard to predict or understand.
When I tried to contribute, I found that there's also a decent amount of
technical debt, and my attempts to improve this through refactoring were
met with such reluctance that I stopped contributing. From
<a class="reference external" href="https://www.reddit.com/r/rust/comments/ag8vrf/think_twice_before_you_decide_to_contribute_to/">recent reports</a>, it seems I'm not the only one.</p>
<p>Since Cargo is so central to the Rust experience, I think we should not
undervalue it; as one example, the interactions between Cargo and
rustc also influence the perception of those compile times
people keep talking about. I was very happy to see the <a class="reference external" href="https://internals.rust-lang.org/t/cargo-team-changes/8651">leadership changes</a>
and increase of the team's size, and I hope those changes will
lead to further Cargo improvements this year.</p>
</div>
<div class="section" id="unused-inventory-redux">
<h2>Unused inventory, redux</h2>
<p>Last year, I wrote <a class="reference external" href="https://dirkjan.ochtman.nl/writing/2018/01/14/rust-in-2018.html">at some length</a> about the concept of unused inventory.
This is still very much on my mind, and I was happy to see Niko talk about
it in very similar terms. I hope we can improve things here.</p>
</div>
<div class="section" id="closing-thoughts">
<h2>Closing thoughts</h2>
<p>I'm still as bullish on the Rust language as ever. Non-lexical lifetimes
and the module system changes have made an already great language even
better, async/await is going to be great. What I most want from Rust, though,
is that more of the software in my life can enjoy its reliability and
efficiency (in performance vs footprint), and for that we need adoption and
sustainability. While the technical side of the project is working well,
I've been concerned about the functioning of the project's governance and processes.
A number of Rust 2019 posts have addressed similar concerns, which
makes me hopeful for 2019.</p>
</div>
Patreon update #22018-09-05T00:00:00+02:002018-09-05T00:00:00+02:00Dirkjan Ochtmantag:None,2018-09-05:/writing/2018/09/05/patreon-update-2.html<p>A little over 2 months ago I started a <a class="reference external" href="https://www.patreon.com/dochtman">Patreon page</a> to see if I could gain
some financial support for my open source work (with the goal of doing more such work).
I posted an <a class="reference external" href="https://www.patreon.com/posts/first-update-19810370">update</a> on Patreon after 10 days to talk about what I had been up …</p><p>A little over 2 months ago I started a <a class="reference external" href="https://www.patreon.com/dochtman">Patreon page</a> to see if I could gain
some financial support for my open source work (with the goal of doing more such work).
I posted an <a class="reference external" href="https://www.patreon.com/posts/first-update-19810370">update</a> on Patreon after 10 days to talk about what I had been up to,
so I figured it was time for an update.
Instead of posting it to Patreon, I'm posting these updates to my blog from now on;
this gives me full control over the posts and will hopefully boost the update
frequency of the blog a little bit.</p>
<div class="section" id="askama">
<h2>Askama</h2>
<p><a class="reference external" href="https://github.com/djc/askama">Askama</a> is the type-safe Jinja-like Rust template engine I created. Askama 0.7.1
was released in on July 23rd with the following improvements:</p>
<ul class="simple">
<li>Finished the <a class="reference external" href="https://github.com/djc/askama/issues/81">support for multiple template directories</a> that <a class="reference external" href="https://github.com/mashedcode">"mash"</a>
initially contributed (as discussed in the last update) by polishing it up a bit.</li>
<li>Worked with <a class="reference external" href="https://rymc.io/">Ryan McGrath</a> on an implementation of the <a class="reference external" href="https://actix.rs/">actix-web</a>
<tt class="docutils literal">Responder</tt> trait for <tt class="docutils literal">Template</tt> types in <a class="reference external" href="https://github.com/djc/askama/pull/104">#104</a>. (Enabled by <tt class="docutils literal"><span class="pre">with-actix-web</span></tt>.)</li>
<li>Enabled <tt class="docutils literal">Template</tt> derives <a class="reference external" href="https://github.com/djc/askama/issues/101">for unit structs and enums</a>. These previously
caused a panic, but I decided that it was okay to allow them after a bug report.</li>
<li>Improved <a class="reference external" href="https://github.com/djc/askama/issues/102">documentation around filters</a>, specifically user-defined filters.</li>
</ul>
<p>Since that release, <a class="reference external" href="https://github.com/botika">botika</a> has contributed a few fixes around nested macro scopes
(possibly a regression from 0.6), which I will release soon.</p>
</div>
<div class="section" id="quinn">
<h2>Quinn</h2>
<p>Although <a class="reference external" href="https://github.com/djc/askama">Quinn</a> (my nascent QUIC implementation in Rust) saw little progress in
code over the past two months, I still have good hopes for the future.
I talked to <a class="reference external" href="https://github.com/Ralith">Benjamin Saunders</a> about merging his <a class="reference external" href="https://github.com/Ralith/quicr">quicr</a> implementation with
Quinn, and it looks like we will move forward with that. For now we'll likely keep
the Quinn name, but start from Benjamin's code base. Since building and
maintaining a QUIC implementation is a lot of work it probably doesn't make
much sense to have two, and I think merging these projects is for the best.</p>
</div>
<div class="section" id="gentoo">
<h2>Gentoo</h2>
<ul class="simple">
<li>Updated the Rust ebuilds to 1.27.0 and 1.28.0. As discussed in the previous update, these
versions now allow installing the cargo, rustfmt and rls components as built
by the Rust build system (or the binaries in the case of rust-bin). As upstream
makes the distribution more monolithic, this will make it easier to get updates into
the Gentoo repository.</li>
<li>Introduced a new virtual/cargo ebuild to abstract over the cargo builds installed by
dev-lang/rust, dev-lang/rust-bin, and dev-util/cargo.</li>
<li>Updated the CouchDB ebuild to 1.7.2 after <a class="reference external" href="https://blog.couchdb.org/2018/07/10/cve-2018-8007/">a vulnerability was reported</a> in 1.7.1.
This was the last CouchDB release in the 1.x range, and since <a class="reference external" href="http://cve.circl.lu/cve/CVE-2018-11769">another vulnerability</a>
was disclosed, 1.7.2 is known to be vulnerable. I have been dissatisfied with the
direction of the project, so I've finally removed myself as a Gentoo CouchDB maintainer.
No one has stepped up, so I'll start the process
for having CouchDB removed from the Gentoo repository soon.</li>
<li>Updated the ripgrep ebuild to 0.9.</li>
</ul>
</div>
<div class="section" id="abna">
<h2>abna</h2>
<p>In June, I released a tiny Python library called <a class="reference external" href="https://github.com/djc/abna">abna</a> that will log into the
Dutch ABN Amro bank's web interface and download your transactions for you.
I forgot to mention it in the previous update, so will expand on it a little.</p>
<p>It's been a long-standing annoyance for me that automating this process was
impossible. For a long time, the web interface used a hardware token which
relies on debit card access and punching in your PIN code. However, in recent
years they've added a five-digit so-called soft token that enables limited
access to the account, including seeing past account mutations.
In June, I decided to reverse engineer their login process and figured out the
code to support it (spoiler alert: it involves some RSA encryption).</p>
<p>In July, there was a nice PR from <a class="reference external" href="http://www.ivanvasic.com/">Ivan Vasić</a> to allow <a class="reference external" href="https://github.com/djc/abna/pull/1">downloading the
mutations for another account</a> than the login account, which I promptly merged
and released as 0.2 after improving the packaging situation a little more.</p>
</div>
<div class="section" id="assorted-other-work">
<h2>Assorted other work</h2>
<ul class="simple">
<li><a class="reference external" href="https://github.com/djc/rnc2rng">rnc2rng</a> got an issue comment talking about <a class="reference external" href="https://github.com/djc/rnc2rng/issues/3#issuecomment-403371753">converting a schema for SVG</a>.
I went and implemented a number of things from the compact syntax spec that
I hadn't gotten around to and asked the commenter for feedback. I haven't
heard anything back, so perhaps I'll just release what I have.</li>
<li>For a client project, I worked on a GraphQL API server in Rust. In order to
cut down on duplicate dependencies for that project, I submitted pull requests
to update syn and quote for <a class="reference external" href="https://github.com/quodlibetor/diesel-derive-newtype/pull/8">diesel-derive-newtype</a> and <a class="reference external" href="https://github.com/graphql-rust/juniper/pull/231">juniper</a>.</li>
<li><a class="reference external" href="https://github.com/djc/tokio-imap">imap-proto</a> got a pull request to <a class="reference external" href="https://github.com/djc/tokio-imap/pull/25">improve compatibility</a> with a particular
IMAP server which got merged after some back and forth.</li>
<li><a class="reference external" href="https://github.com/djc/template-benchmarks-rs">template-benchmarks-rs</a> <a class="reference external" href="https://github.com/djc/template-benchmarks-rs/pull/7">added benchmarks</a> for <a class="reference external" href="https://github.com/kaj/ructe">ructe</a> contributed by
<a class="reference external" href="https://github.com/bhansconnect">Brendan Hansknecht</a> and <a class="reference external" href="https://github.com/djc/template-benchmarks-rs/pull/9">some further tweaks</a> from <a class="reference external" href="http://www.vincentprouillet.com/">Vincent Prouillet</a>
(author of <a class="reference external" href="https://tera.netlify.com/">Tera</a>). I added charts to the README to replace the table of
numbers (although Vincent then proposed adding the numbers back -- and we did).</li>
<li>I tweeted <a class="reference external" href="https://twitter.com/djco/status/1032556807720501250">a poll on Rust source code order</a> which turned out to be quite
popular. I hope to write a blog post soon to explain my position on the issue
(I think <tt class="docutils literal">main()</tt> should come first).</li>
</ul>
<p>Many thanks to my patrons; I hope this is worthy of your support.</p>
</div>
Science fiction recommendations2018-07-15T00:00:00+02:002018-07-15T00:00:00+02:00Dirkjan Ochtmantag:None,2018-07-15:/writing/2018/07/15/science-fiction-recommendations.html<p>I've been thinking about writing more blog posts again recently.
Since I sometimes write in blog-like form elsewhere,
I thought I might syndicate those posts here,
partly out of a desire to make it easier to find things I've written
(compared to digging through my Twitter feed for links).
As …</p><p>I've been thinking about writing more blog posts again recently.
Since I sometimes write in blog-like form elsewhere,
I thought I might syndicate those posts here,
partly out of a desire to make it easier to find things I've written
(compared to digging through my Twitter feed for links).
As such, here's an <a class="reference external" href="https://news.ycombinator.com/item?id=17536654">edited Hacker News comment</a> from a post asking for
SF recommendations.</p>
<p>Isaac Asimov is one of the classics.
I'm particularly fond of the robot novels
(all four of the Elijah Baley ones especially)
and the entire Foundation series.
Of course, you also can't miss the robot short stories.
The End of Eternity is less famous,
but it was one of the first ones I read and classic time travel fiction.</p>
<p>At some point I found Charles Stross' <a class="reference external" href="http://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando-intro.html">Accelerando</a> online,
where it is freely available along with <a class="reference external" href="https://www.antipope.org/charlie/blog-static/fiction/online-fiction-by-charles-stro.html">a bunch of other stories</a>,
and found everything I've read enjoyably fast-paced.
In particular the Laundry series —
though arguably these cross over from SF into fantasy a bit —
and the Halting State trilogy.
(In recent years, I've bought every Laundry book as soon as it came out.)</p>
<p>In terms of online available work, <a class="reference external" href="https://craphound.com/shop/">Cory Doctorow's work</a> is also great.
Like Stross, his work is more near-term SF, which I like.
It often moves fast and is politically interesting;
sometimes the activism shines through a little too much,
but on the plus side it makes me think.
His latest is Walkaway, which I would recommend.</p>
<p>I got into Neal Stephenson through Cryptonomicon, which is awesome and a little crazy.
Stephenson has a rambly wordy style,
but that lends a kind of depth to his stories —
and some of his books make me laugh out loud (including Cryptonomicon)
which is rare for me.
After that, I got into Anathem, Reamde and Seveneves,
all of which are big and interesting and great reads
(Anathem has a slow start, but it's worth it).
I tried one of the more historic ones (I think it was Quicksilver)
at some point but haven't finished it.
His latest work The Rise and Fall of D.O.D.O. is lighter,
but another fun take at the time travel theme.</p>
<p>I would also recommend William Gibson.
I read the Sprawl trilogy a long time ago,
and it's made a lasting expression — I started re-reading it recently.
I also liked the Blue Ant trilogy and The Peripheral,
but have so far missed the more recent work
(which means I have something to look forward to).</p>
<p>In the one-off category:
The Sparrow by Mary Doria Russell is one of my all-time favorites,
and mixes deep SF with psychology and religion in interesting ways.
There's a sequel, but it's not as good.
I also liked The Time Traveller's Wife by Audrey Niffenegger,
which is also on the softer side.
I read John Scalzi's Redshirts and liked it
(though probably not as much fun if you're not familiar with the Star Trek universe).
It feels similar to Ernest Cline's Ready Player One in some ways.
Daniel Suarez's Daemon is fun and easy to digest and has some interesting ideas,
as does its sequel, Freedom.
I liked Robert J. Sawyer's The Terminal Experiment,
but someone I recommended it too found it too flimsy —
I still think it's fun, if not very deep.
I also bought his Factoring Humanity and liked it okay.</p>
<p>I hope this list is useful to someone.
If you have recommendations for me based on this selection, I'd like to hear them.</p>
Rust in 20182018-01-14T00:00:00+01:002018-01-14T00:00:00+01:00Dirkjan Ochtmantag:None,2018-01-14:/writing/2018/01/14/rust-in-2018.html<p>In a <a class="reference external" href="https://blog.rust-lang.org/2018/01/03/new-years-rust-a-call-for-community-blogposts.html">call for blog posts</a>, the Rust community team asked community members to
write up their vision for what the Rust community should focus on this year.
I've wanted to contribute my thoughts and have been thinking about what to
write ever since. I've been able to benefit from …</p><p>In a <a class="reference external" href="https://blog.rust-lang.org/2018/01/03/new-years-rust-a-call-for-community-blogposts.html">call for blog posts</a>, the Rust community team asked community members to
write up their vision for what the Rust community should focus on this year.
I've wanted to contribute my thoughts and have been thinking about what to
write ever since. I've been able to benefit from the many people who already
posted their thoughts to sharpen my own thinking. I came up with 5 categories:</p>
<ol class="arabic simple">
<li>Unused inventory</li>
<li>Meta-ergonomics</li>
<li>Deep docs</li>
<li>Web development</li>
<li>Paper cuts</li>
</ol>
<p>In rough priority order, these range from high level community and product
thinking to technical things that I'd like to see. Let's dive in.</p>
<div class="section" id="unused-inventory">
<h2>Unused inventory</h2>
<p><a class="reference external" href="https://apenwarr.ca/">Avery Pennarun</a> recently wrote an essay called
<a class="reference external" href="https://apenwarr.ca/log/?m=201712#13">An epic treatise on scheduling, bug tracking and triage</a>;
it's long, but if you're interested in software engineering process,
I think it'll be worth your time. One of the points that really stuck
with me is where he talks about unreleased software as inventory.</p>
<p>Avery discusses how <a class="reference external" href="https://en.wikipedia.org/wiki/Kanban">Kanban</a> was invented in the Japanese car
industry, where minimizing inventory was one of the driving factors.
As Avery explains, this very much applies to shipping software:
unreleased code means you have spent the time to design and implement
the feature, but now you have to pay for maintenance of this code
(in terms of complexity in implementing other bugs or features)
without realizing the value of shipping the actual feature to your customers.
And that's not even discussing the opportunity costs of how you
could have spent the time spent on unreleased feature A to ship released
feature B sooner, increasing the value brought to your customers.</p>
<p>If you doubt the analogy, please go and read Avery's essay,
since I cannot do it justice here. My point is this: <tt class="docutils literal">rustc</tt>
has <strong>a lot</strong> of unused inventory.</p>
<p>So I found myself agreeing with Nick Cameron's <a class="reference external" href="https://www.ncameron.org/blog/rust-2018/">Rust 2018 post</a>,
where he describes his wish for 2018 to be "boring", because we should
just finish up what we've got in the pipeline instead of starting new things.
Pascal Hertleif's <a class="reference external" href="https://deterministic.space/rust-2018.html">take on Rust 2018</a> is even stronger, calling for
consolidation and rapidly reducing the number of in-flight unstable
features. Currently there are 113 in the language and 155 in the library;
see also <a class="reference external" href="https://internals.rust-lang.org/t/stabilizing-apis-in-the-standard-library/5906/9">my analysis</a> of open library tracking issues.</p>
<p>If you think about that in terms of unused inventory: how much
time has the community spent on designing and implementing these features?
How much value, in return, has made it into the stable Rust compiler
that most people use?
How much time has been spent on maintaining these features while we were
shipping other things?
How many of the commits going into the master branch during any given release
cycle actually affect stable Rust users' life six weeks later?
When is an unstable feature actually used so much that it has effectively
prematurely stabilized? For example, if there is agreement that the design
can be improved, are we still willing to break the codegen features Rocket uses?</p>
<p>One challenge here is that Rust is an open source community,
not a hierarchically-driven top-down organization where the leaders can just
tell people what to do. Still, if the community can come together and agree
on priorities, we can focus more on shipping features in stable Rust.
We could adopt a rule that features cannot stay in nightly if no progress
on stabilization is being made for more than 4 cycles. Or agree as a
community that no more than 25 language and 25 library features can be
in-flight at any time.</p>
<p>My other conclusion is that stabilizing features is a bottleneck in our
factory for shipping Rust. So while we work on reducing inventory,
we should at the same time try to increase the capacity of bottlenecks
in the stabilization process.</p>
</div>
<div class="section" id="meta-ergonomics">
<h2>Meta-ergonomics</h2>
<p>I've been thinking about <em>metacognition</em> recently as a nice word for
a useful concept. Similar to how metacognition means the awareness
and analysis of one's own learning or thinking processes, and playing
off of the 2017 year theme of ergonomics, maybe 2018 should be the year of
going meta: <em>meta-ergonomics</em>, where we focus on the process of
improving language ergonomics.</p>
<p>In order to scale the number of people that can help design, implement and
stabilize new features and fix bugs, how can we best connect the
high-level goals to the lower-level implementation process? Can we
highlight the current blockers and accessible "good first bugs" where
mentoring is available?</p>
<p>I subscribed to some issues in the GitHub <a class="reference external" href="https://github.com/rust-lang/rust-roadmap">roadmap issue tracker</a> last
year, but did not find myself deriving a lot of value over the course of
the year. There were good write-ups of what needed to get done, but few
updates over time and little connection to the ongoing implementation effort.
The best way I've found to keep track of the non-lexical lifetime effort,
for example, was just to check out links from This Week in Rust, which
mostly talked about stuff that was completed rather than upcoming
opportunities for contribution.</p>
<p>One important part has already been kicked off by Niko Matsakis: the
<a class="reference external" href="https://internals.rust-lang.org/t/so-you-want-to-hack-on-the-rust-compiler-a-plan-for-a-book/6497">Rust Compiler Book</a> will hopefully become an important resource this year
for helping people hack on the compiler. When I tried my hands at one small
language feature this year, documentation like that was sorely lacking.</p>
</div>
<div class="section" id="deep-docs">
<h2>Deep docs</h2>
<p>As with missing documentation for the compiler,
the other area where Rust can improve next year is documentation.
While <a class="reference external" href="https://doc.rust-lang.org/stable/book/second-edition/">The Rust Programming Language</a> is a great resource for people
just starting with Rust,
I ran into a number of cases where it didn't fulfill my needs
and I had to ask around for help.</p>
<p>The community is wonderful in providing such help,
but at the same time it felt frustrating when there was no
documentation to better describe the language's syntax and semantics:
what edge cases are allowed or not, what parts remain unimplemented,
why certain restrictions are there and what relevant RFCs are in the
pipeline. I'm not sure whether this is intermediate-level
documentation or reference documentation, or maybe those really are
the same thing.</p>
</div>
<div class="section" id="web-development">
<h2>Web development</h2>
<p>The big ticket item here is WebAssembly. I believe that
WebAssembly is about to take off in a big way, particularly once it gets
access to DOM APIs. My personal end goal here is to be able to write
web apps where both the backend and the frontend are written in Rust.
Ideally, the UI would leverage functional reactive programming
(see <a class="reference external" href="https://github.com/DenisKolodin/yew">Yew</a>), so that the app's state lives on the client and the server
just ships state updates as required. (Elm, but in a rustic way.)
Lots of progress was made with WASM support in 2017, but making that
really polished would make Rust a top contender for greenfield WASM
projects (that is, not using legacy C/C++), which seems like an
important use case going forward.</p>
<p>Even if you just want to write Rust on the backend,
the infrastructure is still maturing.
<a class="reference external" href="https://rocket.rs/">Rocket</a> has many cool ideas, but only works on nightly and doesn't yet
support asynchronous programming.
<a class="reference external" href="https://gotham.rs/">Gotham</a> seems to be the next viable option, on stable and
with support for futures, but it seems to be in the very early stages
(starting with the documentation).
For simpler API services, it looks like hyper (hopefully soon with
built-in HTTP 2 support) and serde work well together, although
something with more polish and less boilerplate would be nice.</p>
<p>So far <a class="reference external" href="https://github.com/djc/askama">Askama</a>, my take on templating for Rust, hasn't taken off in
a big way (although I've been very happy with contributions!). I'm not
sure why exactly; I'll keep iterating as I haven't seen any competition
that makes more sense to me. I would like to explore how it can fit in
with functional reactive programming -- if that ends up working out,
it may also draw more people in.</p>
<p>In general, it doesn't feel like Rust <a class="reference external" href="http://www.arewewebyet.org/">is web yet</a>, so I hope this
will improve in 2018.</p>
</div>
<div class="section" id="paper-cuts">
<h2>Paper cuts</h2>
<p>And then there's a list of things I've ran into over time that I'd like to see
some traction on somehow. Note that I do pretty much all of my development
on stable Rust, and that contributes to some of these problems.</p>
<ul class="simple">
<li>When I tried to expose a parser result for <a class="reference external" href="https://github.com/djc/imap-proto">imap-proto</a>, I was surprised
to find out you <a class="reference external" href="https://github.com/rust-lang/rust/pull/31179">cannot access enum variants through type aliases</a>.
I wrote up a <a class="reference external" href="https://internals.rust-lang.org/t/pre-rfc-enum-variants-through-type-aliases/6433">pre-RFC</a> to help fix that; stabilizing that in 2018
seems like a challenge.</li>
<li>In one case I wanted to use <tt class="docutils literal"><span class="pre">Vec::resize_default()</span></tt>, which has
been <a class="reference external" href="https://github.com/rust-lang/rust/issues/41758">waiting for stabilization</a> for about 8 months now without
any signs of progress.</li>
<li>Installing clippy or rustfmt (as a stable user) and keeping them running
takes hard work and troubleshooting time. Updating these tools means I have
to update nightly, and the other way around. On the latest update, <tt class="docutils literal">rustup</tt>
warned me I should delete the separate installations to allow it to
install the rustup-managed ones, but that <a class="reference external" href="https://github.com/rust-lang-nursery/rustup.rs/issues/1305">actually didn't work</a>.</li>
<li>Writing a Tokio network protocol client (<a class="reference external" href="https://github.com/djc/tokio-imap">tokio-imap</a>) took a lot of time,
since the documentation focused much more on writing servers at the
time. It feels like Tokio made little progress in 2017; I hope 2018 will be
better.</li>
<li>The ergonomics around futures are not where they should be. I hope impl
Trait and async/await can make enough progress in 2018 to make this better.</li>
</ul>
</div>
<div class="section" id="conclusion">
<h2>Conclusion</h2>
<p>I deeply believe in Rust; I've been trying to articulate that in another blog post
(still in progress).
I hope 2018 will be another great year for Rust,
and I am eager to participate more in the Rust community over the coming year.</p>
</div>
Rust is a big tent2017-05-19T00:00:00+02:002017-05-19T00:00:00+02:00Dirkjan Ochtmantag:None,2017-05-19:/writing/2017/05/19/rust-is-a-big-tent.html<p><em>Cue images of
<a href="https://brson.github.io/fireflowers/">fireflower</a>-decorated
tents.</em></p><p>Over the last year,
Rust has replaced Python as my favorite programming language.
Recently, the Rust community celebrated the second birthday of Rust 1.0,
and the <a class="reference external" href="https://blog.rust-lang.org/2017/05/15/rust-at-two-years.html">birthday blog post</a> mentioned that 438 people contributed for
the first time to the compiler and standard …</p><p><em>Cue images of
<a href="https://brson.github.io/fireflowers/">fireflower</a>-decorated
tents.</em></p><p>Over the last year,
Rust has replaced Python as my favorite programming language.
Recently, the Rust community celebrated the second birthday of Rust 1.0,
and the <a class="reference external" href="https://blog.rust-lang.org/2017/05/15/rust-at-two-years.html">birthday blog post</a> mentioned that 438 people contributed for
the first time to the compiler and standard library this year.</p>
<p>This led me to wonder how this compares to other modern programming languages.
Specifically, I was wondering about Go and Swift,
languages that are of similar vintage and that compete in the same space
of compiled, statically-typed languages with a performance focus.</p>
<p>One of the concerns I have seen from people about Rust's viability is the size
of its community -- can Mozilla and volunteer contributors evolve Rust at
a fast enough pace? And how does that pace compare to these other languages,
both of which are backed by large corporations?</p>
<p>So I pulled Git repositories for each of these three projects and graphed the
number of non-merge commits in each of the repositories from the first day:</p>
<a class="reference external image-reference" href="/writing/files/lang-commits.png"><img alt="Commits over time" src="/writing/files/lang-commits.png" style="width: 600px;" /></a>
<p>Then, I also graphed the cumulative number of unique authors:</p>
<a class="reference external image-reference" href="/writing/files/lang-authors.png"><img alt="Commits over time" src="/writing/files/lang-authors.png" style="width: 600px;" /></a>
<p>Code and data can be found <a class="reference external" href="https://github.com/djc/modern-lang-analysis">on GitHub</a>.
I manually culled the first 4 commits from the Go repository,
which reported being from 1972, 1974 and 1988;
the first commit that I kept is from March 2, 2008.
Swift started on July 17, 2010 and Rust on June 16 of the same year,
surprisingly close to each other.</p>
<p>Clearly, this is a crude analysis (even ignoring the Excel charts).
Repositories may not all contain the same depth of components
(like compiler, standard library, documentation),
and commit sizes could substantially differ due to differing project cultures.
Still, two broad patterns are apparent:</p>
<ul class="simple">
<li>Rust has way more unique contributors than the other languages</li>
<li>Rust gets many more commits than Go, but Swift is moving faster</li>
</ul>
<p>As a result, I'm confirmed in my optimism about Rust's future.</p>