Dirkjan Ochtman: writing

Unused inventory

Avery Pennarun recently wrote an essay called An epic treatise on scheduling, bug tracking and triage; it's long, but if you're interested in software engineering process, I think it'll be worth your time. One of the points that really stuck with me is where he talks about unreleased software as inventory.

Avery discusses how Kanban was invented in the Japanese car industry, where minimizing inventory was one of the driving factors. As Avery explains, this very much applies to shipping software: unreleased code means you have spent the time to design and implement the feature, but now you have to pay for maintenance of this code (in terms of complexity in implementing other bugs or features) without realizing the value of shipping the actual feature to your customers. And that's not even discussing the opportunity costs of how you could have spent the time spent on unreleased feature A to ship released feature B sooner, increasing the value brought to your customers.

If you doubt the analogy, please go and read Avery's essay, since I cannot do it justice here. My point is this: rustc has a lot of unused inventory.

So I found myself agreeing with Nick Cameron's Rust 2018 post, where he describes his wish for 2018 to be "boring", because we should just finish up what we've got in the pipeline instead of starting new things. Pascal Hertleif's take on Rust 2018 is even stronger, calling for consolidation and rapidly reducing the number of in-flight unstable features. Currently there are 113 in the language and 155 in the library; see also my analysis of open library tracking issues.

If you think about that in terms of unused inventory: how much time has the community spent on designing and implementing these features? How much value, in return, has made it into the stable Rust compiler that most people use? How much time has been spent on maintaining these features while we were shipping other things? How many of the commits going into the master branch during any given release cycle actually affect stable Rust users' life six weeks later? When is an unstable feature actually used so much that it has effectively prematurely stabilized? For example, if there is agreement that the design can be improved, are we still willing to break the codegen features Rocket uses?

One challenge here is that Rust is an open source community, not a hierarchically-driven top-down organization where the leaders can just tell people what to do. Still, if the community can come together and agree on priorities, we can focus more on shipping features in stable Rust. We could adopt a rule that features cannot stay in nightly if no progress on stabilization is being made for more than 4 cycles. Or agree as a community that no more than 25 language and 25 library features can be in-flight at any time.

My other conclusion is that stabilizing features is a bottleneck in our factory for shipping Rust. So while we work on reducing inventory, we should at the same time try to increase the capacity of bottlenecks in the stabilization process.

Meta-ergonomics

I've been thinking about metacognition recently as a nice word for a useful concept. Similar to how metacognition means the awareness and analysis of one's own learning or thinking processes, and playing off of the 2017 year theme of ergonomics, maybe 2018 should be the year of going meta: meta-ergonomics, where we focus on the process of improving language ergonomics.

In order to scale the number of people that can help design, implement and stabilize new features and fix bugs, how can we best connect the high-level goals to the lower-level implementation process? Can we highlight the current blockers and accessible "good first bugs" where mentoring is available?

I subscribed to some issues in the GitHub roadmap issue tracker last year, but did not find myself deriving a lot of value over the course of the year. There were good write-ups of what needed to get done, but few updates over time and little connection to the ongoing implementation effort. The best way I've found to keep track of the non-lexical lifetime effort, for example, was just to check out links from This Week in Rust, which mostly talked about stuff that was completed rather than upcoming opportunities for contribution.

One important part has already been kicked off by Niko Matsakis: the Rust Compiler Book will hopefully become an important resource this year for helping people hack on the compiler. When I tried my hands at one small language feature this year, documentation like that was sorely lacking.

Deep docs

As with missing documentation for the compiler, the other area where Rust can improve next year is documentation. While The Rust Programming Language is a great resource for people just starting with Rust, I ran into a number of cases where it didn't fulfill my needs and I had to ask around for help.

The community is wonderful in providing such help, but at the same time it felt frustrating when there was no documentation to better describe the language's syntax and semantics: what edge cases are allowed or not, what parts remain unimplemented, why certain restrictions are there and what relevant RFCs are in the pipeline. I'm not sure whether this is intermediate-level documentation or reference documentation, or maybe those really are the same thing.

Web development

The big ticket item here is WebAssembly. I believe that WebAssembly is about to take off in a big way, particularly once it gets access to DOM APIs. My personal end goal here is to be able to write web apps where both the backend and the frontend are written in Rust. Ideally, the UI would leverage functional reactive programming (see Yew), so that the app's state lives on the client and the server just ships state updates as required. (Elm, but in a rustic way.) Lots of progress was made with WASM support in 2017, but making that really polished would make Rust a top contender for greenfield WASM projects (that is, not using legacy C/C++), which seems like an important use case going forward.

Even if you just want to write Rust on the backend, the infrastructure is still maturing. Rocket has many cool ideas, but only works on nightly and doesn't yet support asynchronous programming. Gotham seems to be the next viable option, on stable and with support for futures, but it seems to be in the very early stages (starting with the documentation). For simpler API services, it looks like hyper (hopefully soon with built-in HTTP 2 support) and serde work well together, although something with more polish and less boilerplate would be nice.

So far Askama, my take on templating for Rust, hasn't taken off in a big way (although I've been very happy with contributions!). I'm not sure why exactly; I'll keep iterating as I haven't seen any competition that makes more sense to me. I would like to explore how it can fit in with functional reactive programming -- if that ends up working out, it may also draw more people in.

In general, it doesn't feel like Rust is web yet, so I hope this will improve in 2018.

Paper cuts

And then there's a list of things I've ran into over time that I'd like to see some traction on somehow. Note that I do pretty much all of my development on stable Rust, and that contributes to some of these problems.

• When I tried to expose a parser result for imap-proto, I was surprised to find out you cannot access enum variants through type aliases. I wrote up a pre-RFC to help fix that; stabilizing that in 2018 seems like a challenge.
• In one case I wanted to use Vec::resize_default(), which has been waiting for stabilization for about 8 months now without any signs of progress.
• Installing clippy or rustfmt (as a stable user) and keeping them running takes hard work and troubleshooting time. Updating these tools means I have to update nightly, and the other way around. On the latest update, rustup warned me I should delete the separate installations to allow it to install the rustup-managed ones, but that actually didn't work.
• Writing a Tokio network protocol client (tokio-imap) took a lot of time, since the documentation focused much more on writing servers at the time. It feels like Tokio made little progress in 2017; I hope 2018 will be better.
• The ergonomics around futures are not where they should be. I hope impl Trait and async/await can make enough progress in 2018 to make this better.

Conclusion

I deeply believe in Rust; I've been trying to articulate that in another blog post (still in progress). I hope 2018 will be another great year for Rust, and I am eager to participate more in the Rust community over the coming year.