Dirkjan Ochtman: writing

September on GitHub (2024)

Published on 2024-10-18 by Dirkjan Ochtman in tech, code, rust

Since leaving my job at the end of August, I figured I would try to write up a report of all the open source stuff I worked on in September. Herewith a not quite complete account of things I was involved in.

rustls

rustls is a pure Rust implementation of the TLS protocol.

Substantial effort last month went into migrating support for PEM decoding from the rustls-pemfile crate (backed by the base64 crate) to the rustls-pki-types crate. While the new decoder is slightly slower, it avoids timing side channel attacks (only used for PEM items containing private keys). We iterated a number of times on making the new API easy to use. Adopting the new APIs should shrink dependency graphs by getting rid of both rustls-pemfile (which got a semver-compatible bump to use the new implementation) and the base64 crate.

We went through a number of iterations reviewing a PR from Bart Dubbeldam and Adolfo Ochagavía adding support for RFC 7250 raw public keys.

ComplexSpaces added deployment considerations to the rustls-platform-verifier README. We now recommend (almost) everyone use the platform verifier for their server certificate verification needs.

Joe has been doing interesting work to improve performance:

I contributed some smaller improvements last month:

Hickory DNS

Hickory DNS is a project to build a comprehensive suite of Rust libraries to build DNS services on top of. Because the project is nearing a (fairly large) feature release, I've been trying to make a number of improvements to the project, cleaning up the API and moving code around to reduce complexity. Here are some of the things I worked on:

  • I moved the RuntimeProvider trait from hickory-resolver down into hickory-proto. This abstraction was created to allow the resolver to abstract over different async runtimes (and their I/O traits and types), but it did not actually depend on the resolver infrastructure itself. Meanwhile, at the lower -proto level we had a somewhat parallel set of abstractions, so I set out to move the RuntimeProvider into -proto and try to make it the focal point of abstracting over runtimes and I/O. This work was split over two PRs, one for UDP and one for TCP. I should probably review whether there is potential for further simplification in the other protocols (H2, QUIC, H3).
  • Hickory DNS depends on the resolve-conf crate in order to find the system DNS configuration (on Unix systems). Unfortunately the resolv-conf crate has been unmaintained for a while. While attempts to contact the maintainer didn't lead to any results, Ferrous team member Andrei Listochkin mentioned that he knew the original maintainer and could talk to them via Telegram. Soon, the repository and crate ownership had been transferred to us, so that we can start fixing some of the outstanding issues soon.
  • After reviewing a PR to allow customization of the DNS over HTTP URL path I noticed that the configuration for the main Hickory DNS binary crate was distributed over the -server library crate and the hickory-dns binary crate. Some of this didn't really make sense, so I submitted a collection of config tweaks to improve on the situation.
  • While reviewing another PR, I noticed that the hickory-server crates had a bunch of instances of Arc<Box<dyn AuthorityObject>>, which seemed unnecessarily complex. I cleaned these up so that we use Arc<dyn AuthorityObject> instead.
  • After we noticed that some tests against the Quad9 name servers kept causing issues in CI, I decided to kill the quad9 tests. We still run tests against Google and Cloudflare to help validate our code.
  • A user reported that the Cargo features for the server were pulling in OpenSSL even when that should be necessary. I submitted a PR to clean up server features which improved the situation.
  • While working on KumoMTA (see below), I noticed that RecordSet::new() took a &Name argument only to clone it. I changed it to take a Name directly so the caller can hand over ownership where possible.
  • Made some minor recursor tweaks.

I reviewed 25 PRs Hickory DNS last month, here are the more notable ones:

rustup

rustup is the tool most Rust developers to use manage their compiler toolchains; I joined the team last year after some of the previous members had left, reducing the available capacity. We're currently gearing up for the beta of rustup 1.28 which contains some larger changes, including:

  • No implicit toolchain installs
  • Default to rustls with the rustls-platform-verifier for downloads
  • Improved logging based on tracing-rs

In September I recruited Chris to the team. He had already been helping out a bunch with the Windows parts of rustup in particular, and we decided it made sense to ask him to join the team, which he happily agreed to.

Notable changes from last month:

  • We finally merged a contributed PR to replace winreg with windows registry, after we coordinated with the windows-rs team and Chris on the best way forward. This included scrapping some unsafe code from the rustup code base because we can now build on better upstream abstractions.
  • Improve the logging setup to better integrate tracing.

pyrtls

I launched a new project called pyrtls last month. pyrtls provides Python bindings to rustls, seeking to expand access to rustls' great performance and security track record as an alternative to the OpenSSL-based ssl module that is a part of the standard library.

I started working on this intermittently back in 2021 and finally decided to write up some documentation and "ship" it by announcing the project on social media. At the end of the month, it earned a mention in last month's issue of the Feisty Duck Cryptography and Security Newsletter. I was happy to receive my first contributed PR adding a typing stub a few days later.

Although the project has managed to earn 70 stars over the past 4 weeks, I've been a little disappointed with the lack of feedback so far. Given that I don't have a use case for this project myself (since I don't really use Python anymore), I probably won't spend much more time on it until there is some uptake.

KumoMTA

I've been doing some consulting work for KumoMTA, a startup building a mail transfer agent (MTA) for enterprise senders. I had connected with their cofounder Wez (of wezterm fame) through issues against rustls and Hickory (both of which KumoMTA uses to build their MTA).

  • My first issue was to build a Redis rate limiting implementation that doesn't use the redis-cell Redis extension. This integrated a Lua script-based implementation of the generic cell rate algorithm which can be used in deployments where custom Redis extensions are not viable.
  • Then, I started working on an implementation of the Sender Policy Framework (SPF) standard. This will be used in KumoMTA to help detect spam. Wez had already started on an implementation, so I picked up from where he had left off and started implementing the core protocol.
  • I noticed that the KumoMTA codebase was using both lazy-static and once_cell, so I decided to spend a little time (after getting approval from Wez, of course) cleaning this up and migrating the entire project to the newly available OnceLock and LazyLock types stabilized in 1.70.0 and 1.80.0.

Quinn

Quinn is the most popular Rust implementation of the QUIC transport protocol. Some interesting PRs I reviewed last month:

I also submitted some PRs of my own:

Askama

Askama is a compile-time template engine using a Jinja-like syntax.

Reviewed some changes to:

I also:

  • Improved the error in case a Rust operator is used as a delimiter in a custom syntax, in response to an issue report.
  • Applied clippy suggestions from Rust 1.81 and fixed dependencies after a new version of Axum was published that uses tower 0.5.

I should really reserve some time to finish the 0.13 release.

tokio

instant-acme

instant-acme is a RFC 8555 client for provisioning TLS certificates.

Reviewed a contributed PR to avoid bad nonce errors which the Pebble test CA generates to exercise client code. As suggested by the spec, we now retry requests that trigger a bad nonce error a few times before giving up.

On sustainability

Although I have been reaching out to organizations that I'd hoped would be interested in funding this kind of work across the Rust ecosystem, so far I have not succeeded in finding any. Although I'm very grateful to my current sponsors, I'd like to find some companies that can help make my work more sustainable. Happy to get on a call to discuss how I could help out your team!

Many thanks to these sponsors (5 USD/month or more):

  • astral-sh
  • codecov
  • thomaseizinger
  • stepfunc
  • MJDSys
  • repi
  • sourcegraph
  • eightseventhreethree
  • paolobarbolini