Dirkjan Ochtman: writing

My first FOSDEM

Published on 2014-08-30 by Dirkjan Ochtman in tech, mozilla

This year, I attended the Free Open Source Developers European Meeting for the first time. For those who don't know, FOSDEM is a big event (5000 people) held every year in Brussels, organized by the community and free to come to (you don't even need to register!). Mozilla happily sponsored my travel and hotel for the 2014 edition, so I wanted to write a few things about my experience. Unfortunately, that took some time; planning for next year has already started!

The schedule was very diverse, and had lots of stuff I found interesting. On Saturday, I mostly hung around the Mozilla devroom, where I was assigned to help out with speaker assistance. It turned out that the extra effort for that wasn't needed most of the time, although I helped out a few times guarding the doors at the start of particularly popular talks (mostly the JavaScript ones). In between, I got to focus on the talks, some of which I really liked.

A moderately full Mozilla devroom

The talk on Firefox for Android provided some nice background about what the Fennec team had been up to, presented in an engaging way. The talk on Persona got quite a bit of interest, though I thought the speaker wasn't great. I helped answering some questions on Persona, and engaged with some of the more interested people after the talk, which was nice. Servo is Mozilla's research browser engine, and it's actually being written in Rust, a very interesting systems language being developed in tandem with Servo. The talk about it was one of the most interesting ones to me, even though I'd heard some of the content before at the Mozilla Summit. The Q&A after the talk was also quite interesting, continuing with a small circle of people outside the room for quite a bit.

Josh Matthews telling us about Servo

At the end of the day, I went to the alpha announcement for Mailpile, which I'd previously seen mentioned on Hacker News. I thought the presentation was great, and the room felt very enthusiastic to me. I even cloned the Mailpile repo during the talk to see if I could get it running; unfortunately there were some issues getting a profile set up. I later tried the second alpha, which exhibited similar failures in the setup processes, so it's been a little disappointing so far. However, I filed an issue and hope things will be better in the beta.

After that, I attended a Gentoo BoF. I've been a Gentoo developer for a few years, but hadn't really met anyone in person before. Since most of my fellow Gentoo devs already know each other or had been hanging around the Distributions devroom all day, it was a bit weird to get in at the end of the day. Fortunately there was a nice round of introductions, so at least I have met some people now. On the other hand, I had to skip the Gentoo dinner for a Mozilla party.

The party was great, of course; it's always nice to get to know Mozillians. Afterwards, four of us went to get a beer on the way back to the hotel, where we had some good discussions about Mozilla's strategy and future. In the end, I always like these conversations in a smaller group the best.

On Sunday, there was no longer any room available to Mozilla and the booth we had was already pretty well-staffed, so I felt free to visit some other talks I wanted to see. I started off early trying to get into the clang talk in the LLVM devroom, but couldn't get in, even at 9 AM! Many of the devroom were full for most of Sunday, which was a pity. On the other hand, being at the door of the Go devroom really early got me a nice seat for the Camlistore talk by Brad Fitzpatrick. Camlistore is an impressive project, and I actually tried to get it running the same day, but the lack of end-user documentation makes it hard to get started (and I'm not a fan of Go) or how to use it fruitfully.

Early in the afternoon we had a small CouchDB community meetup. Benoit Chesneau and I and a few CouchDB fans got together to discuss some things that were going on. I particularly liked talking to our users to hear what problems they were trying to solve with CouchDB. After this I went to the big keysigning session and exchanged verifications with some 80 other hackers, so that my GPG key should be pretty well-connected by now (at least in the FOSS ecosystem).

The final talks I saw were the Python CFFI talk, which was nice but not that interesting since I already had some experience thanks to nnpy (a nanomsg binding), and the satirical NSA keynote from phk, which was fun.

In the end, I had a really good time at FOSDEM. It's a very large event, so I was happy to be able to hang out with the great Mozillians in the devroom and near the booth. Thanks to the Reps program for sponsoring my going there!

Single-source Python 2/3 doctests

Somewhere in 2009, I took over maintenance of CouchDB-Python from Christopher Lenz. While maintenance has slowed down over the years, since the core libraries work well and the CouchDB API has been quite stable, I still feel responsible for the project (I also still use it in a bunch of places). This being a Python project, it always felt like it would have to be ported to Python 3 sooner or later. Since it's working with a fairly deep HTTP API (as in, it uses a large subset of the protocol, with extensive hacking of httplib/http.client), the changes needed in string/bytes handling are quite involved.

My first serious attempt started in November of 2012, as evidenced from some old patches that I have lying around in mq repositories. I picked it back up again about a year later, until I had most of the tests passing, save for one specific category: the doctests. Specifically, the problem I had was with unicode literals (like u'str'). For Python 2.7 doctests, I needed the unicode annotation to pass the test. In Python 3, all strings are unicode; while unicode literals can be used in source code in Python 3.3 and later, the repr() of a string always lacks the unicode annotation. This resulted in lots of test failures like this:

======================================================================
FAIL: client (couchdb)
Doctest: couchdb.client
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.3/doctest.py", line 2154, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for couchdb.client
  File "./couchdb/client.py", line 8, in client

----------------------------------------------------------------------
File "./couchdb/client.py", line 15, in couchdb.client
Failed example:
    doc['type']
Expected:
    u'Person'
Got:
    'Person'
----------------------------------------------------------------------
File "./couchdb/client.py", line 17, in couchdb.client
Failed example:
    doc['name']
Expected:
    u'John Doe'
Got:
    'John Doe'

While these simple cases might have been easy to fix some other way (e.g. by printing the value instead of just asking for the representation), other cases would be significantly harder to fix that way. Here's one example:

----------------------------------------------------------------------
File "./couchdb/mapping.py", line 343, in couchdb.mapping.Document.items
Failed example:
    sorted(post.items())
Expected:
    [('_id', 'foo-bar'), ('author', u'Joe'), ('title', u'Foo bar')]
Got:
    [('_id', 'foo-bar'), ('author', 'Joe'), ('title', 'Foo bar')]

After asking around on the Python 3 porting mailing list, Lennart Regebro (the author of the Porting to Python 3 book) kindly pointed me to the relevant section of his book, but it didn't contain any great suggestions for this particular problem. It took me a few months to get back into it, but I started looking into the doctest APIs yesterday, and managed to figure out a fairly clean solution:

class Py23DocChecker(doctest.OutputChecker):
  def check_output(self, want, got, optionflags):
    if sys.version_info[0] > 2:
      want = re.sub("u'(.*?)'", "'\\1'", want)
      want = re.sub('u"(.*?)"', '"\\1"', want)
    return doctest.OutputChecker.check_output(self, want, got, optionflags)

As it turns out, the doctest API is pretty well-designed, so it allows you to pass in your own OutputChecker object. As its name indicates, this is the bit of code that compares the actual output and the expected output of a given example. By slightly processing the expected value when running on Python 3, we can make sure that actual and expected output match on both versions. Use it like this:

doctest.DocTestSuite(mod, checker=Py23DocChecker())

Fixing these test failures has cleared the way (along with some other fixes) for a Python 3-compatible CouchDB-Python release soon. I hope this will enable other projects to start moving in the direction of 3.x; at the very least, it should significantly lower the barrier for my own projects to start using Python 3.

No Close Buttons

Or, How To Turn Your Firefox userChrome.css Hack Into a Neat Little Restart-Less Add-on, in Five Delightfully Simple Steps.

For as long as I can remember, I've had a small number of tweaks set in my Firefox profile's about:config. One of these was the browser.tabs.closeButtons pref, which I had set to 2. By default, Firefox shows a little close button on the right of every tab, but since I pretty much always use a keyboard shortcut to close tabs, these little buttons aren't that helpful, and they end up obscuring parts of their tab's titles. Setting the value to 2 removes all of the buttons.

In Firefox 31 (to be released to the general public in about 12 weeks), this preference has been removed, leading me to look into other ways of removing the buttons. A commenter on the bug noted the CSS required to remove the buttons again, saying that this could be added to the userChrome.css file in a profile. However, I don't really like that solution, since it would require me to port the fix to every computer I use, and it would be easy for me to lose it. Instead, I wanted to put it in an add-on, which would make it easy for me to install on other computers, in addition to being relatively easy to find. As an added benefit, others can benefit from the same add-on.

The result of this is the No Close Buttons add-on, which I put up on AMO yesterday. It was promptly reviewed by a friendly reviewer from the Dutch community, so that it can be installed without trouble. However, it's currently restricted to Firefox 31 and later, since I figured people on earlier versions wouldn't need it. Because it took me a while to piece together everything for what I thought should be a well-documented process (turning simple chrome CSS hacks into an add-on), I figured I'd document the process here.

First off, create a file called install.rdf:

<?xml version="1.0" encoding="utf-8"?>
<!-- This Source Code Form is subject to the terms of the Mozilla Public
   - License, v. 2.0. If a copy of the MPL was not distributed with this
   - file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<RDF xmlns="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:em="http://www.mozilla.org/2004/em-rdf#">
  <Description about="urn:mozilla:install-manifest">
    <em:id>no-close-buttons@xavamedia.nl</em:id>
    <em:version>0.1</em:version>
    <em:type>2</em:type>
    <em:bootstrap>true</em:bootstrap>
    <em:unpack>false</em:unpack>
    <!-- Firefox -->
    <em:targetApplication>
      <Description>
        <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id>
        <em:minVersion>31.0a1</em:minVersion>
        <em:maxVersion>31.0</em:maxVersion>
      </Description>
    </em:targetApplication>
    <!-- Front End MetaData -->
    <em:name>No Close Buttons</em:name>
    <em:description>Remove close buttons from tabs</em:description>
    <em:creator>Dirkjan Ochtman</em:creator>
  </Description>
</RDF>

Second, add a file named chrome.manifest, with a single line:

content              no-close-buttons        content/

Third, add some JavaScript code to register and unregister the stylesheet to a file named bootstrap.js:

/* This Source Code Form is subject to the terms of the Mozilla Public
 * License, v. 2.0. If a copy of the MPL was not distributed with this
 * file, You can obtain one at http://mozilla.org/MPL/2.0/. */

'use strict';

var sss = Components.classes["@mozilla.org/content/style-sheet-service;1"].getService(Components.interfaces.nsIStyleSheetService);
var ios = Components.classes["@mozilla.org/network/io-service;1"].getService(Components.interfaces.nsIIOService);
var uri = ios.newURI('chrome://no-close-buttons/content/style.css', null, null);

function startup(data, reason) {
    sss.loadAndRegisterSheet(uri, sss.USER_SHEET);
}

function shutdown(data, reason) {
    sss.unregisterSheet(uri, sss.USER_SHEET);
}

Fourth, add your CSS, to the file named in your bootstrap.js. It should go inside the content directory you've already referenced in the manifest.

.tab-close-button { display: none !important; }

Fifth, and finally, if you zip up the four resulting items (install.rdf, chrome.manifest, bootstrap.js and the content directory), rename the resulting file so that its extension is xpi, and drop it into the extensions directory in your Firefox profile, Firefox should prompt you to install your new add-on!

I cobbled together all these bits and pieces by looking at a small add-on by Benjamin Smedberg (who has a number of UI-related add-ons up on AMO), some code from Stack Overflow, and MDN pages on bootstrapped add-ons and chrome registration. I put the result up on GitHub, any feedback is most welcome.

Giving in to Git(Hub)

Published on 2014-03-30 by Dirkjan Ochtman in tech, code

Last month, I moved most of my code from Bitbucket to GitHub:

As a former Mercurial developer, this feels like an admission of defeat. Most of hg's user interface still seems superior to Git's, even if Git was quicker to get the branching model right. The Mercurial code base, in many ways, is a testament to how approachable a Python application can be, and the extension possibilities stemming from writing a few Python functions seem far more attractive than Git's apparent hodge-podge of C, shell and Perl. It's good that people at Mozilla and Facebook are starting to talk more about hg's advantages, though.

While I wanted to learn Git sooner, the lack of usability made me mostly avoid it until about 8 months ago, when I became a CouchDB committer and thus could no longer escape. Two months ago, I also got a new job where Git is the primary VCS, so I've been diving in. Obviously, it's a pretty great VCS, but some aspects of the (command-line) user interface are still baffling to me. This has been written about in plenty of places, so I won't go point-to-point here. And I'll have to admit that many commands are starting to be ingrained in muscle memory, to the point that I sometimes use Git-like commands in places where I use still hg.

However, now that I have basic usage down such that my lack of experience with Git is no longer a limiting factor, the network effect values from Git (and GitHub, specifically) outweigh my usability concerns. The GitHub UX feels more polished (and seems to receive more attention) than Bitbucket's, and makes me quite happy to use it. I also feel that the community on GitHub is quite a bit larger than on Bitbucket, which could make my projects more accessible (see also this account from Eli Bendersky). I've already gathered some stars (mostly for Persona-TOTP, so far) over the past six weeks; I hope that's just the start.

Changing your OS X Mavericks user icon without iPhoto

Published on 2013-10-24 by Dirkjan Ochtman in tech

I wanted to update the user icon/picture for my OS X user (which may include the iCloud/Apple ID picture as well), but it turned out to be harder than I thought. Here's to hoping this post may help others who run into the same problem. tl;dr: use iCloud's web app to upload the new picture for your own Contacts entry.

Update (2013-11-02): on Twitter, both Christopher Lenz and Justin Mayer pointed out that you can just drag and drop an image onto the System Preferences panel. I thought I'd tried that, but apparently not! Still, I wonder if that UI is sufficiently ingrained that discoverability is not important.

Update (2013-11-25): Hugh Hosman, via email, points out that you can also drop an image into /Library/User Images if you have super user privileges.

Like any person who values 0-day upgrades more than their system's stability, I recently upgraded to OS X Mavericks. Going into the Users & Groups preferences panel, double-clicking my current picture provided me with 6 possible options:

  • Defaults: a sample of pictures provided by Apple
  • Recents: contains the current picture, but no others
  • iCloud: is apparently connected to my iCloud Photo Stream
  • Faces: a selection based on the iCloud Photo Stream
  • Camera: take a new picture from my laptop camera
  • Linked: appears to have something to do with my Contacts

In other words, there was no way here to simply link in a JPEG. Apparently, the way to get pictures into the Photo Stream is either through an iOS device (probably through the Camera app) or via Apple's iPhoto or Aperture photo software, neither of which I own (though iPhoto is apparently free for everyone who buys a new machine from now on). I did some Googling, which yielded precisely zero useful results; apparently, using a JPEG was still supported under Mountain Lion, and no one had documented this problem yet. (One of the more promising venues appeared to be the Apple StackExchange site Ask Different.)

But, I figured it out:

  • Go to the web interface for iCloud
  • Go to the Contacts interface
  • Find your own Contacts entry
  • Click "Edit"
  • Click the picture
  • Click "Choose Photo..."

You can now upload the picture. Now, you should be able to go back to the Users & Groups panel and select the uploaded picture from the Linked list of pictures.

Not a great user experience, but at least it works.