Looking Back on Review Board

Just over 3.5 years ago, David Trowbridge and I spent some time discussing the annoyances of the typical patch submission and code review processes in the open source projects we participated in and at companies, and decided to play with some ideas for improving this. At the time, we knew very little about what we intended to do. We had a name for it pretty early on, but that was about all we had. We didn’t even know whether we’d get past an early prototyping stage. But here it is, over 3 years later, and we have the leading open source code review tool with an active support and development community, hundreds of companies using it, and exciting new innovations for aiding in the code review process.

I was thinking a few days ago about how far we’ve come and some of the decisions we made along the way. I went digging through our commit history in order to relive some of the past of our little project. Since so few people were even aware of Review Board’s existence at the time, I thought I’d share some of our history with you. Particularly the interesting and funny bits.

“Add the reviewboard.”

Commit #1. The very first thing we put in our Subversion tree on September 27, 2006. I don’t even remember what was in this change now. We transitioned to Git last year and this commit is now just plain empty. Maybe it was jut the directory structure? Who can say.

Early on, we didn’t refer to “Review Board” as a proper name. It was generally “the reviewboard” or something similar. The codebase was young. We didn’t actually do code review on the project at this point (and it shows!). The first few months are littered with odd or nonsensical commit messages, small breakages, and bad decisions.

A few of my favorite commit messages are:

  • “I suck. Make submitting of reviews.”
  • “Don’t stuff the list of files in the bug list. It’s impolite.”
  • “Avoid failing out with Christian’s wacko form”
  • “Gum.”
  • “Holy apple pancakes. It worked!”
  • “I suck… The array was empty… The tests never had a chance to fail. :(“
  • “‘This is a summary’ sucks. Now we use fortune for the summary, description, and testing done. ‘You’re ugly and your mother dresses you funny.'”
  • “Unbreak things before ChipX86 notices”
  • “I’m just… garhgh”

Nowadays, our commit messages look nothing like that, but that’s the fun of a new project. You get to go commit-crazy while you try to figure out what you’re building.

Dashboard, quips and fortunes

The UI of old looked quite different than the UI of today.

We had a dashboard from the very beginning (before the review request pages, even) but it wasn’t anything like the dashboard we had today. It was a simple page with a table containing all outgoing review requests and a table containing incoming review requests. But it also had one more thing: quips.

The beginning of quips functionality was being built. Quips are just little random quotes that are inserted in the UI. I think the plan was to put quips on certain pages, making Review Board a little more fun. We were using them in the dashboard for empty lists, with variations all saying something about the dashboard being empty. Quips are a neat feature that just never survived the early days of development.

Fortunes are similar. On Linux/Unix systems, there’s a little program called “fortune” that just displays a random quote. Since we at first had to test review request functionality without actually having a repository backend of any sort, and we didn’t want to input all the information each time, we just used fortune to generate the summary, description and testing done text. This made for some really funny review requests early on, but this is of course something that had no reason to survive initial development.

Sometimes we would create a bunch of review requests just to see what kind of quotes we’d get. 🙂

Multiple repositories? Almost didn’t happen.

One of the really critical parts of Review Board today is the ability to talk to a variety of different types of repositories in one instance. But, it turns out, this almost didn’t happen.

The initial goals were not that ambitious. Review Board talked to one repository per instance. Everything was basically hard-coded with one repository in mind. That type of repository, as well as its information, was customizable. You just couldn’t have more than one. At the time, this wasn’t a problem, but it didn’t take long until we had a need to talk to two repositories.

We discussed this and at first decided that if we needed to talk to two repositories, we could just set up two instances. It would have been a lot of work to update it for multiple repositories, after all. And really, this was a small project. Who would really need more than a couple repositories? This started to nag at me, though, and so I spent a couple nights rewriting all of the code as an experiment. It ended up working pretty nicely, and we were able to ditch the multi-instance model.

The importance of rewards

It’s always nice to have a little reward for milestones. Developers sometimes compete over cool bug numbers, revisions, etc. Initially, we were going to use quips to add some fun to the site, but we ended up settling on our current trophy system.

One of our first Review Board instances started to approach review request #1000, which was a huge milestone for us. I decided to commemorate the event by staying up and quickly hacking in a hidden feature for showing a trophy for review request #1000. The way we implemented it, you’d see the first ever trophy at 1,000, and from there you’d see it at every milestone number (1,000, 2,000, 3,000, 10,000, etc.). I didn’t want to stop there, though, so I added support for a second type of trophy, one that has confused people with its appearance to this day. Mission complete.

Of course, when we updated the server and someone finally hit 1000, it triggered a bug in the new trophy code and broke his review request. Oh well, I tried.

Diff viewers are hard

If I could pick one point during the whole history of Review Board where I was ready to completely give up, it would be during the creation of our diff viewer. All three diff viewers.

See, the first diff viewer was a complete and total hack. We generated a side-by-side diff using the diff tools and just parsed the output, basically generating a table of that. It was ugly, though, and limiting. It also caused problems where text on a row would either be truncated or would break the parser. I spent a long time working on this before I totally gave up and went on to try a new approach.

My second approach was closer to what we have today, but also limiting and very, very buggy. We were using Python’s built-in diff generation module, which implements a basic diff algorithm. It gave us insert and delete information, but not replace information. We had to hack that in ourselves, and it was really a hack. Try taking a bunch of inserts and deletes and find out which of those are really changed lines. No, really, try it. It’s harder than you think, and it’ll often be wrong.

Still, we stuck with this for a long time. It was slow, buggy, and didn’t generate the sort of output people expected from diff tools. Most people see diffs from GNU Diff, which implements the Meyers Diff Algorithm (with a few additions and tweaks). These Meyers diffs are much nicer to view than what Python gave us. Another problem we hit was that we didn’t have real line number information, so we had to output fake line numbers. They weren’t really line numbers so much as row numbers in the table. Ugh. Even getting this far was really hard and frustrating, and the result still wasn’t good.

Attempt #3. I decided to build our own diff parser and generator from scratch. What a project. I knew nothing about diff generation and hardly knew where to start. I spent probably a good month or so just trying to work on this new diff code, and was so close to giving up so many times. It ended up being completely worth it, though, as we ended up with a very nice, extensible diff parser.

Without that third attempt, we’d be in the stone age. Review Board would not be as nice to use. We wouldn’t have inter-line diffs (where we highlight what changed in a replace line), syntax highlighting, move detection (coming in 1.5), or function/class headers (where we show which function/class the part of the diff is in — also coming in 1.5).

What else…

Well, there’s a lot more I could talk about. Our initial attempts at JavaScript code for the UI, our trials and challenges with database migration, or our early problems storing diffs with different encodings in databases. This is getting long, though, so I’ll cover these in another post on lessons learned.

iPhone/iPod – The new standard mobile platform?

I’ve been living with a very basic Sanyo phone for many years now, and decided to finally upgrade. My main requirement was a real web browser, and the iPhone fulfilled that nicely. Besides, I wanted to be cool and get all the girls, and what better way to do that than to carry around an iPhone?

I’ll talk in another post about what I think of the device, but one thing struck me as I begun to play with this phone. Apple may have just set themselves in a position to dominate the mobile application market. If they choose to.

Now hear me out. Companies such as Sharp and Nokia have worked hard on creating devices (the Zaurus and 770/N800 respectively) based on open source software in the hopes of creating a developer base. While this worked in both cases to varying degrees (lots of third party apps for the Zaurus and lots of great GNOME/Gtk+ improvements and apps for the Nokia devices), they’ve both had problems in terms of market share. You’re just not going to get every teenager, every student and every businessman wanting one of them.

I love my Zauruses, my Nokia 770 and my N800, but having tried to explain to people in the past what makes them so cool is difficult. They kind of get it with the N800, that they can browse the web and run applications and all that. Most people, though, are still so tethered to the computer that they don’t know why they’d care.

But an iPod? iPhone? People care about that. It’s trendy, it’s sexy, it plays their music and videos. Yes, the N800 does that too, but the out of the box experience is very different, and well, the kids just want an iPod. My little 7 year old sister has no idea what an “mp3 player is” but knows damn well what an iPod is.

Web development

Now, where Apple did well was to bundle both the iPhone and iPod with Safari and to put up guidelines for how best to optimize your websites for the iPhone/iPod form factor and visual style. Since then, many sites have provided optimized versions that work quite well, including Amazon, Meebo, NetVibes, Facebook, and Google Reader. And this all turns out to be quite easy to do, especially with Joe Hewitt’s iUI JavaScript framework for the iPhone.

So, we now have a de facto standard for miniaturized AJAXy web UIs for mobile devices, do we not? I don’t see anybody else developing for other mobile devices en mass in such a way that allows all this fancy web 2.0 stuff. What’s great is that aside from the CSS 3.0 support needed for some of the look and feel of these sites, and the viewport meta tag, they’ll work in any browser, mobile or otherwise.

Native applications

The one thing that these other devices do have over the iPhone/iPod is that you can actually develop applications for them. When the iPhone was announced, we were told that they would not be opening up support for third party applications. Of course, it didn’t take long for people to figure out how to jailbreak the device and install custom applications.

Now anybody with an iPhone can easily put their iPhone in a state where they can install custom applications on it. The installer that ships with the modern jailbreaking software is, from a user’s point of view, surprisingly nice and easy to use. It even handles upgrades for you effortlessly.

The pieces are now together for developers to create applications and for users to quickly find them and install them. The only gotcha is that this is not officially sanctioned by Apple, and we may find that this will break in future firmware updates.

If Apple was to reverse their stance and provide an open framework for developing applications for the iPhone and iPod, they would make these devices much more tempting to a wide variety of people. The gaps left by Apple (proper Exchange support, calendar synchronization, document publishing, instant messaging, etc.) would be provided by third party developers and downloaded by possible millions, legally. New companies would form to develop applications. Existing companies would port applications over. Life would be good.

Why didn’t Apple do this from the beginning? Maybe they’re just against a community of developers forming around this product, but I have a hard time believing that. Maybe it’s pressure from AT&T, but you can develop for other phones.

My hunch is that they just didn’t have enough time to do it right, given how much work it was just to get the thing out the door in the first place. If so, perhaps we’ll see official approval for third party application development someday.

Foo Camp 2006

This year, I was one of the lucky few who received an invitation to O’Reilly’s Foo Camp. Like many people, I had heard just how cool this exclusive get-together is, but I was nowhere near prepared for how mind-blowingly awesome the Foo Camp experience can be.

I arrived in Sebastopol on Friday afternoon and began to unpack my tent. The tent I bought ended up being bigger than I expected I think I grabbed the wrong one, but it worked out fine given that I was sharing it with Alex Graveley. Alex wasn’t there yet, though, so I started to unpack it myself, which ended up being a bit difficult… But then a very kind and helpful Mark Shuttleworth walked by and offered a hand. Before long, the tent was set up and I was ready for the fun.

There were only a handful of people there at this point, so I decided to walk around and see the place. One of the rooms had this interesting table with a map on it. A projector was mounted above, and people were around the table, turning and tilting. It was quite cool. Turn the table and it zooms in and out. Tilt it and the map moves. It was the most natural way of viewing maps that I’ve ever used. Quite addicting in a way.

Outside was this kind of round spinnable art thing. It was like the table, but had this colored fluid sand-like substance inside. It was held up at an angle, and when you spun it, the sand-like stuff made some cool patterns. That was also fun to play with. 🙂

Before long, most of the Foo Camp guys were there, and we gathered under a tent to introduce ourselves and learn what was in store for us. It was amazing to see the people who were there. The creators of Digg, del.icio.us, LiveJournal, Second Life, 43things, Amazon, Meebo, Django, Chumby, people from Google, Microsoft, Intel, Yahoo… The list goes on and on.

I went to several sessions. There was one on the future of IM, another on robots, one on Dr. Who vs. Snakes on a Plane, another on Chumby hacking. I think I went to another one or two, but I can’t remember right now. They were all pretty interesting, and there were many more I didn’t have time to go to.

As mentioned above, one of the session talks was about Chumby. Chumby is a cute, soft, squishy Linux computer that is designed to sit next to your couch or your bed or whatever and stream flash content to you. The software and hardware are all open source, and there’s people modifying the casing of the Chumby and sticking it in teddy bears, Teletubbies, Hello Kitties, etc. It’s currently very pre-alpha (both the hardware and the software), but we all knew that. The Foo campers received free Chumbies. The public release will be in like 6 months or so, and should by then have more features and be a bit more stable. Still, it’s an interesting concept, and I hope they come up with a killer app for it. With an intended price tag of $150, it could be a good purchase with the right target audience and apps.

At nights, we would play Werewolf. This is pretty much a Foo Camp tradition, and is played well into the night. I was a werewolf only once, but we kicked ass that one time 🙂

Let’s see, what else was there… A Google plane flew overhead and took pictures, which will go on Google Maps at a 2 inch resolution. There were flame-throwing robots, a clown, lots of good food, new friends, a reverse scavenger hunt… Probably more things than I can list. It is the best event I have ever gone to, and I hope to be invited back next year. To whoever it was that put me on the invite list, thank you. This was an opportunity I will never forget.

(More pictures available in my gallery and tagged on Flickr.)

Gallery Virtual Appliance

Virtual machines used to be all about managing your data center or server consolidation or running applications that only run on some other operating system. Lately, virtual machines have been gaining momentum in a new area: software distribution.

This is actually quite powerful. Instead of a giving a user a complex set of instructions for installing some application or web service, and telling them what they need to install beforehand and what distros or operating systems it works on, the distributor or project can instead provide a bare-bones virtual machine containing the application or service pre-configured, and users can simply download it and power it on. It’s a great way of previewing applications or even running them day-to-day (depending on the application).

There’s a wonderful service out there called Gallery. It’s a web-based photo gallery that users can install on their server in order to share photos with the world. There’s a number of modules available. Many programs work with it. You can do things like order prints through the web. Great program.

And it’s now available in a virtual appliance! They have a description and instructions for setting it up, and I’ve been informed that they plan to keep it updated with each new release. It contains both Gallery 1 and Gallery 2, giving users a chance to see the differences between the versions and decide which suits them best. It can be used in VMware Player, Server, Workstation, and probably ESX (I haven’t tried). It’s downloadable from their website or from VMTN. You can see h0bbel’s blog post for more information on the appliance.

I think this is awesome and I hope it works out well for them and for the users. I’d love to see more projects go this route, and with any luck, Gallery will have set a precedent in the web services world. And for the developers creating these appliances, please feel free to let us know what VMware could do to make your lives easier. We welcome feedback, and you can send it directly to me.

Lost my hacking mojo

I’ve been trying to get out of this funk I’ve been in the past few weeks. It’s been far too hard to just sit down and code outside of work. I can’t even put together a release. A large part of this is due to the amount of work I’ve been doing for VMware as of late on VMware Server. I think another part of it is that I recently finished up releases of Galago, libnotify, notification-daemon, and Leaftag and

I know this is temporary, but it’s frustrating because there really is a lot of cool stuff I’m looking forward to working on. Some may say to enjoy it, take a break, play some games. Problem is, I don’t even feel like doing that! 😛 Perhaps after work calms down and I’ve spent long enough doing nothing at home, I’ll have more motivation to code.

How do other people usually deal with this?

Alpha channels and release schedules

  • GTK+ and semi-transparent windows

    Mike Hearn wrote a blog entry on writing GTK+ applications that provide semi-transparent Cairo-rendered windows. He suggests a SexyWindow class for libsexy, which actually fits in with some of my plans nicely. More on this… someday.

  • Galago 0.5.0.. Almost

    Galago 0.5.0 is about to be released. I’ve said this for a while, but it’s actually happening now. The only thing left is to get the GalagoGtk# bindings out, but I’ve ran into a problem… I want to call the namespace Galago.Gtk, but then the GAPI-generated code tries to use Gtk.Widget and such, which causes a lookup in Galago.Gtk. I don’t know how to fix this, and may have to go back to the GalagoGtk namespace. Any suggestions?

  • Thanks Federico

    I’m somewhat borrowing Federico’s blog entry format on a trial basis for some posts. I’ve grown to like it. Helps to stay organized without being too verbose.

Spice things up in your relationship with libsexy v0.1.6

libsexy v0.1.6 has just been released, along with libsexymm and sexy-python (mono bindings coming soon). It contains two new widgets, SexyTreeView (a GtkTreeView subclass with support for per-cell tooltips) and SexyTooltip (a tooltip that can have widgets packed into it). It also fixes a few licensing inconsistencies (the header files on a couple files were incorrect) and some bugs.

If you’re using the new notification-daemon, it is advised that you update your copy of libsexy in order to fix a minor visual glitch.