Syntax Highlighting in the Diff Viewer

I was up late last night experimenting with a new feature for Review Board. The idea was to introduce syntax highlighting for the code in the diff viewer through use of the Pygments library. Actually getting syntax highlighting to work was pretty easy, and before long I had it working. The trick, it turned out, was to get it working with our interline diff support.

Pygments generates a nice HTML-formatted string with span tags, but we need to insert our own span tags for the highlighting. This was a pain, since you need to make sure you place them at the correct locations, keeping in mind the existing span tags and entities. The code also needed to avoid messing up the nesting of tags. After working with it a bit, I had that working as well, and now the code is up for review. In a couple of days, Review Board will have nice syntax-highlighted diff output!

I have some new improvements in the works for the diff algorithm that should result in much better diffs (especially when moving functions around) and improved UI for that case as well. I’ll blog about that later when I have something to show 🙂

Syntax highlighting

Why’s there a little person in my button?

I was walking around the new VMware Promontory campus today and found this button.

Little Person in a Button

I’m tempted to press it, but I’m not sure what it will do. Send little stick figures to attack me? Stick me in a little house?

Any ideas?

Update: My apologies to anyone who couldn’t comment. My cache plugin cached the captcha. Oops. That’s been disabled now, so try again.

Recent Happenings in Review Board

It’s only been a couple of weeks since the last update about Review Board, but too much is happening to stay quiet.

New Features

  • Interdiffs

    Ever go through a long patch review process only to find it’s getting harder and harder to review any new changes due to the size of the patch? I’m sure most large projects have had to deal with this at one point or another.

    Review Board can now display interdiffs, which are diffs between versions of diffs. This makes it easy to see what was modified since the last change. Currently, there’s no UI for this, as there’s still a few backend changes that need to be made before we allow commenting on interdiffs, but it’ll be a full, proper feature very soon.

  • New Diff Algorithm

    We were previously using Python’s SequenceMatcher, along with some hacks, to generate the diffs. SequenceMatcher, unfortunately, just isn’t a desirable diff generator, due to its algorithm, quirks (assuming every change between matching blocks consists entirely of inserts, deletes, or changes instead of a mix of them), and lack of extensibility (no leading whitespace trimming to simplify diffs containing lots of indents).

    I’ve known for a while that we needed a new algorithm, so I finally sat down to write one. The obvious choice was the algorithm used in GNU diff (amongst other programs) — Eugene Myers’ O(ND) Difference Algorithm. After a few frustrating nights that left me nearly bald (ok, not really), it worked! And the diffs were much better than the older diffs. I think this is the first Python implementation of this algorithm.

    This also gives us the ability to add some heuristics and other logic to clean up the diffs. The result is much more readable diffs that no longer show indentation changes or huge misaligned fragments.

  • Improved support for project maintainers

    Project maintainers can now quickly download the uploaded diff from the diff viewer in order to apply the patch to their own tree. They’re also able to mark contributors’ review requests as submitted. While small additions, these make Review Board much more useful to open source projects.

  • Better browser and SCM compatibility

    We’re working on better browser compatibility and better SCM compatibility.

    CVS is almost implemented. Contributors are working on Git and BZR support. We’re working on the best ways to get these distributed systems integrated cleanly into the system.

  • Subversion post-review script

    We just received a patch for adding a Subversion-specific post-review script. Soon, we’ll combine the two and add support for CVS and other systems. This will give us one tool for creating review requests in all systems.

We’ve had a number of good contributors come out of the wood work, and from the sounds of it, several companies are using it now. I’m planning to put up a page listing teams, companies and projects using Review Board, so let me know if you’d like to be on the list! 🙂

Unity is now “real” – VMware Fusion Beta 4 released

Yesterday I briefly wrote about the new feature in VMware Fusion called Unity and linked to a video demonstrating what it can do. The buzz we got from that has been a lot of fun. Some people commented on various sites saying it was fake, even, which I found pretty funny having watched this thing being developed. I personally didn’t work on this feature, but I work with those who did, and they’ve sure looked sleep-deprived lately trying to get this feature and the next VMware Fusion beta ready.

They can finally sleep peacefully. We just put out VMware Fusion Beta 4. I want to issue a public congratulations to the developers who have put in a huge amount of time into this product. They’re not done yet, but I think they’re starting to see the light at the end of the tunnel.

Regis Duchesne, one of the developers of Fusion, talks briefly about where things stand and where things are going.

So again, congrats guys. I think I’m finally going to have to buy myself a Mac.

Unity

I remember the day I interviewed at VMware. I was asked what I would do to improve Workstation, and one of the things I said was that it would be nice to make a VM go rootless. That is, pull application windows out of the VM and make them integrate well with the operating system.

I wasn’t the only one. A lot of people wanted this type of feature. It’s been discussed for years, but it’s always been hard to find the manpower to do it. But competition is good, and we finally got some people on this feature. And it turned out spectacularly.

Check it out.

Kudos to the people who have spent many nights practically living here. They’ve pulled off an amazing feature. And we’re not done yet.

I know I’m going to be asked if we’re doing this on Linux or Windows. It’s too early to say what our feature list is like for the next major product. So feel free to just speculate 😉

Update: They moved the link on Digg, so updated my link.

Review Board has a home and a demo server!

I was stunned by the response to my post announcing Review Board. We had so much feedback and so many users signed up. Last I checked, our production server (which has so far been used as a demo/test server by others) has had over 550 registered users. Part of this was due to us being posted on reddit, which actually took down my server while I was sleeping!

Now, immediately we realized three things: 1) Sleeping is dangerous, 2) We needed a demo server, and 3) We needed a friendly home page.

I’ve been trying my best not to sleep, but that’s a ongoing project. As for the demo server, we pointed to our production server. This was, in retrospect, not the wisest of moves, as it ended up with hundreds of test comments. Not good when we’re actually using this for Review Board code reviews!

Today I addressed these problems (not the sleep part).

Behold! review-board.org is born! It provides a brief overview of the features, blog posts from David and I, and a link to our new demo server.

Now that we have a proper demo server people can play with, we’re going to start cleaning up our production server. If you have an account on there but are not working with the project, it will be removed in favor of the demo server. If you’re planning to contribute and would like us not to obliterate your account, please do let us know 🙂

A lot has been happening with Review Board lately and several features are in the works. I won’t go into detail right now, but for all those who have asked, yes, we are adding CVS support and are more than happy to accept patches and proposals for Git, Monotone, BZR, etc. support.