Comments on: Recent Happenings in Review Board

By: Emanuele Aina

Emanuele Aina — Wed, 27 Jun 2007 09:27:00 +0000

Mercurial has an improved diff module as they experienced some problems with the difflib Python module. It is faster and gives good results in terms of patch size: http://www.selenic.com/mercurial/wiki/index.cgi/FAQ#head-4173e68b299fdcfb38baa08f08c3fbb1868e7d1a

It also uses a C module for the speed-critical portions and it can be found here: http://selenic.com/repo/hg/file/tip/mercurial/mdiff.py

By: Scott Perry

Scott Perry — Wed, 27 Jun 2007 06:03:43 +0000

It is likely that Apple will be using Review Board to some extent internally, although, as with most things at work, I can’t really talk about it.

By: Ken Schalk

Ken Schalk — Tue, 26 Jun 2007 16:33:08 +0000

ChipX86: Bram’s algorithm will give good diff results for cases where text has been moved, but it will still appear as “lines deleted here, lines added there.” The main thing it avoids is the kind of confusion GNU diff can produce, like “these curly braces on lines by themselves are unchanged but everything around them is different.”

P.S. FWIW, the merge tool I personally rely on (and a bunch of microprocessor designers at Intel rely on) is using Bram’s diff algorithm (actually the whole precise Codeville text merger which he developed).

By: ChipX86

ChipX86 — Mon, 25 Jun 2007 18:07:52 +0000

Ken: It’s interesting that you said that. As I was going to sleep last night, I started wondering about that same approach and figured it would be especially good to find out if lines have been moved, so that we could mark them up in a certain way. I have some notes with almost that exact approach, but I wasn’t sure how much better it would actually work. If this is indeed being used in other VCSes, maybe I’ll give it a shot.

Thanks for the links!

By: Ken Schalk

Ken Schalk — Mon, 25 Jun 2007 14:12:45 +0000

It’s too bad I couldn’t bring it to your attention before you spent the effort to implement the GNU diff algorithm, but there’s a diff algorithm that a couple VCSes are using that gives really good results (when used on source code). Basically, rather than doing a longest common sub-sequence, it finds all lines that occur exactly once on both sides and matches on those first. Then it subdivides the file based on those matches and uses the same algorithm on those sub-parts (i.e. matching on lines that are unique within that section of the file). This algorithm seems to have a much higher probability of producing short, readable diffs, and a much lower chance of producing a diff that seems “wrong” to a human.

It was developed by Bram Cohen. You can find one Python implementation of it included in the file attached here (which is actually a merge algorithm that uses this diff algorithm internally):

http://revctrl.org/PreciseCodevilleMerge

Bram mentioned recently that bazaar is now using it:

http://bramcohen.livejournal.com/37690.html

By: ChipX86

ChipX86 — Mon, 25 Jun 2007 08:57:18 +0000

Apan: Very valid point. The whitespace issue isn’t a problem unless it’s leading whitespace. Trailing whitespace, as well as lines containing only whitespace, are left alone.

The issue of trashing indentation is a potential problem. There are things we may be able to do, such as mark up indentation-only changes specially while keeping them out of the actual diff chunks. It’s also possible for us to add an option for showing the diff with/without whitespace changes.

So far, the potential for a cleaned diff far outweighs the risks of a change completely screwing up indentation, and there are definitely things we can do about that.

By: Apan

Apan — Mon, 25 Jun 2007 08:19:25 +0000

“This also gives us the ability to add some heuristics and other logic to clean up the diffs. The result is much more readable diffs that no longer show indentation changes or huge misaligned fragments.”

How will one know if a diff trashes the indentation or leaves whitespaces all over the place?

By: Matt Doran

Matt Doran — Sat, 23 Jun 2007 01:18:08 +0000

Maybe you could have made use of the diff code that Google published a week or so ago. I think they might have beaten you to it!

They implemented an identical diff/patch API in Java, Python and Javascript. And it has a friendly license. You can find it here: http://code.google.com/p/google-diff-match-patch/

It could have saved you some work … oh well. Maybe yours is better.