high wisdom



Sat Mar 8 19:47:35 IST 2008

Git-Bugzilla integration

UPDATE: See my blog post about GitZilla - git-bugzilla integration done right. The stuff below is bitrotten and outdated now.

There's always SCMBug, but you can cook your own Git-Bugzilla integration very easily. SCMBug's fine, but it's a lot of code if all you want is simple cross-linking between Git, Bugzilla and Trac (btw - GitPlugin for Trac would throw up an error unless you have at least *two* commits in your repository - talk about undocumented easter bugs!).

Anyhoo - here's what I want :
1. Git should disallow any commit where the commit message does not have a bug number.
2. Git should add a comment to the corresponding bug on a commit, mentioning the author, the Trac changeset link, the commit message and the list of files which changed.

And here's the code to do it. First, the post-receive hook :

#!/usr/bin/perl -w
use strict;

# A hook script which integrates with bugzilla. It looks for bug IDs in
# commit messages and adds the commit message as well as a link to the
# changeset as a comment on the bug.

# This program is released under the terms of the GNU General Public License
# version 2. A copy of the license may be obtained by emailing the author,
# or at http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt
#
# The absolute lack of warranty and other disclaimers as per the license
# apply.
#
# Copyright 2008, Devendra Gera. All rights reserved.
#
# Author : Devendra Gera 

### user configurable section

# The bugzilla has contains the server, username and password for the targeted
# bugzilla installation. There's NO 'http://' in the server line.
my $bugzilla = {
	server		=> "10.147.251.90/bugzilla",
	user		=> "demonuser",
	password	=> "daemonuserspassword",
};

# __PATH__ and __REVISION__ are replaced in $browser->{changeset} and
# $browser->{revision} to get the changeset and revision URLs respectively. 
my $browser = {
	changeset	=>
	"http://10.147.251.90/projname/changeset/__REVISION__",
	revision	=>
	"http://10.147.251.90/projname/browser/__PATH__?rev=__REVISION__",
};

# The bug_regex should extract the bug id from the commit message and place
# it in $1
my $bug_regex = 'bug #(\d+)';


##### End user configurable section

use WWW::Bugzilla;

my $input = <>;
chomp $input;

my ($oldrev, $newrev, $refname) = split /\s+/, $input;
my $commit_msg = `git-whatchanged $oldrev..$newrev`;

# prepare the changeset URL
my $changeset_url = $browser->{ changeset };
$changeset_url =~ s/__REVISION__/$newrev/g;

# author
my ($author) = ( $commit_msg =~ /^Author:\s+(.*)$/m );

# files
my @filelist = grep ( /^:/, split( /\n/, $commit_msg ) );

# prepare comment
$commit_msg =~ s/^.*?Date://s;	# eat everything till the Date: heder
$commit_msg =~ s/^.*?\n//m;	# eat the date line completely
$commit_msg =~ s/^:.*?$//mg;	# eat the file list from the msg.
chomp $commit_msg;
my ($bug_number) = ( $commit_msg =~ /$bug_regex/ );

my $comment = <<END_COMMENT;

------------------------------------
changeset $newrev [ $changeset_url ]
    by $author :

$commit_msg
------------------------------------

Files changed :
END_COMMENT

$comment .= join("", @filelist) . "\n";

my $bz = WWW::Bugzilla->new(
	server		=> $bugzilla->{ server },
	email		=> $bugzilla->{ user },
	password	=> $bugzilla->{ password },
	bug_number	=> $bug_number
);

die "cannot connect to bugzilla" unless defined $bz;

$bz->additional_comments( $comment );

$bz->commit;

And here's the update hook :

#!/usr/bin/perl -w
use strict;

my $refname = shift;
my $oldrev = shift;
my $newrev = shift;

my $commit_msg = `git-whatchanged $oldrev..$newrev`;

# check if the commit message contains a bug number
if($commit_msg !~ /bug #\d+/) {
	exit -1;
}

exit 0;

The TODO :
1. Push every configurable thing to git's config file and access it via git-config.
2. Code cleanups. One definition of the bug regex (which is hardcoded in the update for now).

edit : the here doc in the code was causing a problem with formatting, eating up some of the code. Fixed now.


Posted by gera | Permanent link | File under: tricks, perl, code, hacks | [ hide comments ]
But what is the normal flow for lets say a three developer shop. Does the hook sit in all developers repositories or on the "central" repo?
Amit Upadhyay, 10.03.2008, 11:12h #
of course the "central" repo. The developers might be offline (wrt bugzilla) when they commit to their repositories. Plus, they can commit all the junk they want to their repos.

How it would work is this :

Developer has a tracking branch, tracking the "official"/"central" repo. They keep updating this to get the latest changes from the official repo - and this is *always* a fast-forward (trivial) merge because they don't do any development on this branch. This is just a tracking branch.

Now, they're developing a feature (on a branch). They update and find new changes on the tracking branch and they rebase all their branches on the tracking HEAD. That way, the merge conflicts/commits happen on their branches and not the official branch/tree.

When a dev wants to release some feature, they submit ('git log -p --reverse --pretty=email origin..HEAD > commits.mbox' - which can be applied via git-am) (or publish) their commits wrt the official HEAD. The maintainer for the official repo pulls/applies their commits after review and pushes it to the official repo.

It is _on_ the official repo that these integration hooks run which notify that the commits are now official.

You could have them run on the individual repos, but that wouldn't make much sense and would lead to noise.
Gera, 10.03.2008, 12:03h #
Take a look at MediaWiki's BugzillaReports extension. It's really nice.

Trac ultimately has great SCM integration but doesn't blow me away in the wiki department. I don't know about getting a good git browser into MediaWiki ... but I'm guessing if someone were to write a decent SCM browser extension for MediaWiki that there would be much rejoicing.

MediaWiki + Bugzilla + Git with full integration between the three would be a powerful combo.
Dylan Carlson, 18.09.2008, 04:33h #
meh! MediaWiki is PHP. Trac is Python. I'd write python over php anyday.

And I'd write C over python anyday as well. So I patched pam_mysql to work with Trac + MySQL + AccountManager plugn for trac. Now our vpn, hosts, wiki etc., can auth using pam_mysql. No more LDAP hassles. Next hop is probably nss_mysql.

Another interesting side effect is that Trac/AccountManager gives us a nice browser based interface to change passwords, and once you change it there, it takes effect everywhere.

Using .htaccess to enable mod_auth_pam to protect arbitrary apache2 urls with the same credentials is trivial too (although there's sparse documentation about a small gotcha). I'll write about all that soon, and publish the pam_mysql patch after adding realm support.
Gera, 18.09.2008, 05:01h #
btw, our "official" repo is SVN. Most of us (except one) use git at our ends (via git-svn) which allows everyone to be comfortable and slowly migrate after seeing the light.
Gera, 18.09.2008, 05:02h #
Agreed about python, and I like PHP even less than you do. But in the face of it -- short of buying Confluence -- MediaWiki is the best wiki going. The only way you're going to be able to do as much markup or features in another wiki is if you have a lot of spare time to build it yourself. The MW extensions are growing and as long as someone else does the bulk of coding, you don't have to care what it is written in. Just hack on it. :)

Sorry don't mean to get too pragmatic about code but I don't really care what anything is written in nowadays (except say, VisualBasic) as long as the desired result occurs. Which is why I permitted Zebra in perl :)

Speaking of Zebra I am alternatively hacking away on Puppet right now. I just want to see it from a different angle and Puppet gives me another opp. to do ruby. I'll see what I learn from that and offer input what could make Zebra better.
Dylan Carlson, 18.09.2008, 06:20h #
Hi Dylan - you've just given me an idea on some future enhancements to the BugzillaReports MediaWiki extension that I've been developing. Bringing in SCM reports into the Bugzilla Reports might be pretty useful :)
Ian Homer, 29.11.2008, 20:43h #
Add comment here
name:
e-mail:
website:
spam-protection, please enter code: captcha, sorry
comment:

remember me
request e-mail if someone adds comments
Your e-Mail will not be published,
http://www.URL.com will turn to link automatically,
html is not permitted