Post-Commit Web Hooks

Can a java authentication example be provided? I'm trying to authenticate hooks received in a servlet.

After a timeout, I managed to roll my own auth implementation for GC's web hook messages. An example and the jar file are available at:

http://webhooks.googlecode.com

In case you're wondering how to catch these web hooks with PHP, I posted my solution here. I just cover catching the POST data, which wasn't readily apparent to me. I haven't covered the authentication yet.

Here's a pretty full script for catching and authenticating these hooks using PHP. I use my own library a little, but it would be easy to extrapolate the key details you'll need.

http://code.google.com/p/joshlib/wiki/CatchingWebHooks

I realized a php script that reads the web-hook "modified", "added", and "removed" fields and syncronizes your "non-working copy" on the webserver (not running svn!). You can find the script at: http://www.aleritty.net/progetti/post-commitphp/

Thanks for coming up with this! I've implemented an endpoint of this for the CIA.vc commit tracker, so people with google code can usefully have their commits reported. Latency looks pretty good so far, I'm getting around 5 seconds, most of which might be due to our side. Two things: It may be useful to allow people to set several addresses, "chaining" hook endpoints or setting up one that bounces on to all targets is clumsy. Also, it seems the current implementation treats only HTTP 200 as success: My original code returned a 202 (Accepted) since that seemed like the appropriate code, but this seems to have caused the hook system to retry a few times. You may want to change that.

For those that use Brightkite and Google code. I wrote a script that posts the commit to Brightkite http://pastebin.com/f7ac7cc00

You can use php to proccess the PostCommitWebHooks

<?php
//google code secret key
$google_code_secret_key = 'RD5234999kc51l';
//get revision commit data
$revision_data=file_get_contents('php://input');
//build secret verify info;
$secret_verify=hash_hmac("md5",$revision_data,$google_code_secret_key);
//get google secret info
$google_secret_info=$_SERVER['HTTP_GOOGLE_CODE_PROJECT_HOSTING_HOOK_HMAC'];

//prase revision commit data
$revision_info=json_decode($revision_data);
//var_export $revision info
$revision_var_export=var_export($revision_info,true);

$fc="\n google code secret key : ". $google_code_secret_key;
$fc.="\n secret verify info : ". $secret_verify;
$fc.="\n google secret info : ". $google_secret_info;
$fc.="\n project name : ". $revision_info->project_name;
$fc.="\n revision count : ". $revision_info->revision_count;
foreach($revision_info->revisions as $revision){
$fc.="\n author:" .$revision->author;
$fc.="\n time: " .date('Y/m/d H:i:s',$revision->timestamp);
$fc.="\n url: ".$revision->url;
$fc.="\n message: ".$revision->message;
$fc.="\n revision: ".$revision->revision;
}


file_put_contents("log.txt",$fc);

For those doing continuous integration with Buildbot, I'm working on a ChangeSource? for Google Code web hooks.

It's a good thing that svnsync doesn't trigger web hooks. I just reset my repository and uploaded it back (with some newline corruption from old revisions fixed), and I certainly wouldn't have wanted that to trigger my buildbot for all 1300 revisions.

Here is a sample php code to send commit notifications via gmail.

You will need to install PEAR for PHP at first:

<?php
require_once "Mail.php";

$data = file_get_contents("php://input"); //raw POST data
$data = json_decode($data, true);


// Commit information
$project_name = $data["project_name"];

foreach($data["revisions"] as $d) {
	$revision = $d["revision"];
	$author = $d["author"];
	$message = $d["message"];
}

$from = '**********';
$to = '*********';
$subject = '['. $project_name .'] r'. $revision .' committed - '. $message;
$body = 'Author: '. $author ."\n";
$body .= 'Comments: '. $message ."\n";
$body .= 'URL: http://code.google.com/p/[project-name]/source/detail?r=' . $revision;

// Stick your GMAIL SMTP info here! ------------------------------
$host = "smtp.gmail.com";
$port = "587";
$username = "**********";
$password = "**********";
// --------------------------------------------------------------

$headers = array ('From' => $from,
  'To' => $to,
  'Subject' => $subject);
$smtp = Mail::factory('smtp',
  array ('host' => $host,
    'port' => $port,
    'auth' => true,
    'username' => $username,
    'password' => $password));

$mail = $smtp->send($to, $headers, $body);
$log = fopen("log.txt","a");
fputs($log, $subject);
fputs($log, "\n");
if (PEAR::isError($mail)) {
  fputs($log, $mail->getMessage());
  fputs($log, "\n");
  fputs($log, "\n");
} 
else {
  fputs($log, "Message successfully sent!");
  fputs($log, "\n");
  fputs($log, "\n");
}
?>

@nicolas.alvarez: Do you have any progress for the Buildbot ChangeSource? ?

I build a joomla module for this, if you use jommla and want to show the changes of a project, see http://code.google.com/p/svnupdates

Buildbot Users:

A buildbot changesource (polling on the Atom feed) is available at http://code.google.com/p/pitaara/

Works with both Hg and SVN repositories. Doesn't understand branches.

http://www.nabble.com/how-are-you-using-Hudson-for-non-Java-projects--tt17145574.html is broken under "others" link.

The latest release of Vulcan (a continuous integration java web app) supports these web hooks. See http://code.google.com/p/vulcan/wiki/GoogleCodeCommitHook.

Also, for anyone validating these requests in Java, here's some sample code to do it: SignedRequestAuthorizationFilter.

why can't we use HTTPS protocol in our commit hook GET request?

I would like to know when branches will be recognized in the Post-Commit Web Hook. The branch data is put in our source list, but not the commit hook.

Right now our project at http://code.google.com/p/sm-ssc/ has 7 active branches (and a few more than that that are closed, but that's another issue for another time), and it would be helpful if we didn't have to manually enter branch names on the commit.

Issue for HTTPS support: http://code.google.com/p/support/issues/detail?id=4958

Can somebody please shed some light on this: I have a post commit which compiles something and takes more than the above timeout of 15 seconds allows. Normally, Google treats this as a non-successful hit, and would retry the request in a specific interval. To overcome these retries, I had the idea to manually send a status 200 quickly, before compiling. Well, the url is still loading in that case, and Google may treat the timeout again as an error. Is that the case?

FYI, support for the Google Code Web Hook has just been merged in buildbot: https://github.com/buildbot/buildbot/pull/291#issuecomment-3199354

To use PHP to verify the authentication hash, you can do this. It supports multiple projects and can also run svnsync:

<?php
$auth_keys = array(
    "proj1" => "0123456789abcdef",
    "proj2" => "0123456789abcdef",
);

if (!isset($_SERVER["HTTP_GOOGLE_CODE_PROJECT_HOSTING_HOOK_HMAC"])) {
    die("No authentication hash.");
}

$hmac = $_SERVER["HTTP_GOOGLE_CODE_PROJECT_HOSTING_HOOK_HMAC"];
$data = file_get_contents("php://input");
$json = json_decode($data, true);
if (!isset($json["project name"]) || !in_array($json["project name"],array_keys($auth_keys))) {
    die("Invalid project name.");
}
$p = $json["project name"]; // This can also be retrieved from $_GET["p"] if you set up your url as ?p=%p
$digest = hash_hmac("md5", $data, $auth_keys[$p]);
if ($digest != $hmac) {
    die("Invalid authentication hash.");
}

# Run svnsync with the correct user using sudo (remember to edit sudoers!)
system("sudo -u userwithaccesstosyncrep svnsync sync --non-interactive --no-auth-cache --username svnsync file:///home/path/to/svnsync/repo/for/$p");

However, this requires you to make your web server user (often 'www-data') to be able to launch svnsync without sudo having to ask for a password:

$ sudo visudo

Then add this to the bottom (it's important that it is in the bottom!):

www-data ALL=(ALL) NOPASSWD:/usr/bin/svnsync

However, this is really much work for just running svnsync. I suggest that, if that's your only goal, set up a cronjob to run svnsync @daily (or @hourly). It's easier and more resistant to errors. Run with user that has access to your svnsync repo:

$ crontab -e

And add:

@daily svnsync sync --non-interactive --no-auth-cache --username svnsync file:///home/path/to/svnsync/for/proj1

<?php //google code secret key $google_code_secret_key = 'RD5234999kc51l'; //get revision commit data $revision_data=file_get_contents('php://input'); //build secret verify info; $secret_verify=hash_hmac("md5",$revision_data,$google_code_secret_key); //get google secret info $google_secret_info=$SERVER['HTTP_GOOGLE_CODE_PROJECT_HOSTING_HOOK_HMAC'];

//prase revision commit data $revision_info=json_decode($revision_data); //var_export $revision info $revision_var_export=var_export($revision_info,true);

$fc="\n google code secret key : ". $google_code_secret_key; $fc.="\n secret verify info : ". $secret_verify; $fc.="\n google secret info : ". $google_secret_info; $fc.="\n project name : ". $revision_info->project_name; $fc.="\n revision count : ". $revision_info->revision_count; foreach($revision_info->revisions as $revision){ $fc.="\n author:" .$revision->author; $fc.="\n time: " .date('Y/m/d H:i:s',$revision->timestamp); $fc.="\n url: ".$revision->url; $fc.="\n message: ".$revision->message; $fc.="\n revision: ".$revision->revision; }

file_put_contents("log.txt",$fc);