Git/Tutorial

Git-Gerrit lets us deploy your code faster and makes collaboration easier!

Git
TL;DR Tutorial Workflow Tips Download from Git Commit message guidelines New repositories GUI applications more.. »
Gerrit
Navigation Resolving conflicts Mass approval Splitting a submitted change Watched projects Personal sandbox Workflow for ops more.. »

Welcome to the tutorial on Git-Gerrit Tutorial, originally given at the Berlin Hackathon 2012. The live tutorial will help you with:

• step-by-step instructions and screenshots for getting Gerrit access
• setting up Git on your machine
• configuring Git
• installing Git-review
• submitting a patch
• the MediaWiki code review process

How to Use this Tutorial

Prepare for Tutorial

You will need a Developer access account. You will also need to download Git (already in Ubuntu core) and any optional packages you want to use.

Skim the Content

Be sure this is the tutorial you need. This page was created to house the content of the 1 hour-long, step-by-step tutorial given at the Berlin Hackathon 2012. Git/Workflow has additional information, for instance on cherry-picking. For the quick version, see Git/TLDR.

What is Git?

Git is a distributed version control system (dvcs) written in C originally developed by Linus Torvalds and others to manage the Linux kernel. In the past couple of years, it has taken off as a very robust and well-supported code repository. “Distributed” means that there is no central copy of the repository. With Subversion, Wikimedia’s servers host the repository and users commit their changes to it. In contrast, with Git, once you’ve cloned the repository, you have a fully functioning copy of the source code, with all the branches and tagged releases at your disposal.

Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. Git is easy to learn and has a tiny footprint with lightning fast performance. It outclasses SCM tools like Subversion, CVS, Perforce, and ClearCase with features like cheap local branching, convenient staging areas, and multiple workflows.

What is Gerrit?

Gerrit is a free, web-based collaborative code review tool that integrates with Git. It has been developed at Google by Shawn Pearce (co-author of Git, founder of JGit) for the development of the Android project.

Starting from a set of patches for Rietveld, it became a fork and evolved into a full blown project when ACL patches wouldn't be merged into Rietveld by its author, Guido van Rossum.

Originally written in Python like Rietveld, it is now written in Java (Java EE Java Servlet) with SQL since version 2.

Why did Wikimedia engineering move from Subversion to Git

Three major reasons:

1. To encourage participation: Since Git is distributed, it allows people to contribute with a much lower barrier to entry. Anyone will be able to clone the repository and make their own changes to keep track of them. And if you’ve got an account in our code review tool (Gerrit), you’ll be able to push changes for the wider community to review.
2. To fix our technical process: Subversion has technical flaws that make life difficult for developers. Notably, the implementation of branching is not very easy to use, and makes it hard to use “feature branches”. Our community is very distributed, with many parallel efforts and needs to integrate many different feature efforts, so we’d like to use feature branches more. Git branches are very easy to work with and merge between, which should make things easier for our development community. (Several other large projects, such as Drupal and PostgreSQL, have made the same switch for similar reasons, and we’ve done our best to learn from their experiences.)
3. To get improvements to users faster: With better branching and a more granular code review workflow that suits our needs better, plus our ongoing improvements to our automated testing infrastructure, we won’t have to wait months before deploying already-written features and bugfixes to Wikimedia sites.

Some quotes from our community

• “I love git just because it allows me to commit locally (and offline).” – Guillaume Paumier

• “[Y]ou can create commits locally and push them to the server later (great for working without wifi), you can tell it ‘save my work so I can go do something else now’ in one command, and it’ll allow us to review changes before they go into “trunk” (master)…. without human intervention in merging things into trunk. Gerrit automates this process.” – Roan Kattouw

• Don't attempt to use Git from Windows if you never used tools made for Unix on Windows. I grant you it will be a pain because even simple copy'n paste operations won't work and it has a different understanding about directories. – 77.184.47.28

Set Up SSH Keys in Gerrit

We use SSH keys to establish a secure connection between your computer and Gerrit. Setting them up is fairly easy, but does involve a number of steps. Run the following commands in a terminal; on the Mac, open Terminal.app, usually found at /Applications/Utilities).

To make sure you generate a brand new key, you need to check if one already exists.

Generate a new SSH key

To generate a new SSH key, enter the code below. We want the default settings so when asked to enter a file in which to save the key, just press enter.

ssh-keygen -t rsa -C "[email protected]"

Assign the pass phrase (press [enter] key twice if you don't want a passphrase).

Why do passphrases matter?

Passwords aren’t very secure, you already know this. If you use one that’s easy to remember, it’s easier to guess or brute-force (try many options until one works). If you use one that’s random it’s hard to remember, and thus you’re more inclined to write the password down. Both of these are Very Bad Things™. This is why you’re using ssh keys.

But using a key without a passphrase is basically the same as writing down that random password in a file on your computer. Anyone who gains access to your drive has gained access to every system you use that key with. This is also a Very Bad Thing™. The solution is obvious: add a passphrase.

But I don’t want to enter a long passphrase every time I use the key!

Neither do we! Thankfully, there’s a nifty little tool called ssh-agent that can save your passphrase securely so you don’t have to re-enter it. If you’re on OSX Leopard or later your keys can be saved in the system’s keychain to make your life even easier.

Which should give you something like this:

It will create 2 files in ~/.ssh directory as follows:

~/.ssh/id_rsa : identification (private) key
~/.ssh/id_rsa.pub : public key

Add your SSH key

Open the id_rsa.pub file with a text editor (Notepad, TextEdit, or gedit will do just fine). This is your public SSH key. You may need to turn on “view hidden files” to find it because the .ssh directory is hidden. It’s important you copy your SSH key exactly as it is written without adding any newlines or whitespace. Now paste it into the “Key” field.

cat /home/preilly/.ssh/id_rsa.pub 

Can’t view hidden files? Other ways to copy:

OSX

$ pbcopy < ~/.ssh/id_rsa.pubCopies the contents of the id_rsa.pub file to your clipboard

Windows You can open Git Gui, go to Help > Show Key, and then press Copy To Clipboard to copy your public key to your clipboard

Linux

$ sudo apt-get install xclip # Downloads and installs xclip
$ xclip -sel clip < ~/.ssh/id_rsa.pub

Log into the web interface for gerrit. Click on Settings then SSH Public Keys then add your key.

Setting up Git

On Ubuntu you can install the Git command line tool via the following command:

 sudo apt-get install git-core

Mac OS X

Git for the Mac platform

Windows

Git for the Windows platform

Optional packages:

• TortoiseGit (http://code.google.com/p/tortoisegit/) if you want to have integration with Windows Explorer.
• https://bugzilla.wikimedia.org/attachment.cgi?id=10540 - Windows git-review package

IMPORTANT: Add the git directory to your path.

Linux and Unix

It is easiest to install Git on Linux using the preferred package manager of your Linux distribution.

Debian/Ubuntu

$ apt-get install git-core

Fedora

$ yum install git

Gentoo

$ emerge --ask --verbose dev-vcs/git

FreeBSD

$ cd /usr/ports/devel/git
$ make install

Solaris 11 Express

$ pkg install developer/versioning/git

OpenBSD

$ pkg_add git

For other Linux distributions please check your vendor documentation.

Now that you have Git set up and your SSH keys entered into Gerrit, it’s time to configure your personal info.

Configure Git

Now that you have Git on your system, you’ll want to do a few things to customize your Git environment. You should have to do these things only once; they’ll stick around between upgrades. You can also change them at any time by running through the commands again.

Git comes with a tool called git config that lets you get and set configuration variables that control all aspects of how Git looks and operates.

git config -l

Set your username and email

Git tracks who makes each commit by checking the user’s name and email. In addition, we use this info to associate your commits with your gerrit account. To set these, enter the code below, replacing the name and email with your own.

 git config --global user.email "[email protected]"
 git config --global user.name "preilly"

Download MediaWiki example extension using Git

You can currently download MediaWiki core using Git, as well as any extension currently installed on the Wikimedia Foundation server cluster. By July 2013, all extensions will either be available using Git or move to alternate version control hosts.

The first step is to clone the MediaWiki repository.

In order to collaborate with us, or if you want to get a copy of the MediaWiki repository so you can look at or use the code, you will clone it. You simply run the following on your command line:

git clone https://gerrit.wikimedia.org/r/p/test/mediawiki/extensions/examples.git

This will copy the entire history of the MediaWiki repository so you have it locally and it will give you a working directory of the main branch so you can look at the code or start editing it. If you change into the new directory, you can see the .git subdirectory - that is where all the project data is.

By default, Git will create a directory that is the same name as the project in the URL you give it - basically whatever is after the last slash of the URL. If you want something different, you can just put it at the end of the command, after the URL. So, in this example you will have a core directory.

Prepare to work with gerrit

In order to properly work with gerrit, you need to have a pre-commit hook that adds a "change id" to your commit summary. That unique ID will let Gerrit track your commit. Since Git commit identification is based upon a sha1 of the commit id, whenever you amend a commit, the sha1 will change and gerrit would have lost track of it.

Installing git-review

First, install the release version of git-review using python package installer pip:

$ sudo apt-get install python-pip

On OpenSuse, install via YaST python-setuptools, then run

$ easy_install pip

$ sudo pip install git-review

Note: if you don't have apt-get but have python installed, you can use this:

$ sudo easy_install pip
$ sudo pip install git-review

Setting up git-review

After cloning a repository, you need to set up for using git-review. This will automatically happen the first time you try to submit a commit, but it's generally better to do it right after cloning.

 $ git review -s

Which should give you this:

If you see the authenticity of host gerrit.wikimedia.org can't be established... Don’t worry, this is supposed to happen. Type “yes”.

This may ask you for your git username, if it's different from the shell username you're using.

Internally, this does the following:

• checks whether accessing the remote repository works
• if it doesn't, asks for a username and tries again
• creates a remote called 'gerrit' that points to gerrit
• installs the commit-msg hook

How to submit a patch

The main avenue for submitting changes to MediaWiki is to first join the MediaWiki development community so you can submit changes to Gerrit, our code review tool. Getting developer access is relatively easy.

Update master

Make sure that your master branch (the branch created when you initially cloned the repository) is up to date:

git pull origin master

Create a branch

First, create a local branch for your new change. Give the branch a short but reasonably descriptive name.

git checkout -b BRANCHNAME master

This will create a new branch (BRANCHNAME) from 'master' and check it out for you. This is equivalent to doing

git branch BRANCHNAME master
git checkout BRANCHNAME

Use a descriptive branch name. If you are working on a bug, include its id, for example "2012/bug12345".

Make and commit your change

Usual way

Modify your local code in some fashion.

vim Example/Example.body.php

Then check the changes you've made, within the file(s) and within the directory:

git diff

Without any extra arguments, a simple git diff will display in unified diff format (a patch) what code or content you've changed in your project since the last commit that are not yet staged for the next commit snapshot.

Then check the changes you've made, within the file(s) and within the directory:

git status

You run git status to see if anything has been modified and/or staged since your last commit so you can decide if you want to commit a new snapshot and what will be recorded in it.

This will show all modified files. To prepare submitting a file, you should add your changes to the index (the staging area between your working copy and your local repository), which is done by using the git add command. (Note that git add newly stages files. But git -a stages any files previously committed.)

git add Example/Example.body.php

You run git add on a file when you want to include whatever changes you've made to it in your next commit snapshot. Anything you've changed that is not added will not be included - this means you can craft your snapshots with a bit more precision.

While doing so you can always review the change already added to the staging area by invoking git status and look at the diff with

git diff --cached

.

The git diff --cached command will show you what contents have been staged. That is, this will show you the changes that will currently go into the next commit snapshot.

Once you are happy with the change list, you can add them to your local repository by using

git commit

.

Now that you have staged the content you want to snapshot with the git add command, you run git commit to actually record the snapshot.

You will then be prompted with your favorite editor to add a descriptive message for this commit. This is what other people will see when you will later push that commit to another repository. If you do not manually add a Change-ID line to your commit message, it will be automatically generated and added for you.

You can repeat this step over and over until you have a set of changes that you want to have pushed to the master branch. One of the cool things about git is that when you git commit, you are committing to your local copy. This means you can commit as often as you like without potentially screwing things up for another developer on the project, unlike in SVN where you would want to be very careful that the changes you commit would not cause things to break.

Hence the workflow is something like:

# Add change:
$ git add <some file>
# Verify list of files added to the staging area
$ git status
# Review diff of changes staged:
$ git diff --cached
# repeat until you are happy with your changes
$ git commit
<edit commit message>

Prepare to push your change set to Gerrit

Before your changes can be merged into master, they must undergo review in Gerrit.

But first, it's a good idea to synchronize your change set with any changes that may have occurred in master while you've been working. From within the branch you've been working on, execute the following command:

git pull origin master
git rebase master

git pull will update the code in your local copy of the master branch.

Then, git rebase will temporarily set aside the changes you've made in your branch, apply all of the changes that have happend in master to your working branch, then merge all of the changes you've made back into the branch. Doing this will help avoid future merge conflicts. Plus, it gives you an opportunity to test your changes against the latest code in master.

Once you are satisfied with your change set and you've rebased against master, you are ready to push your code to Gerrit for review.

Push your change set to Gerrit

If you installed git review and you ran git review -s to set it up for this repository, the command to push changes to Gerrit is very simple:

git review -R

Upon success, you'll get a confirmation and a link to the changeset in Gerrit. New patchset: preilly; "Added get version method to extension" [test/mediawiki/extensions/examples] (master) - https://gerrit.wikimedia.org/r/9332

(If you forgot to run git review -s, "remote" will complain about "missing Change-id in commit message". But it will also suggest a commit message with a Change-Id: INNNXXXNNN... line. Copy that line starting with "Change-Id", run commit --amend, and paste the Change-Id line at the end. Then repeat git review -R and it should complete.)

You can view this change in the Gerrit Web UI:

If your commit addresses a bug in Bugzilla, please comment on that bug to note that the commit is in the merge queue, and link to its changeset in Gerrit.

Amending a change

Sometimes, you might need to amend a submitted change. You can amend your own changes as well as changes submitted by someone else, as long as the change hasn't been merged yet.

If you have git-review, checkout the change like this:

git review -d <change number>

Note, if you already have the change in a branch on your local repository, you can just check it out instead:

git checkout <branch name>

For example:

git review -d 9332

Or, if you already have the branch,

git checkout 2012/bug12345

should accomplish the same thing.

Next, make some changes.

vim Example/Example.body.php 

git add the files as needed, then commit the change (ensuring you are amending the commit):

git add Example/Example.body.php
git commit --amend

NOTE: DO NOT use the -m flag to specify a commit summary: that will override the previous summary and regenerate the Change-Id. Instead, use your text editor to change the commit summary if needed, and keep the Change-Id line intact.

Push the change

git review -R

The -R is important here. It tells git-review to not rebase your change against master, which clutters diffs between patch set 1 and 2.

New patchset: preilly; "Added get version method to extension" [test/mediawiki/extensions/examples] (master) - https://gerrit.wikimedia.org/r/9332

git review complains about multiple commits

If git review asks you if you really want to submit multiple commits, and lists a bunch of unrelated commits from different branches, try either of this:

git fetch --all

git remote update

Both commands do exactly the same thing, they fetch objects from all remote repositories set. So just pick the command you remember easily and forget about the other one.

How we review code

The things we look for in code won't change, but the workflow is different than it was before the Git switch.

Review before merge

It's important to us to have a review-before-merge workflow for MediaWiki core and also for any extension we deploy. We will also offer that option to any extension author who wants it for their extension. The one exception is localisation and internationalisation commits, which will be able to be pushed without review.

Who can review? Gerrit project owners

Who has the ability to do code review?

We use gerrit to manage code review. Anyone can ask for a Gerrit account (Get an account!). Within Gerrit, anyone can comment on commits and signal their criticisms and approvals. Anyone can give a nonbinding "+1" to any commit. However, for any given repository ("Gerrit project"), only a small group of people will have the ability to approve code within Gerrit and merge it into the repository. (Within gerrit, this superapproval is a "+2" even though that's a misleading name, because two +1 approvals DO NOT add up to a +2.) These people are "Gerrit project owners". To learn about becoming a Gerrit project owner, see Git/Gerrit project ownership.

Even within a Gerrit project, we can also specify particular branches that only specific people can pull into.

How to comment on, review, and merge code in Gerrit

A sample changeset, with annotations

Side-by-side diff

Review screen

Anyone can comment on code in Gerrit.

Viewing and commenting on code

• Make sure you have a https://gerrit.wikimedia.org login (Get an account!). If you don't know, try logging in at https://labsconsole.wikimedia.org; the username and password should be the same. If you can't, ask in #mediawiki^connect for someone to help.
• Log in to Gerrit. If you know the changeset you want to look at (URL will look like https://gerrit.wikimedia.org/r/#change,8939 ), go to that. Otherwise, use the search box and try searching. There is no fulltext search in Gerrit, but you can search by author ("Owner"), Gerrit project, branch, changesets you've starred, etc. The Gerrit search documentation covers all of the different search operators you can use.
• The changeset has a few important fields, links and buttons:

• Reviewers. 'jenkins-bot' is the autoreviewer that auto-verifies anything that passes the Jenkins tests. If it passes, you see a green checkmark. If it fails, a red X. A changeset needs a green checkmark before anyone can merge it.

• Add reviewer (manually ping someone to request their review. It'll show up in their Gerrit stream)

• Side-by-side diff button:

• Opens the diff. You can double-click on a line and comment on that line, then save a draft comment! Then, click "Up to change" to go back to the changeset.

• Abandon Change button (you'll see this if you wrote this diff. This action removes the diff from the merge queue, but leaves it in Gerrit for archival purposes)
• Review button:

• The purpose of this page is to signal your thoughts on the commit. You can leave an overall comment under Cover Message. You can also view inline comments from the diff that are still in draft form and awaiting publication. If upon code review, you approve, use "+1" under Code Review; otherwise, use "-1" to disapprove. These numbers are nonbinding, won't cause merges or rejections, and have no formal effect on the code review. If you have actually tested the code, and it behaves as intended, use "+1" under Verfied; if you tested and it behaves incorrectly, use "-1". If you did not test this change, leave Verified at 0. To publish your draft inline comments plus your comments on the diff as a whole, click Publish.

Comparing patch sets

Every time you amend your commit and submit it for review, a new patch set is created. You can compare the different patch sets like this:

• Select the older patch set in the "Old Version History" list.
• Expand the newer patch set details by clicking the arrow near it.
• Click Side-by-Side. Note that in Gerrit 2.3 this only works if you open the diff in the same tab, so don't open it in a new tab.

Formally reviewing and merging or rejecting code

If you are one of the Gerrit project owners, you'll also see:

A Gerrit review screen with approval and veto options

• Abandon Change button
• on the Review page, additional Code Review options to +2 (approve) or -2 (veto) a diff, and a Publish And Submit button (publish your comment and merge diff into the branch, in 1 step)
• Submit Patch Set 1 button (merge -- only useful if you or someone else has already given a +2 approval to the diff, but not merged it)

And once you've merged something into the example Gerrit project you'll see it in https://gerrit.wikimedia.org/r/gitweb?p=test/mediawiki/extensions/examples.git;a=summary .

If you merge a commit that references a Bugzilla bug, please go to that bug and mark it RESOLVED: FIXED and reference the merge ID.

See also

Also useful are these pages:

• Git and gerrit in Wikimedia labs
• An intro-to-git "training mission"
• An introduction to git-svn for Subversion/SVK users and deserter by Sam Vilain
• GitMagic A git guide with translations - recommended
• A git branching model
• git: the simple guide
• Git Community Book will take you gently into Git internals. Actually it is very hard to "get" git until knowing something about how it works internally. After this, everything become simple, just hidden without convoluted and anti-productive user interface.
• How we're going to use Git: slides by Brion Vibber, as presented autumn 2011
• Audio from Git presentation by Brion Vibber, autumn 2011
• Notes from Brion's presentation
• Git's website with documentation
• MediaWiki Git Guide (MGG) - selection of relevant Git pages in the MW Virtual Library; for PDF generation or printing
• A very useful Git guide

Also useful (if you subtract away GitHub stuff):

• http://wiki.openstack.org/GerritJenkinsGithub
• Gerrit bugs that matter
• https://gerrit.wikimedia.org/r/#q,status:open,n,z
• Download from Git
• Git-review
• labsconsole:Help:Access
• Gerrit