\[ \newcommand\inv{^{-1}}\newcommand\invt{^{-t}} \newcommand\bbP{\mathbb{P}} \newcommand\bbR{\mathbb{R}} \newcommand\defined{ \mathrel{\lower 5pt \hbox{${\equiv\atop\mathrm{\scriptstyle D}}$}}} \] Back to Table of Contents

26 Source code control

Source code control

systems, also called revision control \index{revision control systems|see{source code control}} or version control systems, are a way of storing software, where not only the current version is stored, but also all previous versions. This is done by maintaining a repository for all versions, while one or more users work on a `checked out' copy of the latest version. Those of the users that are developers can then commit their changes to the repository. Other users then update their local copy. The repository typically resides on a remote machine that is reliably backup up.

There are various reasons for keeping your source in a repository.

There are various source code control systems; in this tutorial you can learn the basics of Subversion (also called svn), which is probably the most popular of the traditional source code control systems, and Mercurial (or~hg), which is an example of the new generation of \indextermsub{distributed}{source code control} systems.

26.1 Workflow in source code control systems

Top > Workflow in source code control systems

Source code control systems are built around the notion of repository : a central store of the files of a project, together with their whole history. Thus, a repository allows you to share files with multiple people, but also to roll back changes, apply patches to old version, et cetera.

The basic actions on a repository are:

Adding your own changes is not always possible: there are many projects where the developer allows you to check out the repository, but not to incorporate changes. Such a repository is said to be read-only.

Figure  1 illustrates these actions for the Subversion system.

Workflow in traditional source code control systems such as Subversion

Users who have checked out the repository can edit files, and check in the new versions with the commit command; to get the changes committed by other users you use update.

One of the uses of committing is that you can roll your code back to an earlier version if you realize you made a mistake or introduced a bug. It also allows you to easily see the difference between different code version. However, committing many small changes may be confusing to other developers, for instance if they come to rely on something you introduce which you later remove again. For this reason, distributed source code control systems use two levels of repositories.

There is still a top level that is authoritative, but now there is a lower level, typically of local copies, where you can commit your changes and accumulate them until you finally add them to the central repository.

Workflow in distributed source code control systems such as Mercurial

This also makes it easier to contribute to a read-only repository: you make your local changes, and when you are finished you tell the developer to inspect your changes and pull them into the top level repository. This structure is illustrated in figure  2 .

26.2 Subversion or SVN

Top > Subversion or SVN

This lab should be done two people, to simulate a group of programmers working on a joint project. You can also do this on your own by using two copies of the repository.

26.2.1 Create and populate a repository

Top > Subversion or SVN > Create and populate a repository
Purpose

In this section you will create a repository and make a local copy to work on.

First we need to have a repository. In practice, you will often use one that has been set up by a sysadmin, but there are several ways to set up a repository yourself.

You now have an empty directory project.

\practical{Go into the project directory and see if it is really empty.}{There is a hidden directory .svn}{}

The project directory knows where the master copy of the repository is stored.

\practical{Do svn info}{You will see the URL of the repository

and the revision number}{Make sure you are in the right directory. In the outer directories, you will get \n{svn: warning: '.' is not a working copy}}

26.2.2 New files

Top > Subversion or SVN > New files
Purpose

In this section you will make some simple changes: editing an existing file and creating a new file.

One student now makes a file to add to the repository\footnote{It is also possible to use svn import to create a whole repository from an existing directory tree.}.

a
b
c
d
e
f
^D
This file is unknown to svn:
?       firstfile
We need to declare the file as belonging to the repository; a subsequent svn commit command then copies it into the master repository.
A         firstfile
A       firstfile
Adding         firstfile
Transmitting file data .
Committed revision 1.

\practical{The second student can now do svn update to update their copy of the repository}{\n{svn} should report A firstfile and Updated to revision 1.. Check that the contents of the file are

correct.}{In order to do the update command, you have to be in a checked-out copy of the repository. Do svn info to make sure that you are in the right place.}

\practical{Let both students create a new directory with a few files. Declare the directory and commit it. Do svn update to obtain the changes the other mde.}{You can do svn add on the directory, this will also add the files contained in it.}{Do not forget the commit.}

In order for svn to keep track of your files, you should never do \n{cp} or mv on files that are in the repository. Instead, do \n{svn cp} or \n{svn mv}. Likewise, there are commands svn rm and svn mkdir.

26.2.3 Conflicts

Top > Subversion or SVN > Conflicts
Purpose

In this section you will learn about how do deal with conflicting edits by two users of the same repository.

Now let's see what happens when two people edit the same file. Let both students make an edit to \verb+firstfile+, but one to the top, the other to the bottom. After one student commits the edit, the other will see

Sending        firstfile
svn: Commit failed (details follow):
svn: Out of date: 'firstfile' in transaction '5-1'
The solution is to get the other edit, and commit again. After the update, svn reports that it has resolved a conflict successfully.
G  firstfile
Updated to revision 5.
Sending        firstfile
Transmitting file data .
Committed revision 6.
The G at the start of the line indicates that svn has resolved a conflicting edit.

If both students make edits on the same part of the file, svn can no longer resolve the conflicts. For instance, let one student insert a line between the first and the second, and let the second student edit the second line. Whoever tries to commit second, will get messages like this:

svn: Commit failed (details follow):
svn: Aborting commit: '/share/home/12345/yourname/myproject/firstfile' 
remains in conflict
C  firstfile
Updated to revision 7.
Subversion will give you several options. For instance, you can type \n{e} to open the file in an editor. You can also type p for `postpone' and edit it later. Opening the file in an editor, it will look like
aa
<<<<<<< .mine
bb
=======
123
b
>>>>>>> .r7
cc
indicating the difference between the local version (`mine') and the remote. You need to edit the file to resolve the conflict.

After this, you tell svn that the conflict was resolved, and you can commit:

Resolved conflicted state of 'firstfile'
Sending        firstfile
Transmitting file data .
Committed revision 8.
The other student then needs to do another update to get the correction.

26.2.4 Inspecting the history

Top > Subversion or SVN > Inspecting the history
Purpose

In this section, you will learn how to get information about the repository.

You've already seen svn info as a way of getting information about the repository. To get the history, do svn log to get all log messages, or svn log 2:5 to get a range.

To see differences in various revisions of individual files, use \n{svn diff}. First do \n{svn commit} and svn update to make sure you are up to date. Now do svn diff firstfile. No output, right? Now make an edit in firstfile and do \n{svn diff firstfile} again. This gives you the difference between the last commited version and the working copy.

You can also ask for differences between committed versions with svn diff -r 4:6 firstfile.

The output of this diff command is a bit cryptic, but you can understand it without too much trouble. There are also fancy GUI implementations of svn for every platform that show you differences in a much nicer way.

If you simply want to see what a file used to look like, do \n{svn cat -r 2 firstfile}. To get a copy of a certain revision of the repository, do svn export -r 3 . ../rev3, which exports the repository at the current directory (`dot') to the directory ../rev3.

If you save the output of svn diff, it is possible to apply it with the Unix patch command. This is a quick way to send patches

to someone without them needing to check out the repository.

26.2.5 Shuffling files around

Top > Subversion or SVN > Shuffling files around

We now realize that we really wanted all these files in a subdirectory in the repository. First we create the directory, putting it under svn control:

A         trunk
Then we move all files there, again prefixing all comands with svn:
        svn mv $f trunk/ ; done
A         trunk/firstfile
D         firstfile
A         trunk/otherfile
D         otherfile
A         trunk/myfile
D         myfile
A         trunk/mysecondfile
D         mysecondfile
Finally, we commit these changes:
Deleting       firstfile
Adding         trunk/firstfile
Deleting       myfile
Deleting       mysecondfile
Deleting       otherfile
Adding         trunk
Adding         trunk/myfile
Adding         trunk/mysecondfile
Adding         trunk/otherfile

You probably now have picked up on the message that you always use svn to do file manipulations. Let's pretend this has slipped your mind.

\practical{Create a file somefile and commit it to the repository. Then do rm somefile, thereby deleting a file without svn knowing about it. What is the output of svn status?}{svn indicates with an

exclamation point that the file has disappeared.}{}

You can fix this situation in a number of ways:

  • svn revert restores the file to the state in which it was

    last restored. For a deleted file, this means that it is brought back into existence from the repository. This command is also useful to undo any local edits, if you change your mind about something.

  • svn rm firstfile is the official way to delete a file. You

    can do this even if you have already deleted the file outside svn.

  • Sometimes svn will get confused about your attempts to delete a file. You can then do svn --force rm yourfile.

26.2.6 Branching and merging

Top > Subversion or SVN > Branching and merging

Suppose you want to tinker with the repository, while still staying up to date with changes that other people make.

  <the URL of your repository>/trunk \
  <the URL of your repository>/onebranch \
  -m "create a branch"


Committed revision 11.
You can check this out as before:
          projectbranch       
A  projectbranch/mysecondfile
A  projectbranch/otherfile
A  projectbranch/myfile
A  projectbranch/firstfile
Checked out revision 11.

Now, if you make edits in this branch, they will not be visible in the trunk:

Sending        firstfile
Transmitting file data .
Committed revision 13.
At revision 13.

On the other hand, edits in the main trunk can be pulled into this branch:

--- Merging r13 through r15 into '.':
U    secondfile
When you are done editing, the branch edits can be added back to the trunk. For this, it is best to have a clean checkout of the branch:
A    # all the current files
and then do a special merge:
--- Merging differences between repository URLs into '.':
U    firstfile
 U   .
Path: .
URL: <the URL of your repository>/trunk
Repository Root: <the URL of your repository>
Repository UUID: dc38b821-b9c6-4a1a-a194-a894fba1d7e7
Revision: 16
Node Kind: directory
Schedule: normal
Last Changed Author: build
Last Changed Rev: 14
Last Changed Date: 2009-05-18 13:34:55 -0500 (Mon, 18 May 2009)
Sending        .
Sending        firstfile
Transmitting file data .
Committed revision 17.

26.2.7 Repository browsers

Top > Subversion or SVN > Repository browsers The svn command can give you some amount of information about a

repository, but there are graphical tools that are easier to use and that give a better overview of a repository. For the common platforms, several such tools exist, free or commercial. Here is a browser based tool: \url{http://wwww.websvn.info}.

26.3 Mercurial (hg) and Git

Top > Mercurial (hg) and Git \emph{Mercurial} and git are the best known of a new generation

of distributed source code control systems. Many commands are the same as for subversion, but there are some new ones, corresponding to the new level of sophistication. Mercurial and git share some commands, but there are also differences. Git is ultimately more powerful, but mercurial is easier to use at first.

Here is a translation between the two systems: \url{https://github.com/sympy/sympy/wiki/Git-hg-rosetta-stone}.

This lab should be done two people, to simulate a group of programmers working on a joint project. You can also do this on your own by using two clones of the repository, preferably opening two windows on your computer.

26.3.1 Create and populate a repository

Top > Mercurial (hg) and Git > Create and populate a repository
Purpose

In this section you will create a repository and make a local copy to work on.

First we need to have a repository. In practice, you will often use one that has been previousy set up, but there are several ways to set up a repository yourself. There are commercial and free hosting services such as \url{http://bitbucket.org}. (Academic users can have more private repositories.)

Let's assume that one student has created a repository your-project

on Bitbucket. Both students can then clone it:

updating to branch default
0 files updated, 0 files merged, 0 files removed, 
    0 files unresolved
or
Cloning into 'yourproject'...
warning: You appear to have cloned an empty repository.
You now have an empty directory your-project.

\practical{Go into the project directory and see if it is really empty.}{There is a hidden directory \n{.hg} or .git}{}

26.3.2 New files

Top > Mercurial (hg) and Git > New files

\heading{Creating an untracked file}


Purpose

In this section you will make some simple changes: creating a new file and editing an existing file

One student now makes a file to add to the repository:

a
b
c
d
e
f
^D
(where \verb+^D+ stands for control-D, which terminates the input.) This file is unknown to hg:
? firstfile
Git is a little more verbose:
git status
On branch master


Initial commit


Untracked files:
  (use "git add <file>..." to include in what will be committed)


firstfile


nothing added to commit but untracked files present
  (use "git add" to track)

\heading{Adding the file to the repository}

We need to declare the file as belonging to the repository; a subsequent hg commit command then copies it into the repository.

A firstfile
or
On branch master


Initial commit


Changes to be committed:
  (use "git rm --cached <file>..." to unstage)


        new file:   firstfile
  [master (root-commit) f4b738c] adding a first file
  1 file changed, 5 insertions(+)
  create mode 100644 firstfile

Unlike with Subversion, the file has now only been copied into the local repository, so that you can, for instance, roll back your changes. If you want this file added to the master repository, you need the hg push command:

pushing to https://YourName@bitbucket.org/YourName/your-project
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: bb/acl: YourName is allowed. accepted payload.
In the push step you were probably asked for your password. You can prevent that by having some lines in your \verb+$HOME/.hgrc+ file: {\footnotesize
[paths]
projectrepo = https://YourName:yourpassword@bitbucket.org/YourName/my-project
[ui]
username=Your Name <you@somewhere.youruniversity.edu>
} Now the command hg push projectrepo will push the local changes to the global repository without asking for your password. Of course, now you have a file with a cleartext password, so you should set the permissions of this file correctly.

With git you need to be more explicit, since the ties between your local copy and the `upstream' repository can be more fluid.

git remote add origin git@bitbucket.org:YourName/yourrepo.git
git push origin master

The second student now needs to update their repository. Just like the upload took two commands, this pass also takes two. First you do \n{hg pull} to update your local repository. This does not update the local files you have: for that you need to do hg update.

\practical{Do this and check that the contents of the file are correct.}{In order to do the update command, you have to be in a checked-out copy of the repository.}

\practical{Let both students create a new directory with a few files. Declare the directory and commit it. Pull and update to obtain the changes the other mde.}{You can do hg add on the directory, this will also add the files contained in it.}{}

In order for Mercurial to keep track of your files, you should never do the shell commands \n{cp} or mv on files that are in the repository. Instead, do \n{hg cp} or \n{hg mv}. Likewise, there is a command hg rm.

26.3.3 Conflicts

Top > Mercurial (hg) and Git > Conflicts
Purpose

In this section you will learn about how do deal with conflicting edits by two users of the same repository.

Now let's see what happens when two people edit the same file. Let both students make an edit to \verb+firstfile+, but one to the top, the other to the bottom. After one student commits the edit, the other can commit changes, after all, these only affect the local repository. However, trying to push that change gives an error:

abort: push creates new remote head b0d31ea209b3!
(you should pull and merge or use push -f to force)
The solution is to get the other edit, and commit again. This takes a couple of commands: {\small
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)


merging firstfile
0 files updated, 1 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)


M firstfile
pushing to https://VictorEijkhout:***@bitbucket.org/
                       VictorEijkhout/my-project
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 2 changesets with 2 changes to 1 files
remote: bb/acl: VictorEijkhout is allowed. accepted payload.
}

This may seem complicated, but you see that Mercurial prompts you for what commands to execute, and the workflow is clear, if you refer to figure  .

\practical{Do a cat on the file that both of you have been editing.

You should find that both edits are incorporated. That is the `merge' that Mercurial referred to.}{}{}

If both students make edits on the same part of the file, Mercurial can no longer resolve the conflicts. For instance, let one student insert a line between the first and the second, and let the second student edit the second line. Whoever tries to push second, will get messages like this: {\small

added 3 changesets with 3 changes to 1 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
merging firstfile
warning: conflicts during merge.
merging firstfile incomplete! 
    (edit conflicts, then use 'hg resolve --mark')
0 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges 
    or 'hg update -C .' to abandon
} Mercurial will give you several options. It is easiest to resolve the conflict with a text editor. If you open the file that has the conflict you'll see something like:
<<<<<<< local
aa
bbbb
=======
aaa
a2
b
>>>>>>> other
c
indicating the difference between the local version (`mine') and the other, that is the version that you pulled and tried to merge. You need to edit the file to resolve the conflict.

After this, you tell hg that the conflict was resolved:

 hg resolve --mark
M firstfile
? firstfile.orig
After this you can commit and push again. The other student then needs to do another update to get the correction.

Not all files can be merged: for binary files Mercurial will ask you:

merging proposal.tex
merging summary.tex
merking references.tex
 no tool found to merge proposal.pdf
keep (l)ocal or take (o)ther? o
This means that the only choices are to keep your local version (type~\n{l} and hit return) or take the other version (type~o and hit return). In the case of a binary file that was obvious generated automatically, some people argue that they should not be in the repository to begin with.

26.3.4 Inspecting the history

Top > Mercurial (hg) and Git > Inspecting the history
Purpose

In this section, you will learn how to get information about the repository.

If you want to know where you cloned a repository from, look in the file .hg/hgrc.

The main sources of information about the repository are hg log and hg id. The latter gives you global information, depending on what option you use. For instance, hg id -n gives the local revision

number.

  • [\texttt{hg log}] gives you a list of all changesets so far, with the comments you entered.

  • [\texttt{hg log -v}] tells you what files were affected in each changeset.

  • [\texttt{hg log -r 5}] or hg log -r 6:8 gives information on

    one or more changesets.

To see differences in various revisions of individual files, use hg diff. First make sure you are up to date. Now do hg diff firstfile. No output, right? Now make an edit in firstfile and do \n{hg diff firstfile} again. This gives you the difference between the last commited version and the working copy.

Check for yourself what happens when you do a commit but no push, and you issue the above diff command.

You can also ask for differences between committed versions with hg diff -r 4:6 firstfile.

The output of this diff command is a bit cryptic, but you can understand it without too much trouble. There are also fancy GUI implementations of hg for every platform that show you differences in a much nicer way.

If you simply want to see what a file used to look like, do \n{hg cat -r 2 firstfile}. To get a copy of a certain revision of the repository, do hg export -r 3 . ../rev3, which exports the repository at the current directory (`dot') to the directory ../rev3.

If you save the output of hg diff, it is possible to apply it with the Unix patch command. This is a quick way to send patches

to someone without them needing to check out the repository.

Back to Table of Contents