Wednesday 2 May 2012

Virtualizing, Installing and Using Subversion - Complete Tutorial


Introduction
This weekend, I was helping my mate set up an SVN server for his use, he wanted to share work on a project between himself and a third party, including the resources and code... So we jazzed together a desktop install of Ubuntu and set about setting up the server aspects.  So he has a nice desktop environment but also the code held locally in a respository.  I'll recap this in a mo.

Problems arose however when he posed a few innocent little questions, like "So how do I create a branch"... and "How do I keep his work/resources/sound/graphics/ideas/documents [delete as appropirate] separate to mine?"

I realised I needed to give him a little intro to SVN, not just set it up and leave him loose to his own devices, but to actually induct him into an idea of how best to put the software to good use, to actually make the time spent now productive for his project.  So, I set about showing him from Tortoise SVN (on windows - as its the quickest way to get started) to work with the repository browser, to set up folders and areas and parts within the respository, how to pull code, commit, branch, reintegrate and how basically I have come to work and view subversion after nearly ten years working in groups employing it as the main source control method.

This post is the result of that effort and to help recap all that was discussed, and to help you guys & gals out there do the same. 

Everything will be in sections, firstly setting up a virtual machine with an install of Linux (Ubuntu), then setting up Subversion itself within that virtual machine, and then actually communicating with and working within that repository from our main code machine (a windows machine).

Installing Ubuntu in a Virtual Machine
You can either download then burn a DVD and install your Linux box on some spare box you have lying around, or if you have plenty of ram (4GB+) and a couple of CPU cores you can at least host a virtual machine - I'm going to show you how to set up Linux just like that, as a virtual machine in VMWare Player (really this could be a whole post on its own, but I'm going to jam it in here).

I'm assuming you're starting from a windows machine.

1. Download VMWare Player from www.vmware.com - its free you just need to register an e-mail address, confirm your mail and use the download link they provide.

2. Download the Ubuntu ISO (I'm going to use the a desktop version of Ubuntu, but you can use either) from www.ubuntu.com - again its free and you don't even need to sign up for this!

3. Ensure you have around 10gb of hard drive space free - a virtual machine will need a decent amount of disk space, 10gb is the bare minimum for Ubuntu, you may want more however as your project files will go into Subversion on the virtual disk file we'll create.

4. a. Install and start the VMWare Player program - the installer should do everything for you - when its done, you should be presented with "Welcome to the VMWarePlayer" you need only select "Create a New Virtual Machine" on the right...


4. b. To install Vmware player on Ubuntu you need to download the Vmware player installer - this will come down with a .txt file extension, open a terminal, and change into the directory you downloaded the script to and run "sudo sh ./VMware-Player-#####.txt", that'll let the shell run the bash script to perform the install for you.

5. I always set up the VM image and then install the OS, so my next option is "I will install the operating system later."



6. If you've downloaded the 64bit Ubuntu Image, then use the "Version" drop down to select the 64bit Ubuntu option, otherwise; like me; use just the vanilla "Ubuntu" option - this is the 32bit virtual machine - you can't host a 64bit virtual machine on a 32bit host computer - so I'm sticking to 32bit across the board to make sure I cut down confusion.



7. Next select a path where you want to store the virtual machine:



8. The important part the size of the image, I'm going with a small size, as this is only an example - you may want 20+gb of space in the virtual machine - you can make the virtual machine disk image larger later, and you can copy the virtual machine onto a different drive and enlarge it later so if you're restricted right now, don't sweat it.



9. The final step to configure the image lets you play about with what virtual resources the machine will have, it comes up with a lot of options, many of which you don't need...e.g. a Floppy drive.  You can chose to remove these virtual devices or just leave them for now.  One thing you may want to do is change the amount of RAM being used by the virtual machine, I try to make sure a Ubuntu server has about 512mb of RAM, but a desktop version needs around 1GB to run properly.  You can change this later to use as much, or little, as you like.




10. Once you're happy, click Finish, and the virtual machine will appear in the list available, on the right (when selected on the left) you will see that the machine is off and has whatever settings.  You can edit the settings here, if you think you gave it too little ram etc.  However, we need to now install the OS, as that virtual machine image is utterly blank.

11. Select to "Edit Virtual Machine Settings".

12. Select the CD-ROM/DVD Drive you have in the virtual machine - or add one if its missing.


13. Select to "Use ISO Image file" as the drive and browse to the installation ISO you downloaded in step 2.  Once you're happy, select "Okay" and then "Play Virtual Machine".


14. You will probably get asked about USB devices to connect to the virtual machine and definitely asked about updating the VMWare tools.  For now, ignore the USB stuff and select "Remind Me Later" for any updates.


15. The Ubuntu CD should soon have loaded into your virtual machine, to install Ubuntu just follow the prompts on screen, getting a username and password - please remember the password you enter we'll need it very soon.  Once you've got the virtual machine set up and it has booted into the desktop have a look around - and pat yourself on the back you've got a major step of this tutorial done!  - You also now know how to set up a virtual machine to try other Linux distributions, or other OS's!! -


16. When the install is over and the installer asks you to "remove the installation media" simply go to the top of the player and use the menu "Virtual Machine" (or press CTRL+D) to access the settings again, drop into the CD/DVD and set it to "Use Physical Drive", so the ISO won't be used any more, and make sure you don't have a CD/DVD in your physical drive :)


You can then reboot the virtual machine by pressing ENTER and it'll start back up... Voila, you have your virtual machine all working in the VMWare Player!!


Installing Subversion - Recap
Assuming you have Ubuntu installed and running either for real or as a virtual machine you can now open a terminal prompt by selecting the virtual machine (so your mouse and keyboard are directing their input into the playing virtual machine) and CTRL+SHIFT+T.

With the prompt up, we want to do the following commands, in this order:

For now just blindly follow this example, my other post regarding setting up SVN, goes into detail, so go search "SVN" within this blog and you'll find it all explained!!

sudo apt-get install apache2 subversion libapache2-svn
sudo svnadmin create /svn
sudo nano /etc/apache2/mods-enabled/dav_svn.conf

Make the file look as follows:

<Location /svn>
  DAV svn

  SVNPath /svn

  AuthType Basic
  AuthName "Subversion Repository"

  AuthUserFile /etc/apache2/dav_svn.passwd

  Require valid-user

</Location>

Save (with CTRL+O), and exit (with CRTL+X).  If you don't like using nano, try using "sudo gedit /etc/apache2/mods-enabled/dav_svn.conf" to use the visual geditor built into the gnome desktop, or if you're on a KDE based distro "sudo kate /etc/apache2/mods-enabled/dav_svn.conf".

UPDATE (27/4/2016): Ubuntu 14.04 Server - If you are using the server edition, especially 14.04 I've found that htpasswd is missing, or "not found", you need to install it with the package "apache2-utils", therefore:

sudo apt-get install apache2-utils

Next we need to create users in the password file we've just pointed the service at, so for the first user, we create the file and the username (here username1 - could be anything - "Eric" for example).

sudo htpasswd -cm /etc/apache2/dav_svn.passwd username1

We also add another user - but this time there is no need to "create" the file so loose the 'c' flag;

sudo htpasswd -m /etc/apache2/dav_svn.passwd username2

Next we need to restart apache to take all the new settings

sudo /etc/init.d/apache2 restart

Finally we need to let apache own the /svn folder, where our repository is, so we set the ownership and allow read/write access to the www-data user thus:

sudo chown -R www-data:www-data /svn
sudo chmod -R 770 /svn

Again restart apache:

sudo /etc/init.d/apache2 restart

Now we need to test whether the virtual machine is able to talk to the outside world, well, really to keep things simple, we'll just talk to the host computer, so we need to tweak our virtual machine a tad, so pop into the settings again and select the network adapter and opt for "Host-only: A private network shared with the host".


Once set, either wait a moment, or peform:

sudo /etc/init.d/networking restart

And then run:

ifconfig

With this output you're looking for the "eth0" item and on its second line you should see "inet addr:" and then an IP address value, mine reads 192.168.182.128.  This is the IP address your host computer can see the Virtual machine on, so you can minimise the whole VMPlayer now and open a command prompt in windows:


We can quickly try the apache server, on my host computer (windows) I open an internet browser, I prefer to use Chrome, and I type in the http:// and whatever the IP is, to see the web server running in the virtual machine:


Next check the SVN repository is working by adding "/svn" at the end of that same URL.  Now, this will prompt for the username and password we added as an SVN user to the dav_svn.passwd file above.  Because we set the repository to "require valid user" it will always prompt like this, even for reading from the repository, this is important to keep your code your own and not downloadable if you expose this virtual machine to the internet.

So, lets add the /SVN and see what my browser does:


Lets enter username1 from above and its password:


Voila, its working... Our repository is empty, at revision zero, and ready for use... Lets download a client to help us put it to good use.

Since we're using windows, I'm going to use one of the better ones - Tortoise SVN - its not perfect, indeed, the tools are sometimes quite flakey, but we need to learn, so lets delve in.


Using Tortoise for Subversion Team work
First things first, take a deep breath, if you've followed this whole tutorial in one chunk up to now, you've got a lot of work done and maybe learned a lot of new things, so take a breather, put the kettle on, relax a mo, whilst the kettle boils, on your Windows machine go to http://tortoisesvn.net/downloads.html and download the client you need, either 32bit or 64bot according to the needs of your Windows variant... Install that application, and then lets sit down with that cuppa and relax a moment.

Before we touch the mouse, we need to think about the structure we want in our SVN repository... I want a main code area called "trunk" and then a folder called "branches" and under "branches" I want a folder for each user to put their branches.  Then I want a separate area to put resources, and under that two subfolders called "graphics" and "audio".

How the heck can I put all that into the repository?  Well, right click on the desktop, or any folder on your Windows machine (which has tortoise installed) and you will see two new SVN related items in the mouse menu... 


We have nothing to check out at the moment, but under the other area you will see an expanded load of tools:


The one we're interested in is the "Repo browser" it allows us to connect to our repository to create and move folders around, create back ups and generally nosey around on the data stored in our repository.

So, click the repo browser and enter the "http://192.168.182.128/svn" URL as you used before in the web browser...


You'll be asked to prove you are a valid user again, so enter the username details as you did to set up the server...


So we can see the inside of the repository, and there should be nothing there...


However, with the folder on the left we can right click the mouse and do several things, one of them is create folder:


Lets create the folders I just mentioned... When you do, you'll be asked to enter a comment or log message for each action, its probably best you get into the habit of actually entering useful information for your log messages...


The log message will be saved into the repository against this change (revision) under the username you used to gain access to the site, so if from above we have two users (username1 and username2) and the first enters a folder there shall be a comment labelled by them, if the second user performs a change, likewise, the change will be labelled by their name.  Keeping track of who did what when...

If you're working in a group, its probably best each user come to the server to enter their own password, that way they can't blame one another for items performed under their usernames... In fact "Blame" is one of the tools inside SVN itself :)

Anyhew, lets add the rest of the folders and sub-folders each with a comment... If you come to add a similar folder, which needs a similar comment, you can use the "Recent messages" button to see the last lot of used comments by yourself and so re-use a comment (with a tweak to make it obvious its not just a repeat message).

When you're done, you should see the respository looking how you want it, for this example mine now looks like this:


You will notice that each change to the repository gets a new revision number, so when we just added the trunk folder, that should have been revision 1, therefore on the right now you can see my "trunk" has a "Revision" of 1, branches I did a little later it is the seventh thing I did so it is "Revision" 7... We'll talk about this later, but if I now pulled just "Revision" 1 of the repository it would not look like the image above, it would only contain one thing the "trunk" folder, because at revision 1 that was the only thing I had created... If I pulled just revision 5 then I would get the trunk and the resources folder with the audio and graphics subfolders as they're what I had done by revision 5.

You can never loose a revision, so if you have a file present in one of the folders, and that is revision 9.  But for revision 10, you delete the file, then the file is gone from the top most layer... But you can still get it back, just by looking at revision 9...

That may seem a little mystic, but for now trust me, you can't loose anything once its in the repository, not even if you chose to delete it!

So, armed with our folders, lets close the repository browser and create a folder somewhere.  I'll create my folder called "Code" on my desktop, this is just an empty folder...


What I want to do is make this new folder a "copy", actually a working copy, of the repository as it looks now, to do this I right click on the folder and choose "SVN Checkout" from the mouse menu...

The tool should remember the last SVN location we went to, so automatically finds the URL to the repository on the virtual machine.  And it says we're going to checkout the whole repository to the "Code" folder on my desktop... To the right of the URL you can see a button marked "..." this button lets you use a miniature version of the Repo Browser to select just one folder, or just one branch, or just the trunk from the repository... For now however leave it reading "http://192.168.182.128/svn/" for the whole repository.


Go a head and check it out... It'll most likely ask you to prove you're a valid user again...


The check out screen then tells us what has happened, the subversion software has added all the contents from the server into the folder we clicked on, so now we can work in the folders on our desktop to add files, create code, add graphics or whatever and commit them back to the server.

You'll also notice that the folder now has a big green tick on it, this shows us that folder is in sync with the server, that is we have pulled items from the repository but not yet changed anything. Don't trust this indicator 100%, over time the service responsible for keeping the indications up to date can get a little lax and it can also ignore some changes as "unchanged" e.g. remove a whole checked out file - and it sometimes thinks nothing changed.

Also if like me you are a heavy machine user sometimes the icons used can get mixed up.  For example, at the moment the green tick on my main development machine is actually being shown as an icon from a popular game franchise from CCP... For no apparent reason.


If you delve into the folder, you'll see each subfolder as you created them on the repository, each also with a tick, you can add files to one folder and that folder will show as changed in its own right, letting you manage which parts of the respository change as you commit data back up to the server.


One thing you can't spot at the moment, least on my folders, is a hidden folder called ".svn", lets show all the hidden files in the folder... Though, it seems tortoise may also store these hidden files elsewhere.


Now we can see the ".svn" folder....


Don't ever mess about with these files, also make sure if you move files from one place to another within your checked out folders you don't move this hidden folder as well, that can cause real problems!  So, if you have /Code/trunk/.svn and you move it to /Code/branches/ then you will screw up all the checked out information about the files, loose cohesion and generally break the checked out copy and have to check out a new fresh copy into a new folder somewhere on your machine.  So, rule 1, don't move the .svn folders.

But, lets go create a few files... We're getting this project started, so lets go into the trunk and create a program code file....


You can see the file is now showing, but its icon has a blue marker, a "?" in fact, this is because there's no record of that file in that folder's .svn information, a service on your windows machine has parsed the folder over and decided that something changed, the marker is telling us that tortoise wants to know what to do with that file...

We want to add that file to the respository, go a head, right click on the file itself and chose to Add it to the repository, thus:


The icon should change to a blue "+", because the SVN service knows what you want to do with the file... If yours does not appear as a "+" quickly you may need to press "F5" to refresh the folder view, or exit the folder and come back in - this again is one of the poor parts of this tool set, but a common problem amongst SVN services as they try to keep track of what you're doing without crippling your machine for performance.


If we now go up a level we can see that the folder shows as changed...


So, we know we must commit the changes to that folder into the repository, we could click on just the "HelloWorld.cpp" file and do this, or click on the folder, I'll do the latter...


I'm going to Commit these changes up to the server...


The commit page asks for a message for the log, and shows me what I'm going to do, that is commit the "added" file to the repository... Once I select "Ok" I'll again be asked to prove I'm a valid user, and the file will commit across the link to the respository...


You'll note that each file in the commit page has a check box to its left, if you uncheck that file, it'll not be commited to the repository.  Likewise, as we'll see shortly, if you delete a file they are automatically NOT checked, so you have to check a file to implicitly tell the repository to remove the file.

In our example though you'll see the file get added to the repository first - you added it to your folder, so it gets added to the folder on the repository - and then the contents of that file, the actual code within, will be sent over the network and you'll see that the repository revision has changed.

We can now view our code on the server by hopping back to our web browser and entering the URL:

http://192.168.182.128/svn/ and then we can hop about inside the respository, even viewing the code itself, this is a great way for new users to get to know your code base, as they can view without changing anything.


Right, well now we have a file in our respository, lets do something radical, something powerful... lets delete that Code folder we created...


OH MY GOD we just lost all our work, our hard drive crashed, or the cat pressed shift delete.. ZOOOOMG we've lost our code!?!?!!... No, no we've not, create the folder again, lets call it "Code2" this time, right click on it and check out again...



Voiala, the folder is restored, your work is back, the HelloWorld file is back in trunk on your machine, you see you only deleted your local working copy, you didn't delete it from the server.

The only way you could loose work from your working copy, is that if you've changed a file (it shows the red exclamation) or you've created a new file - but not added it - and then not commited them back to the server.  If SVN did not know about an addition, or a change, then you can't blame it for loosing information.  The process of updating the server is not automatic, instead you the developer, you the driving intelligence of your computer, you have to make it do something and that somthing is commit Commit COMMIT!!!

Very quickly, now we need to remove that HelloWorld.cpp file, its rubbish, so go to your working copy "/Code2/trunk" and delete the file.  You may, or may not, see the folder show a red exclamation, but the file has been deleted... If we now move up a level, right click on the "trunk" folder and commit we'll be asked whether we meant to delete that folder...


The file shows as "missing" in the Changes made section, if we put a check in that check box against the file and continue the commit, then the file will be removed from the repository's next revision... i.e. revision 9 will not contain that file - revision 8 still will :)

However, we didn't mean to remove that file just yet... So cancel.

And now right click on the "trunk" and "SVN Update" the folder, this will bring the file back as we're still on revision 8, and revision 8 contains that file!


Voila, the file is "Restored"


But, oh dang, we really did mean to delete it, so this time lets delete the file and commit the file as really actually missing/deleted...



The file is Deleted from the repository in the next revision, which has gone up from 8 to 9 now.

If you use the web browser now, you're going to see revision 9, so the HelloWorld.cpp file has been removed, but fear not, it is still there on the server, just as revision 8 which you can't see.

That's enough of that single file stuff though, lets create a real project, I'm going to use Visual Studio to create a new project in the trunk folder, I'm going to add a little code to it and compile the project and then decide which parts of the generated list of files I'm going to add and commit to the respository...


When I'm resented with the Add now (after right clicking on the folder inside trunk created for me) everything comes up checked on, however, we don't want to commit certain things to our respository, for example in our code we don't want to send the "binary" folders, we want each developer (when they pull the code) to check & build the code themselves) so I've unchecked the "/bin" and the "/obj" folders - the tool has automatically unchecked all the subfolders and files in those two paths for me.  I have also removed the SUO file (if you're working with Visual Studio please note you don't want the SUO from one machine going onto the folder used by another).

So above we can see the folders for the project have been added, the project file & solution file as well as the code files as generated.  I'll click okay...


Remember, this means that the files in your working folders have been added to the local .svn information, they're not yet on the server.... If you deleted the files now they would be lost and your .svn data would not match what was expected... So don't loose information, commit this data now by right clicking and selecting "SVN Commit".


You'll see now that the folders were created on the respository (just as if you had used the repo browser) and the files each added to them on the server and then the file contents transmitted.  You'll notice that things like the folders sent no data they were created with commands not data, only the black lines of text are actual transmissions of data.

The revision has gone up a step too, from 9 to 10.  And we have just learned we can send whole bunches or tree's of files to the server in one go!

We could delete the whole tree too, if we delete the "HelloWorldApplication" folder from the "trunk" folder and commmit that change we'd move the server onto revision 11 which would have no record of the files or folders in this project... But (just to emphasise the point) revision 10 would still exist and it would have all your code!

Now, lets pretend that there are two people working on the project, they could pull a copy of the "trunk" folder and work on it and then commit their changes... Then you could pull a copy and work on it and commit your changes... Working one after the other, making sure you update/pull your copy after the other person has done working is a valid, if very awkward, way of working.

But what happens if you've both pulled a copy and both worked on it and then both try to commit...

Well the person to "commit first" would get their changes into the repository and the second person would find their work showing as "Conflicted".

Lets demonstrate this with a few pictures, I'll take another working copy of the example above, so we have a copy in "/Code2" and a copy in "/Code3".

Both are revision 10 right now... I start to work in "Code2"... 


And I commit those changes to the server... The server is now revision 11...

But the other folder "/Code3" is still Revision 10... If I start to work on there...


It itself knows nothing of the changes already made from the other working copy (or it could be another machine - or another developer)... So when I come to commit those changes...


We get an error, this is telling us our working copy was out of date compared to the one on the server... We should update from the server before doing any changes...

To rectify this situation we pop into "/code3/trunk/" down to the files we've changed - we note down what we changed - or take a copy of our changes somewhere else, just file copy them somewhere for reference... Then we delete the local conflicted/edited files, so anything with a red exclamation we delete... Then go back up to the "Code3" folder and right click and "SVN Update"... This will bring in the changed files from the server and show us that its revision 11 now...


We can then sit and with the reference copies of our working code splice it back into the project and check everything is working correctly.

But this is a really really BAD way of working, its terrible for your sanity and morale, its infact more than BAD its WRONG WRONG WRONG, so don't so it...

Never work on the same folder on the server as someone else, what we need to do is learn how to make branches...

A branch is a copy of the source taken at a certain moment in time for work which keeps your work separate from others and then you can merge those changes back into the trunk later.


Generally speaking there should be just one person responsible for merging branches back together, any developer on the team can take a branch of an existing trunk/folder/branch and work on it, but the owner of that branch, the software architect as it were, should be responsible for merging them back into one unified form.

That merging process should be watched over, checked, and once complete builds and rebuilds should be performed and testing conducted to ensure the validity of the end result.  Don't just blindly believe the process was right, its software, software only does what you (the user) have told it to do, that could be the wrong thing!

Another strength of the branch approach of working is that you, say you are working from a common root - in this example "trunk" - and someone  else delivers some part to merge into the trunk.  Theirs can be merged, tested and committed as working and then you may pull that updated trunk information up into your branch,  meaning you can keep a breast of changes and ensure your current work is still valid with the over all project.

So, branches, very powerful, very strong things.  Merging has to be polliced but it easily opens up the most powerful aspects of subversions use in a team environment.

How do we make a branch then?

Well, first of all, you'll remember the folder we created in the repository called "branches", that's where we're going to branch to... The basic idea is to pick something (so in our example the trunk) and branch it to somewhere else... The effect of this is to take whatever it is, as the latest revision, and branch it off somewhere else for working on its own... You do all this on the server, within the repository, the key to accessing it though is having a local working copy.  So once you've taken a branch on the server you have to either pull that branched code or switch your local working copy to that branch... Lets do some of these operations.

First off, create a new folder on the desktop called "trunk" and check out the trunk into that folder...


Now this working copy we want to branch on the server, into our branches area under "username1"... So, we right click on the folder, and access the tools and choose to branch...



Don't forget to add a meaningful log message into the window...



If you get an error, this means you just selected a folder which exists (I just did this, we all make mistakes)... What our target URL should have been is the branch folder for the user, plus some name...


The "example_branch" there, I typed into the box, after I had selected the root URL as my branches/username1 area.


Because we skipped switching the folder we clicked on (aka our working copy) to that branch, we get told so by the check out copy box.

To switch to that branch on our working copy, we could simply, right click on the folder and opt to switch, using the repo browser select the new branch folder and voila, it would switch for us and when we check in from that folder we'd no longer be committing to "/trunk" we'd be sending to "/branches/username1/example_branch".

But the folder has the name "trunk", how very confusing... Lets delete the trunk folder and check out the branch itself...


You'll notice when this checks out that unlike the trunk, the branch has a different revision (one up in fact), this is because you performed an operation, you took the branch, so there is now a version of the whole repository with your branch (rev 12) and a revision without (rev 11).

With the check out complete I have a folder to work in which has a much more meaningful name, its named after the branch I'm working on... Lets just hop in there and change some of the code, so we've done some work on our branch.  Change a file and commit that change.


So, we've done with our branch, we've done some work... Lets reintegrate it into the trunk...

This is where we do things "backwards", we have to take a new copy of the trunk... a fresh, clean copy of the target location, we bring this copy onto our working machine... So go a head and check out a new copy of the trunk onto the desktop in a folder called "trunk".

Then right click on this folder and chose "Merge" from the SVN tools menu...


For this merge, we know we took the branch, we worked on it and no-one else has worked on the trunk, so we know we just want to reintegrate the branch into the trunk...


This next page is the key, we need to select where we're going to merge code into our working copy of the trunk from... Luckily the tools are a little clever here and they generally guess the correct location (as you can see)... However, they may not, you can use the "..." button again to access the repo browser and choose a location to take source...

And you can see the target location is NOT on the server, its actually the working copy of trunk on my machine... We are going to merge from the server into a local copy of our trunk.

Until we commit any changes nothing has changed, if you think anything went wrong simply delete the folders and start over, everything is still on the server!!!


If we hit "Next" then we get more options, and probably the most important tool for merging... "Test merge".


Taking this screen from the top:

  • "Merge Depth" is where on the local machine we're going to merge down to...
  • Ignoring ancestry is a way to alter the three way merge of the code being carried out...
  • Ignoring line endings is important when bringing code from a different platform .e.g Linux onto Windows, the line endings are different, merging changes between them may result in EVERY file showing a change, even when no real work has been performed upon them.
  • White space compare... Sometimes you may want to record a re-factoring or tidying of the code... sometimes not... Just as with line endings, some editors can be set to remove white space at the end of files, this may result in lots of white space being flagged up as having changed, when really its not.
I'll leave you to explore those merge options... Because, we can merge again and again with nothing happening thanks to the "Test Merge" button.

A test merge will happen, carrying out the merge on the project and showing you the programmer what has changed between the two files.  So, in our example here I only changed the one code file, if I perform a test merge and I see other files changing, I can stop the merge operation and check why those files; supposedly unchanged; are showing changes.

So, for now click test merge....


That looks like the merge is correct to me, only the one file changed and nothing was flagged up as a problem.  I'll click "okay" on the test merge and carry out the real merge.


Once complete, you will see the trunk folder, aka the target of our merge, now shows up as changed... 


Remember, even though we've performed the merge what we've done is merge the code on our branch on the server, into a local copy of a different branch/trunk checked out on the working machine...

So, now I could run up that changed local trunk and check everything is okay.  This is important.

ALWAYS CHECK YOUR MERGED CODE AFTER THE AUTOMATIC TOOLS HAVE WORKED!!

Check it builds, check it is correct, and check the files which changed in the merge actually are still valid code.  Merging can go wrong, its a very dumb operation to splice the changes together and nowhere near as clever as you are, so go and look at it, go check it, go make sure its all correct.

Once you're happy, THEN commit your merged copy of the trunk back into the server.


Give it a decent log message, explaining what you did in the merge, what revision and branch you merged into the trunk, what revision the trunk was when you did this merge, what tests you performed on the merge result before commiting as this new revision of the trunk.

We've merged that branch we took now, so lets use the repo browser, lets go into the repo select that /branches/username1/example_branch and remove that folder from the repository.


Again a useful message, explaining you are removing this branch because you've merged and tested and are happy with it is nice.

The younger a branch is when you come to merge it back into the source from which you spawned it, the easier merging is.  Its no use taking a branch at the beginning of a project and then expecting after two years of work to merge your work back into the trunk.  You need to decide yourself how long, or how much, work you allow on a branch before taking a fresh branch, but you can start to implement your own working methodology, you can give branches interesting names and start to think about using an end to end build system.

That's the very basics of branching, when working alone on a system like ours that's all you need to know really... But... What if there are two programmers?... Well, lets run through that scenario now.  First of all, take branch of the trunk into the "username1" area called "branch1" and another branch of the same trunk into "username2" area called "branch2".


Note, that I don't switch either of them.

You can now delete the trunk and the other old working copies... We need to take out check outs of each of these two new branches, lets do one at a time... take "branch1" do some work on it and commit that work... then take "branch2" do some different work on it and commit that work...


Now, both these changes can be committed...


As you can see, both got different revision numbers...

Now delete both those working folders, and check out a clean copy of the trunk, remember before we said only one person should be responsible for merging at anyone time, well this is that time...

You've been tasked with merging the changes of branch1 into the trunk, so, check out a clean copy of the trunk, select merge, reintegrate branch, browse to the branch1 folder, perform a test merge, see no problems and accept the merge...

Remember, until we commit nothing is on the server... So go a head and commit your merged trunk now.

Make sure you have a nice clean, new if you wish, trunk checked out... because, you've just had a mail telling you the other programmer is ready to merge their work, they have branch2. 
Oh dear we commited our merged trunk... This means the source they added too is deemed out of date... If we just try to repeat the merge as we did with branch one, we will be told there is a conflicted file, because two people worked on the same file.

Note; If the two worked on separate files there would be no conflict, but that really is dicing with death when it comes to merges because inevitably more than one person can work on the same file...


So, what the heck do we do now?...

Well, first of all, let me show you what NOT to do... DO NOT:

  • Take a copy of the trunk.
  • Select to Reintegrate the Branch2 into that trunk.
  • Accept the Merge
  • Commit
Because this will result in the branch2 code simply overwriting the branch1/trunk hybrid code.

What you actually want to do, is very scarey... You need to do a merge which ends up conflicted, what this does is bring in all the code changes from the source branch which are nothing to do with the files both the original and new branch have changed... but then will make the files which are mutually changed by both parties as "Conflicted".

What we then need do is resolve the conflicts.... As I said before, you're a lot smarter than the merging software and this is the main demonstration of that, you as the programmer have to go through the conflicts and resolve them either to use the original, use the new one from the branch, or to edit both pieces of code together and use the one in the working copy.... Once you have cleaned up your merged working copy you can commit it back to the repository as the new trunk...

So, how do we do this?  First of all, take a working copy check out of the branch the other programmer was working on, so in our example this is branch2.  I'm going to check out a copy of that... Then access the merge tool on that copy...


What we're going to do it merge a range of revisions into this branch...


We're going to merge all the trunk as it stands now into this working copy...

When we perform a test merge on this, we get conflicts, this is to be expected, the trunk copy has been changed by the merge of branch1 into it...


We need to accept these conflicvts... So close the test merge, and opt to merge...

For each file it will not ask you what you want to do with the conflict... If the file is NOT a code file, then generally I accept the repository version (with the "Use Repository") button.  So, for a project file I use the repository one and note what has been added to the branch - so add new files to a project manually on the resulting working copy...

For code files it is best to leave the file conflicted and come back to edit it, so select "Resolve Later" for your code files.


Once all the files have been considered we get told that the merge attempted and that there is a conflict, each file listed in red therefore needs looking at manually...


This conflicted state introduces a new icon for us, the yellow exclamation, if we drill down into the folder we can find the file which is conflicted and several other files... The file itself has the differences within:



This view of the code is useful to keep open as this is the file you can commit back to the repository, and you can no only see the "working" copy code - i.e. branch 2... but also the trunk copy - i.e. the trunk with branch1 merged into it...

<<<<<<< .working
            Console.WriteLine("I am branch 2");
=======
            Console.WriteLine("I am branch 1");
>>>>>>> .merge-right.r21

So, from this we could simply edit the file to contain both the pieces of code in the correct order...

       static void Main(string[] args)
       {            
           Console.WriteLine("aaaa");            
           Console.WriteLine("bbbb");            
           Console.WriteLine("I am branch 1");
           Console.WriteLine("I am branch 2");
       }

Or you could change the code to be more efficient...

       static void Main(string[] args)
       {            
           Console.WriteLine("aaaa");            
           Console.WriteLine("bbbb");            
           Console.WriteLine("I am branch 1 and 2");            
       }

When you are performing this part of a conflict resolution you are really taking the role of the softwaer architect, this is your bread and butter.  To ensure the integrity of your code, of your whole project, to invoke the builds on it and ensure it all interoperates and passes all the tests properly.

This is also an excellent opportunity to apply and enforce your coding standards on the resulting code.


The other three files are actually the version in the working copy, the version in the left of the merge and the right... so these are the trunk and branch versions alone.  You can chose to right click on the file now and "Resolved" it, this resolved will allow you to pick the respository version, your working copy or the left hand side of the merge copy... Otherwise you need to open and edit the code yourself (like we have), and save the file..


Once you are happy with your solution, you need to right click on the conflicted file and resolve the conflict...


Once resolved the other files will disappear and you ar left with just one new file to commit... You need to check and recheck your project and code are complete, don't just assume that the "It compiled" attitude is enough, its not.

Now we have the trunk merged up into the working branch, we have both pieces of code... so commit our working copy....


Take note of where this file is being committed to...

Now what do we have?

Well, we have a trunk with branch1 merged into it (lets call that the hybrid trunk) and we have that hybrid trunk merged into branch2... What we need is to merge that branch2 with all the changes back into the trunk...

Very long winded I know... But easy now.... Remember the bullet points above not to do?... Do them...

  • Take a copy of the trunk.
  • Select to Reintegrate the Branch2 into that trunk.
  • Accept the Merge
  • Check your project is a match for all the hard work you just did on branch2 checking its all fine and working and correct
  • Commit
Note: Any optimizations may not merge cleanly and you may need to reintegrate optimizations AFTER reintegrating the branch...

However, this example has been based on one major problem, that of the programmers working on the same file... Indeed working on the same line of code!... You really need a project plan... and in that plan you need to keep people working in two separate files at the minimum, that way the ONLY things that will conflict are the different files and the only things you need worry about at merge time are the project and soltion file (where by as advised opt for the trunk and add to it the new files from the othe programmers).

The goldn rules for SVN however are, to never trust the merge software, three-way, or even two-way, merge algorithms are hidiously stupid and sometimes the output from them is rather more counter productive than you would have thought possible.  Always remember you, the programmer, a trained expert, is better at merging than the software here, use the subversion tools to get the bulk of the unchanges items together and then just check over those few conflicted items.

But above all this discussion, take sometime with this set up, you have your own little local respository, go try things out, try checking out, try merging, try all three kinds of merge, try using the repo browser to take a "copy to" of some folder.  Try to remove a folder or file from the latest revision and then check out, but check out the previous revision with the file still in it, and keep that file, so when you pull the head revision again you can add it back in, effectively restoring the once lost file from the version in the repository.  Use the internet browser to check over your code and do sit around with your team, with the code up in a browser and code review the contents, bring in new ideas make a list of things which could be improved.

Once you're using the tools you'll come to grips with them at your own ease, and put them to your own uses.

6 comments:

  1. You are Awesome. This helped me a lot. Thanks.

    ReplyDelete
  2. Thank you, it really help me.
    Regards.

    ReplyDelete
  3. Excuse me, thanks for your awesome tutorial in advance.
    However, I'd like to ask about accessing svn from windows. I still cannot access my svn by my ip. Is there still something to set ?

    ReplyDelete
    Replies
    1. Try to access it via 127.0.0.1... Other than that the tutorial shows accessing the repo via TortoiseSVN, which I would recommend as your client. But you can access it via your web browser too.

      Delete
  4. You're a beast, Xelous! This is simplified and eloquent as fcuk. Much appreciated.

    ReplyDelete