The Worst (or maybe second worst) Source Control Ever

I've not made a bit deal out of this, but I like source control systems, I like git, I even like subversion, I like when folks even take the time to zip up the whole project tree and stick it on a whole other machine.  All are different source control strategies, each have their place.

What doesn't have a place in my opinion is poor implementations of this... such as... When there is no source control, that folks "remember" how the code was and can work back to where it was when things go wry.

Because inevitably they can't remember.  Solutions, subtly and satisfaction with what was present can easily be lost.  And in such situations you can only blame yourself for not having a back up of the original or yesterdays or whatever.

So it was as part of the tyrannical "lead" of one developer I came to meet the worst source control system ever, it was based on floppy disks, had no back up and was a manual process.

And it starts with an initial checkout, to a CD-R.

The CD-R was known as the "master" disc, and it contained a set of the SQL scripts to build an instance of the database, a series of sequential updates which each quarter were compacted down and made into one new "CREATE DATABASE/TABLE" thing.

Then there was an auxiliary folder with a bunch of tools which you copied to your hard drive and used.

And finally there was the source tree, you installed Microsoft Visual C++ 6 and opened the project and built the code.

You then ran a tool and put a floppy disk in, this disk was used as a swap between you and the lead developer at the end of the day, so a nightly build could be carried out (or a morning build on a Monday after a weekend) on a slightly more powerful machine.

As a developer then you had you IDE, this background tool running and you worked as you normally would.

Every time you would save in the IDE the tool saw the change in date of the source code file blatting that file onto the floppy.  The whole file, overwriting the old, not a delta, not a diffable solution, it just progressed forwards only.

You could not revert, not compare, not really do anything we take for granted in modern development.

This was a bad system.

The bad became worse when there was a delay between the save & the tool picking up, and then a floppy disk took time to write ( a lot of time ) and it was soon realized whilst it was writing the file from the HDD to the floppy it locked the file, so you couldn't write another copy of the file whilst this slow write was in progress.

As such lots of developers didn't run the tool, they would run it once at the end of the day, or just copy all the files listed as date/time changed to the floppy manually.

This floppy was then the feedback of all changes to the lead, in charge of merging the changes... Well I say merge, what happened was the mainline version was simply over written with whatever the developer handed over.  They were meant to qualify their code was coo, they were meant to run the tool which performed the copy.

So all the pieces were on the board and check-mate was one move away.

For the worst moment came on one particular Friday, a developer was toiling on tidy up duties, those little tasks you put off until the end of a day to fill time until you can go home, his task this day at 5:03pm was to correct two typo's in a file, one higher up and one lower down.

Just spelling mistakes, no need to recompile that, taking an hour, on a Friday with the pub and a takeaway calling.

So the chap performed his save, saw the light come on the disk and then did the second correction, hit save, the floppy light went off... and he turned his machine off... just dead off... pulled the floppy and dropping it into the "merge tray" on the leads desk started his weekend.

I walked into this on the Monday morning at 8am, as a very junior developer, the managers were all present, the lead was present, two senior devs were pawing over floppy disks in no discernible order.

For it turned out that the tool had started to overwrite the second save of the file, it had gotten so far through the write when the power had been pulled.  The original was lost on the developers machine for he had pulled the power... rather than shutting down.

And this source control system only went forwards, you could never ever regress.

The seniors were trying to build a delta list of each change for the last month, back to the start of the last CD-R... From floppies which had been used and reused, they had to manually inspect the changes between the file in question in the floppies.

If the copy they had was deemed to be older the disk was put aside.

If younger the file was copied into a directory to slowly try and catch the code up.

Around 10am the developer responsible wondered in, listened to the complaints, listened to everyone being exacerbated and then without a blink of irony said "Told you this wasn't source control".

Fabulous...

The worse Source Control ever?... Perforce... Hands down, Perforce.

Software Engineering : My History with Revision Control (Issues with Git)

I'm sure most of you can tell I'm one of those developers whom has been using Revision control systems for a long time... So long in fact, that I wrote a program for my Atari ST; whilst at college; which would span files across multiple floppy disks and use a basic form of LZH to compress them.

Later, when I graduated I worked for a company using a home-brew revision control system, imaginatively called "RCS".  Which basically zipped the whole folder up and posted it to their server, or unzipped it and passed it back to you, there was no way to merge changes between developers, it was a one task, one worker, one at a time system; almost as lacking in use as my floppy based solution from six years prior.

During my years at university, revision control was not a huge issue, it was NEVER mentioned, never even thought about.  Yet today we, sometimes happily, live in a world where software engineers need to use revision control.  Not only to ensure we keep our code safe, but to facilitate collaborative working, to control the ever growing spread of files and the expanding scope of most all projects beyond the control of a single person.

Now, I came to using professional grade revision control with subversion, in early 2004.  I think we were a very early adopter of Subversion in fact, and we spent a lot of time working with it.

If you've ever taken a look around my blog posts you will see Subversion is mentioned and tutorials exist for it all befitting for nearly twelve years working with it.  And unlike the comments made by Linus Torvalds I totally believe Subversion works, and works well.  It is not perfect, but I find it fits my ways of working pretty well.

Perhaps after twelve years my ways of working have evolved to adopt subversion and vice versa, but whatever the situation, I'm currently being forced  down the route of using git a lot more.

Now, I have no issues with git when working with them locally, ALL my issues are with using git remotely, firstly the person whom (in the office) elected to use git, started off working alone, he was creating a compiler project with C# and so he just had it all locally and used Visual Studio plug-ins to push to a local repo, all was fine.

I've used git with local repos without problem.

All the problems come with pulling and pushing, with remote, and controlling that access.  Git intrinsically fails to protect the access to the repo easily, relying instead on the underlying operating system.  Which is fine, when you have a controlled, and easy to manage user base as with a Linux server, however, with the minefield of integrating with Active Directories, domains and whatever on windows based infrastructure nothing but problem comes up.

The next problem I've had with Git has been the handling of non-mergable files.  We have lots of digital files, movies, sounds and plenty of graphics.  As such we've had to work around git, by having people work on files one at a time, and to cross reference which files they are responsible for.  With an art crew of five people, this means a flip chart or white board is constantly listed with the media files, and someones initials are next to it, just to help control access.

"Surely git should be able to lock these files", they constantly cry.  No, how can it, how can a distributed control system manage locks across five or more repos's which are not talking to one another, and if you did elect one to be considered the master, how do you then transmit out to the passive clients every time you lock or release a file?  You can't, the artists would each have to remember to pull, or shout to each other to pull now!  It simply doesn't work.

And as a way of working the white board is pretty poor, but it's all we have right now.

The next problem we had was the massive amount of disk space being used by the repos.  We boot our machines off of very small (128GB) drives, then use either NAS or SAN for our main storage.  This was fine, and efficient, and critically it was all well backed up on the infrastructure we use, and it worked for twelve years with subversion.  However, with Git our huge files are constantly being snapshotted, this growth in the size of the overall repo is replicating files over and over and over.

In short, despite someone else, and the world at large turning its back on Subversion, we here in my area are strongly drifting back to Subversion.

Trouble is, it feels as though we're swimming against the tide, despite all these slight deficiencies in Git, the over all organisation; and even external projects I'm working on; are pushing Git.  Torvalds himself calls people still working on Subversion "brain dead".  But has he thought about the short comings?  Or these case-studies we can give where subversion is a better fit for our working style?

Above all this wrangling internally has been my problem expressing our situation with Git to both the initiated and uninitiated.  When talking to those advocates of Git, all sorts of acronyms, actions and comments are made "use git this", "use git that".  The problem being, there are something like about 130+ commands in Git, that's a huge amount of things you can work with.  But, we can break down what we've done as "git init", "git checkout", "git add", "git commit", "git push", "git pull" and "git status" (as I've said merging utterly failed, so I'll gloss over that right now).

Given this huge scope of possible usage, and such a small exposure experience it's hard to put words against why things were not a good fit with Git, the initiated always seem to argue "you didn't give it a good crack of the whip".  But we don't work in an environment where we can try one thing and then another, it's an old working structure, which has evolved over time, people are used to it; and I'm nearly 40 yet I'm the youngest guy here!  Training those around me in new ways of working is very much an uphill struggle.  So, when introducing something as alien to their mindset as Git, it was always a loosing battle.

To express this to the uninitiated is even harder, they don't know what an RCS does, nor what we mean by centralised or distributed control, they just want to see our work kept safe, and our work to be released to the customer.  Gripes about Git and Subversion make no in roads with them, they're just unimpressed when you explain that these solutions are both open source and have no support.  The fact that they're free has been wildly ignored, yet I could - for the price of the support contract of another system here - easily buy and operate a whole new SAN just for our needs!

Lucky for me though, after struggling with this issue, I ran across Peter Lundgren's post on the same topic, of expressing what's wrong with Git.  He doesn't advocate Subversion, or anything, over Git, he just lists the problems he had with Git, and he crosses much of the same ground I have had to.

Check out his post here:  http://www.peterlundgren.com/blog/on-gits-shortcomings/