Development : The Art of Fungibility

Fungibility is about being able to substitute one thing for another seamlessly, let us say you have a paint sprayer which contains red paint today, but tomorrow you want to paint the wall blue, you simply swap the paint... The unit remains unchanged... The paint is fungiable.

When it comes to software, especially in the past, it was of paramount importance to write your code to conform to the system, you had to write code which was compiled for one processor only, you had to conform to the calling conventions of the underlying architecture and most importantly the OS... Software was generally not very portable.

High level languages, like C, were created to get over this problem and indeed the Unix system as created by Dennis Richie could be recompiled from source on various platforms, however, things were sill very specific and the "portability" was the case of recompiling and overcoming the localisation quirks of each underlying platform.

However, with the advent of portable interpreted languages like Java or Python, and manageable tool-chain assistants like Docker and Kivy you can write your code once and have it move between systems without effort.  The first of these which I encountered really was Java, I had this run on both a Sun Sparc station and a PC, and when I wrote my degree dissertation (Parallel Computing in an Open Environment) in order to open up that platform the interop module was all Java, running on a Windows PC or any other Java 1.1.4 (AWP) functional system.

Little did I understand, at undergraduate level, how important a trend this cross platform context for an application would become.  Smart Phones, Tablets, Consoles, various PC Hardware, Linux, Windows... At one point I was even writing work in Personal Pascal on the Atari ST, then porting it to Turbo Pascal on the PC for profit, when I didn't own a PC myself, so one had to be careful and thoughtful.  There were not so many avenues for a system to truly be the same on one platform or the next.

So, as I learned to program you had to sit down with a pencil & paper, with a process flow chart stencil and work out how you wanted to lay the system in order.  To order what was being written when.

This lead to bring frugal with your time, and ultimately lead me to see fungiability as a key component of my way of working.  Because it was such hard work to cover all the bases and to make your code truly portable across machines.

As great as Java was, and python is now, or technology like Docker & Kivy are though, I believe they are pushing fungiability into decline, sometimes with it not even being considered, take my post of a few weeks prior, replacing an older system with a new, I simply looked at the API being exposed, boiled it down to a few function calls then re-implemented them and the system came back up.  Replacing the code inside the functions was hard work, but the API swapped over for me.

I've since learned that other, younger, developers would simply have re-written all the code, they'd not have cut out what wasn't needed; because they have the tools, they have the machine power (ram and processor) to support continuing with the added load, they don't pair things down, they don't binary chop what's not required, they only create and ever expanding gaggle of code (If you are young and don't do this, please don't think I'm painting all with the same brush!)

So as you start to consider scale-ability, on demand computing and cloud computing, keep fungibility in mind, think and design like for like replacement of systems, can you swap the service back end from an instance of Ubuntu on AWS to running on your local XEN host?  Can your developers in Canada send their data to dedicated local machine today, but deploy to a virtual instance hosted by yourself in the UK tomorrow?

Consider your file system advanced RAID arrays offer a form of Fungibility, that you can swap bad disks out and replace them with new, ZFS does this in software in the same manner, so long as you include your disks into the pool by disk-id.

Fungibility, in code, in provision, in hardware... It's an art which is sadly not often mentioned, indeed it's not even in my spellchecker's dictionary.

RIP : Richard Hatch

Having just heard the sad news of the passing of Richard Hatch, I'm genuinely saddened... Perhaps best remembered for being in both the original Battlestar and the fantastic re-imagining... He brought a character of menace and quiet perseverance but recognisable motivation to the latter.

I heard the news about his passing only after seeing RedLetterMedia post a rather wonderful snippet of their talking to him about one of the many schlock Sci-Fi titles he was in, and his laugh, his humour tell you more about the man than perhaps we here in the UK had any real idea of.

But, hit that Turbo button and see the stars now... My Apollo.

People : Office Space

I do not buy into the open plan style... There, I said it.... I think that a few people in a team can work well in a room together, and I think that a communal space for meeting, chatting or relaxing is very important, but that space, that place for conversation & distraction should be somewhere other than around the teams desk.

I also think that if whatever development methodology you're running, the progress should not be held up in public, no-one wants to have their co-workers see they've been behind on some ticket or story, they just want to get their heads down and get on with the task at hand...

So, when your staff members start to build artificial walls for themselves, made of machines, plants, books or empty coke cans... Then maybe they're telling you something... Perhaps this open plan this has over stepped it's purpose.  Perhaps they feel they are being shown up in review that they're not performing.

And it can only be an instant face palm for the likes of the Scrum Master or Product Owner to shove this down a developers throat, to make them worry, and take them off of track, rather than leaving them to work.

That said, what would be my perfect working space?

Well, I like the idea of my own desk, not hot desking, I also like the idea of being able to carry my work away from my desk, to a whiteboard, or a soft chair with a laptop to think about a problem.  Giving a personal level of separation from the work and a space to spread the mind.  Here in the UK the weather and the space generally precludes walking a problem through your mind as you literally walk, but walking desks might be an option, certainly I think height adjustable desks are a huge bonus.

A communal area, like a large meeting room, with both a large central desk, white boards and smaller tables for conversation is a key asset, not having this lends itself to staff being almost confined to their desk, this doesn't mean the desk isn't a place for conversation, but it at least leaves the desk as a focus of engineering a solution, rather then the talk of it, the thought of it, I find you need to clearly define the place of action from the place of inaction.

This includes consideration of a place for luncheon or formal meetings, the former should not be the desk and the latter should not be that communal area I've already discussed.

Everything else in an office space has to follow on from this, and there's a lot to consider, where are the power and network outlets, where is the light going to come from, is it natural or artificial light, is there a view, are there any ambient noises such as traffic.  Everything has so be considered when positioning a creative team.

I believe taking this all into consideration before a project begins, before a team of an office layout is cast in stone is important, and proves itself vital as people feel able to work without being considered to be snubbing others, but then able to get away from the pressure of the engineering chair, to let off some team or co-create with a team member over a problem they have.

All the environment relies upon the right work ethic in the team members, however, when it's right, it is right.  You soon notice when things go wrong.

Developer : What helps you picture a system?

Systems are complex things, they contain many components, from the physical to the ethereal software running around inside them, you have many considerations, from load to speed and timing, to ease of access and security, so many skills and so much to consider...

So how do you go about creating such systems?

I've had a wealth of experience, in different industries, which arm me with some hard earned experience of how I can best filter through all the complexities before committing too far in the wrong direction, and even more experience of how to rescue a system already too far in the wrong direction!

But as a new developer, a new system guy, or just an experienced guy joining a new team, how do should one best go about approaching the task at hand?

Here are my top tips...

1. Always listen, take notes, record or otherwise pin down the specification.... Walk and talk to the people asking for the system, get to know their mind-space as best you can.

2. Think before you act, many experienced developers take time to think of a solution, they'll diagram, discuss, brain storm or just walk through a problem.  These mental acrobatics are sometimes key to a project getting off on the right footing.  Younger, eager, developers often over look having a period of quiet contemplation.  Don't mix this up with not implementing the basics, an XML loader or a generic piece of code can be written, but what those general tools are put to use doing, the actual architecture, that needs thinking about.

3. Discuss the system, from top to tail.  If you're handed a spec, dissect it, talk to the author, talk to the target customer, talk to the developers who are going to lash themselves to the mast and go along with this vessel, whether it were to sink or sail.  Discussion is not thinking, from the previous point, you need to think your own path through the problem, and remember your path maybe different to others, your solution maybe different to others, this next tip to discuss the problem lets you exchange with others and ensure you are both reading from the same story sheet.

4. Repeat.  Once you have started to work, record what you are doing, talk to others about what you want to do, discuss how things are going and feed all this back into a new round of work, the next sprint, the next deliverable...

These are my positive tips... Have I got negative tips?... No, I have experience, and sadly I can't pass that onto you directly.

Games : I don't get...

I don't Diablo, there I said it, I don't get it... I play D&D, I play Pathfinder, I played World of Warcraft, I played loads of other RPG stuff, but never Diablo... There... I'm out, I'm officially out...

To Paul and Max whom both told me I should play Diablo, and indeed I think Paul you lent me the game, I never played it... I don't get it.

Development : Phone Link

I wonder around all day with an Android phone in my pocket, however, I can't always answer it... "No Problem" I can hear you cry "leave it to go to answer phone".

However I purposefully have no answer phone, if you can't get a hold of me, you can't get a hold of me, but what I would like is a method of receiving a message which I can digest in my own time, not a voice mail, not a text message, I want to be able to have left my phone at home and still get the information from it that I've had a call....

I'm a developer... I can do this right?!!??!

Well, yes and no, first of all I started with a Java Application which would read the missed calls log and forward it to one of my servers, this worked, but didn't let the person at the other end know I had been made aware of the missed call, it also logged junk mails and just gave me the number.  If I didn't recognise the number I could be left lost.

So, I set about a but of middleware, I had the missed calls log forward to my server every few time it changed, the server then looked up each number on a white list and a black list.  If found on the white list is looked for them in contacts and sends them an automated message that I had received a missed call and to e-mail me...

Fine for contacts.

Last week however I went a step further, if a number is not black listed it looks them up as a contact then if not found it googles for them and looks scrapes the top ten results.  E-mailing the number and these search results to me.

I've today added a series of regular expressions to pattern match any number or name from the results, and if more then two match it flags it as a contact and googles for them, finding an e-mail address is the target... It will go to my various e-mail addresses and alias, looking for previous missives and will send them a mail that way.

The person has to have had a contact with me, this is not blindly spamming people, and I had had to use a POP3 connection to one of my own email providers as Virgin Media stop SendMail from just working... But, so far it's worked.

Today alone I've had nearly eighteen calls, and it's sent about five messages today, since last night (when I started it) it's send just over twenty in total....

I dub this "Phone Link" for Android, and I may very well push it out to the world if I can polish it up a little and not have it so closely bound to my various e-mail services.  What do you all think?

Development : Simple Versus Complex

I have been reading some of Alan Turings works, specifically some of his work for the NPL in the late 40's and early 50's, before he went to Manchester in frustration with the NPL's lack of progress with his ideas.

One of the ideas Turing was pressing at the time was to do as much as he could in software, don't jump to hardware solutions; which at the time fixed the functionality of the machine to a single purpose; instead he wanted a general computing device he could put to any uses he desired.

Today the average programmer, hobbyist or indeed employed software worker, such as myself, don't have the opportunity to define new hardware for a bespoke task, indeed the machines we are driving are far more powerful than Turing ever imagined and we can do as Turing suggested, we can write software to get around an issue.

You need encryption, why pay to plug in a dongle doing this, when the processor you have is equipped to do the task for you, indeed some processors even have encryption instructions and powerful parallelism built into assist in such work!

So today's lesson is, take on board Turing's ideas, do in software as much as you can, even if you want to later move to real hardware emulate your idea now.  Here's a very good example:

nVidia's emulation lab is an extreme example, however, it is indicative of the kind of problems we are meeting today as older engineers are coming away from dedicated hardware devices (because their machines were too slow in the 1990's) and we have today's extremely powerful machines right there at our finger tips.

Why does this exist?

Last week I set about looking at a quite old system (circa 2008 it was last attended to) it contains a message passing system, which allows passing of messages (events if you will) between producers in any process to consumers in any other process subscribed to the service.  It did this with a windows service driver, which called down into a USB dongle to queue the message.

The consumers then polled the USB device (round robin style) to determine if the next message was for that consumer... You might imagine this was extremely slow.

It was also very complex, the code within the USB dongles was not mutable; being set in silicon; and the consumers could end up locked out of the service queue by a message being present for a consumer which was not able to consume the message.

This clearly needed replacing.... So, I had to look at it, a single afternoon, it was just extremely complex inside, so one had to take a step back, don't look at the contents of the code, look at the API, the functions being used....

The producer code relies on just three functions and the consumer relied on a thread safe queue which was woken up from a spin wait to call the registered consuming function... Not rocket science.

I set about writing new code with a very simple UDP server and clients, taking the message in, converting it to XML and posting it to the loopback IP Address of the master UDP listening port which then sent it out to the listening consumers on their loopback IP and port number.

Voila, in an afternoon, I had replaced five years (ish) of complex, hard to maintain code, with a working prototype of a solution.  And it dropped straight into place.  The dedicated, slow, old, hardware could be unplugged and the system just carried on as it always had.

The customer for this product is delighted to see it running faster, and the overhead of the hardware costs being removed.  All by simply doing in software something which was being offloaded to hardware in yester-year.