Friday, 26 November 2021

Naming a new Storage Server?

 Server names?  In practice I've always been in places or projects where folks are bad at naming things and I am as guilty as the next person in that realm.  I've been on projects naming releases for obscure lord of the rings characters, for characters in a random fantasy story the lead developer had written, for birds, biomes, planets and stars.... Generally these are very niche, if you know the reference you're fine, and it brings a little character to your server.

The trouble arises when a name becomes attached to the history of that thing, the worst I never had was a project with a release named Elrond, where the release itself was re-released, so we ended up with Elrond-2, 3 and 4, then Elrond4RC1 for an update to that successful update, it was just a minefield.

In servers, the worst I ever managed was a box called "Diomede", which would often fail because of bad (read that as cheap) hardware, so the name became inextricably linked to bad performance and issues, despite being a really nice Tomcat & IBM Websphere host with a near 99% up-rate, but that 1% let it down because EVERYONE worked on that one box.

At home at the moment I have:

* Wellington

* Napoleon

* Elba

* Ney

* Waterloo

I'm sure you can tell the reference, but I've had about three versions of "Waterloo", I've also had two versions of "Ney" and at least five "Napoleons".  The names are getting mixed up, as I leave a home project and come back I'll go look up my notes, pull up that machine storage only to realise "oh this was on the OLD version of that name" which was long whisked away into the ethereal world of my replication server or just lost to time.

As such, I want to have ONE big disk server, and I really want to make it nice and neat.   I've already retired all my older machines, like this one:

But I do now have TWO cases of this specification, with the many internal drive bays:

In one of these I have a low-power Core i3 T class CPU, I forget the exact spec, but it's a low-power chip (35W) just enough power to run storage backups IMO, so this is where I'm going to start my quest.

Spinning rust drives are the challenge, I have four 4TB drives just sat idle, plus a host of smaller 1TB drives lying around, these need sorting out, mounting, checking and I then need a strategy.

I'm very likely to go with ZFS creating a mirror across three 1TB for critical storage and then some striped storage across the 4TB drives.  I then want this to make files available across the network, both to my 10GBit and my 1Gbit LAN's.... That's going to require a bit of work to install the former, but the latter should be available out of the box.

This starts to completely retire my 4Gbit fiber channel interconnects too.

So what to name this first stepping stone?.... Comments below:


Since it's about storage, I think my theme of names points to one of the islands which hosted Napoleon in exile... but lets see what you guys come up with.

Wednesday, 24 November 2021

What's Going On?

Well, we're moving house again!.... Yes, this has been a bit mad, there's a lot going on and logistics are in place, but I'm going to be getting my own little office room back again rather than having to split creative thinking with a bedroom/high traffic area.

There's no real reason for this beyond this house didn't suit us for our living needs of space and specification, sure it's very close to our horses but it's not the house for us.

We've made a modest profit on what we've paid for it, with the rise in the market happening all around, but that's been offset by our paying more for a target property than it is really worth, and it itself being inflated in price by the market.

But whatever, a move... so that's where I've been, as well as being very very busy with work.

I am also working 100% at home now, so I've found that break between wanting to program in my own time somewhat removed by being at home all day working at programming, and of course I can't talk at all about my work on these pages; one day maybe.

Today though, a head of the move, I've been thinking about my network set up, at the moment my network is rather broken up, some of my machines are already at the new house, some are in storage with the in-laws and others are here sat with me, but without power.  I have though received some nice kit from my employer to work from home... New screens and a lovely spec PC are the big ticket items, but generally I'm just in a holding pattern until we move.

Here however is my planned network...

We will go over this in a follow up post towards the weekend.  And I'm going to be changing things up a bit, I want to support PXE booting several things, replacing some of my servers/services with VM's on the lower-power machines I have.  And then only waking up the high power machines when serious work is required, and all that waking and sleeping going on automatically through scripts & network magic packets.... Lets see where this evolution goes.

Wednesday, 6 October 2021

Home Engine - Loading PNG Textures into DirectX11 with SDL2

ANYONE Interested in my sharing the code to do this?  Leave a comment, 10 comments and I'll blog it.

I'm using SDL2 as the background framework for providing cross platform input and window handling, but I'm writing my own renderers for various platforms (DirectX11, 12, OpenGL and Vulkan).

All of them support coloured shape and line renderering to one extent or another, so I decided to take the leap into texture mapping.

First of all I put in all the framework to represent textures in my engine in a cross platform manner, and for ease I decided to use SDL2_Image to load my pixel data.  And I started with a simple PNG file I created in MSPaint....

Immediately I ran into issues, PNG is RGBA, the png I created had a transparent background as far as I knew it should all work.  I swapped to RAW pixels and I was getting 4 bytes per pixel, it was only after a bunch of reading and then stepping into the SDL_Surface format did I realise the data might be whatever format but the surface was clearly 3 bytes per pixel.  And funky things were happening.

My assumption that MSPaint was outputting 4 bytes per pixel was just wrong, instead it's 3 bytes, RGB.  And there's no matching DXGI_FORMAT for just RGB - least none I could spot.

A quick reexport in GIMP setting the explicit pixel format to 8bpc RGBA, which then of course just worked....

Therefore, it's official, using an SDL2_Image load of a PNG as your DirectX11 texture is perfectly possible.

And for me, this reduces the need to include yet another library (other than SDL) to handle image loading.


Not even sure how, but this older video got attached to this post, which is about "SuperSampling".... but really just changing the renderer to window resolutions...

Wednesday, 22 September 2021

House Buying - Sealed Bids

Until yesterday I'd never heard of sealed bids, they are apparently common in Scotland, but in the five house purchases I've done over the years I'd never come across them.

Just parse that again, FIVE house purchases, that's above the national average, I've bought and lived in five different houses; I'm not talking about buy to let, I'm talking about my main house.

And in all those transactions the process was see the house, offer accordingly, offer rejected or accepted etc, so you knew your budget, you chose the value and off you go.

Now, as a vendor, this worked but you were never sure what other offers you might get, you certainly never found out whether the buyer was able to pay what they're offering; and recently that has been an issue for us, we rang up about £400 in solicitors fees only to find the buyer hadn't gotten enough from their property sale and they pulled out, so our chain collapsed.

Knowing a head of time if that buyer was going to be good for the offer they'd made would have been nice.

Indeed, I've only myself been able to offer once I was sold (subject to contract) and had a mortgage agreement in principle to offer against, this is what the vending estate agent is primarily paid to do, ensure the chain can be set up.

So 36 hours ago we offered on a house, lets say we offered £250,000.  This was summarily dismissed, fine instant feedback, I can live with that.

We therefore upped our offer, £252,000... a 2K sweetener, literally.   And we expected, just as the first offer, this to be accepted or rejected.

The vending estate agent they got our hopes for a simple transaction up by asking for both our agreement in principle and status of our sale, the proof needed for the vendor to know whether our offer was firm and real or not.

I duly handed these over, still expecting the call back that day saying "Great, accepted" or "No, rejected".... Now, I've shown my hand here, my agreement in principle was much higher than the price offered, I could offer more; much more in fact, and they now know this.

I don't know if there are any other offers, they've not just rejected my offer as too low, so I'm now flying blind.  I can spent more, but I'm not willing to unless I have to.

Only now do "Sealed Bids" appear on the horizon, I receive out of left field this request for "best and final offer by 2pm".

I don't even know what this means, and I ask, but get literally ran around until not only am I annoyed with the agent, but I'm annoyed with the vendors, they just rejected an offer, clean, simple and normal the day before, today there's some shenanigans going on?

They've decided, or the agent has decided, AFTER seeing we can potentially offer more, to go for sealed bids.  I literally say to her "I could offer more, but I'm not throwing money into the wind, either there are other offers and ours is now too low, or there's not and our offer can be accepted".

But she persists in this sealed bid, I don't change my bid.

And this annoys me, I'm thinking:  "Are they just pushing up the price by scaremongering?"  They've seen I can offer more, are there truly more offers in?  If there are more offers in, are they now getting pushed around the same as me?  or not..... Because the experience, the strategy they have here has just changed.

All in all I think it does the vendor down, this method is very off putting, they could get lots of low-ball offers, which any number of the potential buyers might be able and willing to offer more than, and ultimately it makes the buyer quite stressed.

Buying a house and moving is one of the most stressful things already.

And this model isn't even like an auction, lets take eBay, if I see an item bidding at £1, and I put a max bid of £20 on it, it doesn't just put £20 on the item, it puts £2.... then whatever up to the ending price.

That's not how this whole sealed bidding is presented, this estate agent wanted out best and final offer... it could have been £275,000.... and we'd be the best offer... except the next best offer might have only been £250,500... and we've just MASSIVELY over paid.

It's already stressful, the housing market is already crazy, if you run into sealed bids folks just don't get involved.

Monday, 20 September 2021

Home Engine - Lines and Camera Interaction

 Yes, yes, it has been a while; both here and on the YouTube channel, I've just been very busy okay?... Okay.  Friends?... Friends.

So what have I been up to in code?... Well today I added line rendering to my home engine, this is still using the underlying "Triangle" raw data, so I have to always have a cleanly divisible by 3 (for the sides of a triangle) number of indices in the data, but otherwise it just seamlessly integrated.

Its using "LineList" as the underlying primitive type, which maps to the D3D11 type under.

I also noticed that these lines from 0, 0, 0 to 1, 0, 0 and 0, 1, 0 and 0, 0, 1 respectively are "backwards", that is my unbeknownst to me at some point my camera has started to spawn around in front of there my scene objects are, so I'm looking back along the Z axis not "up" along it... Am I making sense?

I also improved the camera some by linking the rotation of the camera to the mouse, which makes it super intuitive.

The failed part of the work this last month was an effort to update both the Vulkan and the OpenGL renderer implementation.  This was an utter failure, the simple reason being that I've gone too long with my limited knowledge with DirectX.  I'm sure the "port" to DirectX12 will be easier, but yeah, I've already gotten the other API's so far from my implementation (abstract base classes) that it's quite alien to return to them.

I'll circle back on that, maybe break down the API's with failed/broken builds until I can slowly etch away at the challenge.

Wednesday, 25 August 2021

Hex Messages when Reverse Engineering

One of the things I did in 2018 was a bit of reverse engineering, the company I worked for had bought a bunch of older hardware which had to be reused, the trouble being?  They had bought the hardware NOT the software.

It look a lot of USB sniffing and signal trapping but eventually I had the device handshake and could see it spew me messages, from these I saw the header format was pretty simple just a pair of bytes and then a length byte followed by whatever the message was.

Interacting with the device stimulated it into sending more messages, so we could start to map physical actions to these USB packet ghosts, eventually we had all the ones we could just prod out of the unit, so it was time to work on more advanced interactions and actually try to tell the device to carry out actions for us.

This immediately met with problems, you see the messages coming out had a counter in the footer, it turned out that if you sent a message you had to use the next number... so receiving messages 1, 2, 3 and then you send 4 and receive 5.  Keeping this in sync was a little bit of black magic, but it soon made sense the unit was serial in operation, you could only talk when it was listening, and it only listened after certain choice messages it sent to you.

This all made sense.... What didn't make sense was a four byte message I saw every few minutes, this frustrated me no end....


I converted these to two into an integer... 3203391149... It didn't ring any bells, but I persisted and was rewarded when I switched it into hex...


This is a common message left in hex in code, as a joke, but it was also apt, the board was not going to talk until a full handshake was given again and the packet series stream started over to order things.  Cute.

I've found other hex strings, but this one was the first on this particular board

Saturday, 14 August 2021

That Time I Found a Pub

I don't mean just I was out in the country having a stroll and happened upon watering hole.

No, I was in my garden, and I found a pub.  Specifically the pub's kitchen floor.  The pub in question was the Miners Arms at Selston, obviously long since demolished, but they'd not broken the ground concrete up, just dug the nice neighbour diagonal's garage hole into it and so about seven inches down under my lawn was the white tile floor.

This made my lawn growing efforts a bit of a pest, they kept drying out.

Tree's also rooted along the tile level and it was impossible to landscape effectively.

The ground itself, the natural ground was very clay riddled, clay deposited during the last ice age as part of Alma Hill.  But still a challenge to dig out.

Not as challenging as the neighbours, but then, I've mentioned their mania before.