Monday 29 August 2022

Home Engine - Bank Holiday Camera Controls & More Models

After the silence earlier in the year I'm so happy to be blogging more often, and I hope you are all enjoying it too; I can see some regulars popping by and a few new faces in the stats list; but still far too many of you are totally anonymous, drop me a comment!

Anyway, what I have I been up to?

Well, under the hood I've totally revamped how the uniform buffer objects are passed to the rendering operation in the vertex shader.  I've also moved the view and projection matrix calculations into a nice camera system class, so they're contained and sensible to use.

As a result I've then added some key bindings to move forward, strafe and rise/drop the camera on the plane of x-z.  Likewise I've added a free mouse mode to let me move the mouse and fly over the terrain.

Speaking of terrain I've added a checker board to show me the X-Z plane at about -0.5 Y.


I also noticed a winding order in my Blender exporter tool, I've not fixed that in the python, just patched it in my loader and created a few more models to start building out a play space.

Technology wise I need to make the camera more usable - a full third person camera basically.  Then I would like an orbit cam, which moving in the strafe would move it around the locked object.

Then I'm going to think about texturing and lighting.

The solid coloured aesthetic is intentional though, so anything more fancy is a low priority for me.

I am feeling much more comfortable in Vulkan though; and of course enjoyed moving all the glm implementation from the first (and second passes) over this project.

That's perhaps something I've not mentioned to you before, this project as a home engine... Well I've started and restarted it a few times.  The original was in DirectX9 way back; I wrote a page about it in 2004.

Then I re-wrote that in fixed function pipeline OpenGL; that didn't go very far, indeed I received a letter to cease and desist from an IP owner for that particular project's name was known in the wider world.  That IP has moved onto yet another owner now, so best of luck to them.

I however was knocked for six and it wasn't until 2018 that I started to look at a game engine at home again, spurred on by my change of job; and I really enjoyed that first pass, but it didn't focus on the graphics side of things, it was far more about the technology such as threading, memory management, and something you may have heard of, an Entity Component System.

All those pieces remain.

But this new project is very much just about my putting features together into a renderer, indeed the application level item is called "renderDemoApplication" and that is all this is.

What might the game be?  I'm still thinking about that, but I have three strong contenders and when I've picked one, you folks will be the first to hear about it.

Thursday 25 August 2022

Home Engine - Blender Export & Model Loading

Some progress in my home engine has been had, first of all multiple geometry rendering through my fixing up and better understanding both render passes and descriptor sets.

All of this is nuts and bolts, what I want to get to; just like the DirectX 11 rendered version; is to play with 3D geometry and placing objects into a scene.

To this end my hand written geometry is no longer hacking it, and you will have seen from my raw game engine posts I did use Milkshape 3D.

Milkshape, lovely as it is to my eye, is not a modern well featured 3D suite and sadly does not let me easily plug into the geometry live within it (one can only export and perform post-processing).

To that end I delved into how one might write a Blender add-on to export your own format...

Long Story short, here's my text intermediate output from just such a plug-in written in python.

The plug-in is far from robust, I followed the lines of another open source plug in and applied my own, copying it into blender with a batch file and clearing the pycache each time I perform an edit.

But it is doing the job I initially want, and I get my own format of file output.

And indeed I can load these into my engine...



I can work on the axis for the data output and start to ensure my camera system from the main client works as expected in vulkan; since the maths here is exactly the same code I don't anticipate any issues.

This should free my mind up to think about a lighting system and to finally get back to where I was with the coloured cubes in the DirectX 11 version and ultimately onto a full scene.

I have four constant things in the world, a ground plane, a skybox and a center marker flag (showing X, Y and Z axis respectively) with a quartet of boxes in the center of the scene.  This is the test scene for the "world" loaded into the scene graph.  And it's where I want this to be this time next week perhaps.

Since progress has been much quicker now I'm past the deep dive into learning the initial set up of vulkan.

I can also appreciate much more why folks just pick up an off the shelf engine, I myself make little prototypes in Unreal Engine 4.  But this kind of denudes me of the walk through these technologies.

For the most part creating an engine like mine, which is planned to have solid shaded low poly vehicular combat, well it's a solved problem, so I'm enjoying following in a lot of well trod foot steps to better my understanding of the processing, memory, mathematics and challenges in writing a game engine.

Friday 19 August 2022

Home Engine - Set Course for Vulkan

Over the last year I've been experimenting with writing my own home game engine, initially I spent time writing my own line rendering and then a wrapper with SDL2 all using my own mathematics library as a refresher.

Then I set up an engine framework, switched my own none-peer reviewed and frankly shaky maths for glm and started to use DirectX11. You can see a clip in my earlier blog post here.

I dabbled with other graphics interfaces, but ultimately I wanted to land on one of the two big modern interfaces, either DirectX12 or Vulkan.... And since I'm a linux guy at heart Vulkan won out. The difficulty? Well, setting up a Vulkan renderer just to render the "Hello World" triangle is over a 1000 lines of code.... And well, I wanted to write a framework as I went so I could just port this Vulkan back-end work into my abstract renderer API for my existing DirectX11 driven engine. That is my next step, but tonight, after a few hours each night this week I've finally gotten to the 3D step and I'm so excited I wanted to share with you.
So here we see I have multiple pipelines & render passes. It's creating and submitting multiple command buffers, I have integrated ImGui very roughly too.

But this is the first time I've used the uniform buffers, descriptor sets and descriptor layouts to send a uniform buffer (MVP) to the shader and get actual 3D.
I'm strongly following this online tutorial.

Though I also have the Vulkan Programming Guide official book and I'm writing my own functions as I go so the porting to the existing Home Engine API is minimal (or rather as minimal as I can get it).


One of the key pieces of tech in the base engine (driven happily by DirectX 11 when I switch to it) is a full camera fly mode system and a scenegraph rendering models.  I've also got a python driven exporter written for my own model format from Blender (and desperately need to get better with using Blender).  And I have a Python driven project generator (which calls down into CMake quite neatly).

The engine is structured into six major modules:

Base - which contains a host of utility functions, data types and tooling assistance.

Ecs - yes, I'm writing my own Entity Component System.

Renderer - The abstract renderer API which the application level creates and uses in the main game loop.

Application - Main app & game loop, I'm using SDL2 as the main interface here for input & window management in a cross platform manner too.

Physics - A basic hand written rigid body physics implementation, at the moment it really only supports rays, cubes and sphere intersection tests and I'm thinking of shopping around for an off the shelf something.

Game - Contains specific game logic, like spawning, camera controls, input mapping and uses base to provide timing.

This is all very rough and ready, it's a project to keep me interested in my craft really.

And putting Vulkan into the mix has been a real nice way to resurrect where things where, as you may gather I had the DirectX 11 and basic camera system all working in October 2021; but I've been a busy boy since then, both with work, life and moving house....

Now the new office is done and I'm building models in real life, I figured it'd be a nice time to build them in virtual spaces too.


P.S. I have no idea why some blocks are in white backgrounds, very annoying of Blogger to do that?

Friday 5 August 2022

I hate FizzBuzz and Will Analyse anyone who uses it

One FizzBuzz Interview so far.... News flash... The moment I am asked to write FizzBuzz during a technical test I intentionally stop and ask the questions from there on out.  I've done this a couple of times.  And you know what I learned... I learned a whole lot more about the interviewer and their processed than they found out about me.

Let me take you back to a technical interview I had about five years ago.

I drive up to the building, it is way out in the sticks; just about still in my home county, but so obscure it's in a part I have never ever been in.

I pull up and the building is like a box of black glass, no character, no soul, just a box.  The non-native tree species planted around it to discourage birds actually using the tree's and the austere grass puts me off.

Things don't get any better, for I walk through the door and there's no secretary, no desk, nothing... The lights aren't even on; this doesn't bode well.  I look around for a buzzer, a phone even a limp notice of what to do.  When a door opens and a man approaches offering me a hand to shake.  He leads me to a similarly deserted corridor with a chair in it and I'm left to my devices once again.

I read some inane notice about some award some member of the staff had received (this is important later in the interview) and I read the fire evacuation notice.  It's quiet, quiet and silent.  There's one door nearby with a few people within and another which is a closed office piled with stacks of desks..... Empty corridors - which would be dev spaces, massive rooms with a few people in them, no secretary and piles of desks?  What have I walked into?

About ten minutes later I'm ushered into this room, I've brought my CV a few piece of detail about me and some references to my technical projects at home; as well as references to some of the discussions I have on these very pages and my github.  You know actual technical things, which you can see me achieving.

I can't talk about my current work with them... Heck I can't talk to you guys about my work now.  But I do it, and I pay a mortgage; that's all the proof I can proffer.

Anyway, I seat myself and a lady explains their technical lead will come and perform the technical test, he enters a moment later with a massively overpowered gaming laptop.  I hate gaming laptops.

I much prefer a simple sleak laptop, a decent screen, nice keyboard and plenty of battery life; and I do my work on a remote machine via SSH or whatever machinations you wish.  I do not generally work on my laptop, it's a portal onto other more powerful machines which are tethered, and this way I can go take my laptop with me anywhere and get amazing performance from it, as I'm only interested in a bit of internet, RAM and then remoting into other services.

Anyway, he plonks this behemoth down and proceeds to tell me about test driven development... TDD.

I HATE TDD.

Now, I know test have a place, heck I really like doctest, including tests into code you've written, to ensure it is within specification is perfect, and validates your progress.

My hatred comes from writing tests up front, I find this backwards logic, and I don't think you could ever persuade me otherwise.  Define the scope of your input, validate it in your code (I often assert up front when data is unexpected or malformed, or at the very least bail early) and then you run your code, and then TEST the result.  I do not believe you should write a test and then further write the code to fulfill that test.

You will miss something, you will be thinking about the test as the solution, rather than the solution itself.

If anything writing the test and then producing the code to meet that test may very well be called TDD today, but when I was a lad it was called Prototyping.  And though I'm sure we can all agree prototyping is best done in a language different to your final product, it is not really prototyping when you are not producing the product, you're producing a test.

I find it wasteful of time, and often ignores enough scope, for instance I've seen a fair few hundred thousand up front tests written and then the code to meet them in my time, never have I seen a validation of the input data, the input has generally been hard set into the test.  That's not as wise a test-scope of ensuring the result after the fact.

(This does not mean test suits after the fact are the best, I just think they're the best for my style; not least as they are opt-in, for those times you are in a swing, and easy to add later).

So, TDD... Why is he telling me about TDD?

He does, he explains they have a method and that method is structured, he does not offer me any test framework specifically, instead he opens notepad and asked me to write FizzBuzz.

And the interview stops there; I don't think the guy realised it, but I stopped being interviewed at that point, I was not going to accept their job even if offered, but I wanted to see where this lad (and he was young) was going with his thinking.

This half empty hulking building and they're trying to hire engineers, and he is going on about TDD but FizzBuzz?

So I play total dumb, I simply start to write "int main"... And he surprises me.

"No No, start with the test".

Ohkakkkakay....

I have NO idea what test suites he has installed, I don't use G-Test or Boost-Test, I use doctest.  So I write a SCENARIO; he blinks and just takes the laptop off of me and starts to open some other window... I didn't know he would allow me to open some other window, this isn't my machine, I don't know what he has installed.

This sort of sums up TDD to me, the tools aren't easily defined.

Anyway, twenty minutes later and I just about have it Buzzing on 5's and I'm not very interested, I've come to a conclusion.

He isn't impressed; rightly so, but he has no idea he's just been investigated by an older and more cynical mind than his.

He leaves and no less than three people come into continue the conversation, and boy do they, they talk to me for a solid 120 minutes; I timed it... I was so bored.  But I engaged a little when they talked about their award winning technology.

I recall the poster for the lady who had won this award and mention her, by name.

"What, how do you, who has been talking to you?"

This is very strange, "no" I assure them "no-one, I saw the poster on the notice board".  One of the two girls flanking the chap in the middle here makes a note.

They talk to me about the company a bit more, they talk about their re-shuffling engineering and wanting to press their position forward to a virtual machine the customer hosts on hardware; previously they had been providing a dedicated box.

I'm terminally not interested in this boring black blocky building which now perfectly reflects the boring blocky people inside.

And I give up, I lean forward, elbows on the table, and I gather the copies of my CV from the hands of the three people and I say:

"Look, I've spoken with your technical lead, and I'm scared to my core of your processes, I don't like what I've heard, it seems you're struggling to recruit and there are reasons for this you will have to identify yourselves; I don't think I can help you"

They are scowling a little here, I'm sure they're far more used to candidates who want the job.

"Indeed, I think somewhere you have a beige room with ranks of screens and engineers in stuff suits not talking with a great board or wiki with all the failing builds traffic lit from your little test driven development board...."

At this point the lady on the left looks at the guy in the middle, who has a little switch to his eye.

"And such a stifling, control freak, desperate development environment is not one I wish to be part of"

They thank me, and I thank them sincerely.  And I leave.

As I come out of the building the sun has come around, the black glass face of the building is now not so black, instead I can see inside, there are three long benches of PC's all facing a pair of chaps at the front sporting bluetooth ear phones in one ear.  Above their heads is a large TV screen with a list of running continuous integration tasks and each has a traffic light.  As I watch one goes red, one of the two larger men touches the headset at his ear and an engineer on one of the benches glances up and starts to scrabble about on their machine.

It is a sweat shop.  I was not only right, but so happy I'm driving off.

I felt like going and knocking on the glass and cooing at them "Come, follow me, be free, be free".

But instead I just drove home.




Prior me: Just coming into edit this before it goes live on Friday, guess what I'm watching... 


And I disagree with the approach of analysing FizzBuzz; analyse the interviewer.