Sunday, 20 January 2019

C++: Undefined behaviour from realloc and the Clang Optimizer

I was wondering around in some system code and found a strange behaviour between two modes in a piece of code, where the code switches from double to triple buffer mode there's a problem, we've already got two buffers of everything we just want to allocate another but the underlying structure for sets of buffers wants to own them all... So the code went from:

SetOfBuffers
{
Buffer* one;
Buffer* two;
}

To:

SetOfBuffers
{
Buffer* one;
Buffer* two;
Buffer* three;
}

Creating the third buffer is fine:

SetOfBuffers::three = malloc(X);

But, the first two is a re-alloc, to reuse the existing buffer:

SetOfBuffers::One = realloc(OldSet::one, X);
SetOfBuffers::Two = realloc(OldSet::two, X);

The problem?  I'd start to modify the values in the new set of buffers, the third buffer being used and present.  Then the first buffer would be changed present... The second buffer changed present and the information is wrong (I over simplify massively here).

Anyway, I was remotely SSH'd into my server for this, so I went to Visual Studio, same code... Worked fine... So I go into my local VM and it's fine too, so I went back to the server and compiled manually and suddenly it's fine too.... WTF.

I literally spent an hour looking at this, the problem?  Well, it appears to be a bug in Clang, the reason the problem disappeared was my Makefile contains a $CC constant for the compiler to use and it was "clang" when I built by hand I used "g++".  Worse still, if I switched to a clang debug build the code worked fine, so this was something about my compilation process not a bug in the code per se.

So, perplexed I went in search of an answer.  And it appeared to be something about the clang optimizer, about which I found this talk from CppCon 2016.

Where there's this example:

#include <cstdlib>
#include <cstdio>
int main ()
{
int* p = (int*)malloc(4); // The original buffer above
int* q = (int*)realloc(p, 4); // The new pointer to the same old buffer
// Allocate a vlaue
*p = 1;
*p = 2;
if ( p == q )
{
printf("%d %d\n", *p, *q);
}
}

What do you expect this code to display?... Well, I expect it to print "2 2".  And it does on VC and G++ and even clang without the optimizer...





But you optimize the compile and its wrong:


Now, this is undefined behaviour and not caused by your code, it's the optimizer and very scary.  Not least as this was identified a while back (the talk along is from 2016) and g++ has solved the problem... Eeeek.

Friday, 18 January 2019

C++: The Dilemma of using Exceptions

As a C++ developer I've recently been in the midst of the Exceptions debate.  Exceptions as a feature of the C++ language have long divided the community, naively I've often put this down to their coming to the language from elsewhere (C perhaps) where exceptions were not the norm.

This was a mistake, for in the projects I've previously worked upon the overhead of an exception compared to the ramifications of the error resulting in a crash have often totally outweighed the overhead, making deciding to employ an exception exceptionally easy (*bum bum*).  Exceptions were the norm, still are and should be to you too (according to the C++ Core Guidelines https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#Ri-except).

The difficulty, the dilemma if you will, comes not from the feature, but the cost.  If you application domain demands high performance you can not risk using exceptions, the argument carries on therefore whether in not using them to avoid that overhead in performance you are actually cutting off your nose to spite your face... What are the downstream costs if an unexpected, in tested for, error occurs and some strange behavior ensues, or worse it propagates up and out of your application to crash?!!

In the modern era of social media such feedback is near instantaneous, can you afford the egg on face?

And so you spin yourself around and dig further into the ground your problems.

Voices from industry do little to alleviate the problem, google for instance (https://google.github.io/styleguide/cppguide.html#Exceptions), almost seem to advocate using exceptions if from the ground up you can build them in.  You stand the cost of retraining developers and change your ways of working around (e.g. embrace RAII).

They even go so far as to state "the benefits of exceptions outweigh the costs", but then add the caveat "in new projects".  How old or new your project should not define your error handling mechanisms, surely?  Plus from experience I've always found a code base is as old or new as the minds working upon it.  If you have an engineer firmly rooted in C without exceptions they will continue to churn code in that style through your compiler (which will happily work with it), not because it's a better way of working nor because it's a worse way of working, quite often it's continued along simply under the inertia of experience that developer has and they've not time to stop to take stock.

Even given that time, which the project I've just begun working upon has, there's weight behind the past, the old way of working, what worked before, and if that was non-exception handling code people stick with it.  Tell those self same people "you really should use these, oh and you have to retrain your minds, plus it'll take longer to compile and more time to run" they're going to stare at you like you're mental.


The question I'm left with is therefore not whether to use exceptions or not (well I am left with this problem, and this post is no help) instead we're left with the question of when will the benefits of exceptions, their ease of use, simplicity to write and visualise out weight the retraining, compile time and runtime costs?... I don't think that's anytime soon, it should be after so many years, but it's simply not.

x

Tuesday, 15 January 2019

Cross Compiling Clarity

I have an important distinction to make clear... Cross Compiling is using a machine with a different processor type than the target machine for your program.

So, you have a PC with an Intel Core 2 Duo in it, and you use a compiler to output an ARM executable, that is cross compiling.

If you're on an Intel Core Duo toting 15" MacBook pro (from 2006) and you use that to produce the same ARM executable, that's still cross-compiling.  BUT if you use that MacBook to generate a windows executable it's not cross compiling, as the processor is the same in both targets, you're not crossing over...

Some folks argue this differently, because you're crossing between the Windows and Apple OS's you're cross-compiling.  In my book, and by the definition given elsewhere, you're not, in this case you are cross-platform (the platform being the OS) but you're not cross compiling because the processors are the same family.

That's all, I just wanted this out there, argue as you all see fit.


Sunday, 30 December 2018

Programmer Between Jobs...

Announcement time, I'm unemployed... Yes, as of the 28th December 2018 I have not been gainfully employed.  But fear not, I am starting my new job on Wednesday 2nd January.

I've covered this in a nattering brief on my YouTube, which I now link, but stick with this post for more information.


I've never really spoken directly about the job I've had and how its evolved over the years I've been with it, so I was at that role for 14 years.  That's a very long time, especially in today's markets, but as you all know I keep myself busy and informed on these very pages.

I did work in what's euphemistically called "the pay to play industry", this is the gambling industry to the rest of the world, but saying that I didn't write the actual games.  I was a systems engineer, so I wrote software which drove the money accepting and paying hardware, and especially software which worked to store the current state of a machine, avoid frauds, count, collate and communicate metrics to central servers or offline (USB) uploads.

And I did all this according to a set of quite strict legal guidelines and requirements, so we're not talking about the much media covered "crack cocaine of gambling the FOBT", we're talking at most about £1 stake machines, family game machines, questions and answer style stuff.  The more cuddly end of the market if you're asking me.

I am leaving that industry, all that knowledge of BACTA standards and other stuff has been slowly ebbing out of my pores and leaving me, as I actually resigned in early October and in late October (when it became public knowledge in the office I was leaving) they placed me on gardening leave.  We'll gloss over that as some folks got their righteous fury out and pointed it in my direction; needless to say, being on gardening leave this long has let the move happen seamlessly, but it will be a shock to return to early morning rising to get into Nottingham City Center... Plus, no more day time tele... Oh thank god, no more daytime TV!

So, where am I going?  Well, I'm going to a company called The Multiplayer Guys, this is an actual game company, a computer game, console game crew with close ties to an actual game house Super Punk Games.

Now I can't share anything about what I'm going to be working on, but I'm carrying my C++, Windows and Linux experience high on my shoulders.  I'm eager to learn more about the Client Side structure of modern game engines and leverage my server-side and networking experience with the high-speed low latency efficient demands of a modern game.

So, yes, I know what game I'm going to be working on, that's the bit I can't share with you, maybe over time, but right now, nada, nothing, no comment... Move along, move along.



P.S.  I got an e-mail at 16:42 that my Laptop shipped, from China (despite being told the Lenovo reseller was in the Republic of Ireland) by UPS Global... I'm wondering if it'll be here within the week now, I really am.

Thursday, 27 December 2018

C++ High Performance - Mistaken Statements

I am a huge fan of using the language to communicate my intent, especially when it comes to function parameters, I've talked about this before in terms of simple typing (by utilising "using" statements - to give meaning to simple/trivial types) and const correctness.


But being Christmas, I've had a C++ gift or two, and one of them is C++ High Performance by Victor Sehr and Bjorn Andrist, I've only spent a few minutes looking through this book, but one comment did jump out at me as odd... Not overtly wrong, just odd to me, maybe a different way to look at things...


No, I'm not talking about their constant calling back to compare with Java (rolls eyes) I'm talking about the two stanza's at the bottom:

"C++ Arguments passed as references indicates that null values are not allowed"

No, no that's not the point of references, the point is that you do not allocate memory for and copy the values from the passed resource into a local value for use within the function which will have a scope life-time of the function and therefore be de-allocated at the end of the function.  A reference, to me, has always primarily been a method of communicating copy semantics, we're referencing the remote value, if the parameter is not constant it maybe modified, thus:

#include <iostream>
#include <string>

int g_value(42);

void Change(int& p_Value)
{
p_Value += 1;
}

int main()
{
std::cout << "Before: " << g_value << "\r\n";
Change(g_value);

std::cout << "After: " << g_value << std::endl;
}

Or not modified, thus:

#include <iostream>
#include <string>

int g_value(42);

void Change(int& p_Value)
{
p_Value += 1;
}

void Print(const std::string& p_Pre, const int& p_Value)
{
std::cout << p_Pre << ": " << p_Value << "\r\n";
}

int main()
{
Print("Before", g_value);
Change(g_value);

Print("After", g_value);
}

If we were to allocate a value as a pointer we could send the reference to these same functions and they would be unaware of subtle problems (without modern runtime checks):

#include <iostream>
#include <string>

int g_value(42);

int *g_PointedToMemory(new int{ 100 });

void Change(int& p_Value)
{
p_Value += 1;
}

void Print(const std::string& p_Pre, const int& p_Value)
{
std::cout << p_Pre << ": " << p_Value << "\r\n";
}

int main()
{
Print("Before", g_value);
Change(g_value);
Print("After", g_value);


std::cout << "--------------\r\n";

Print("Pointed To Before", *g_PointedToMemory);
Change(*g_PointedToMemory);
Print("Pointed To After", *g_PointedToMemory);
}

With modern runtime checks this code would still compile, you are not "defending" from null pointers by using the reference syntax, you're simply stating not to take a separate copy of the thing being passed in.  However, if you dereferenced a null pointer (nullptr) int* as the parameter being passed, the code would still compile, but you would receive a runtime access violation, as you're trying to write to nullptr.

#include <iostream>
#include <string>

int g_value(42);

int *g_PointedToMemory(new int{ 100 });

int *g_Unallocated(nullptr);

void Change(int& p_Value)
{
p_Value += 1;
}

void Print(const std::string& p_Pre, const int& p_Value)
{
std::cout << p_Pre << ": " << p_Value << "\r\n";
}

int main()
{
Print("Before", g_value);
Change(g_value);
Print("After", g_value);

std::cout << "--------------\r\n";

Print("Pointed To Before", *g_PointedToMemory);
Change(*g_PointedToMemory);
Print("Pointed To After", *g_PointedToMemory);

std::cout << "--------------\r\n";

Print("Unallocated Before", *g_Unallocated);
Change(*g_PointedToMemory);
Print("Unallocated After", *g_Unallocated);

}

This code is clearly dereferencing an unallocated piece of memory, it builds no problem... This is where my mind skews on the comment in this text, it's a throw away line, but so miss leading to my eyes.


But this code will not run:


This is where I totally take issue with the statement in the text.

Getting this kind of runtime error, because we've followed the advice of this text, we may move onto the next line of the advice:

"C++ arguments passed as pointers indicates that null values are being handled"... EREEEEERGGGGH.  No it does not, lets look at that, lets follow Alice down this rabbit hole with a new function, to take our unallocated integer pointer and try to dereference and print it... The text tells us "null" are being handled, we'll be fine, let us update our code:

#include <iostream>
#include <string>

int g_value(42);

int *g_PointedToMemory(new int{ 100 });

int *g_Unallocated(nullptr);

void Change(int& p_Value)
{
p_Value += 1;
}

void Print(const std::string& p_Pre, const int& p_Value)
{
std::cout << p_Pre << ": " << p_Value << "\r\n";
}

void PrintPtr(const std::string& p_Pre, const int* p_Value)
{
std::cout << p_Pre << ": " << *p_Value << "\r\n";
}

int main()
{
Print("Before", g_value);
Change(g_value);
Print("After", g_value);

std::cout << "--------------\r\n";

Print("Pointed To Before", *g_PointedToMemory);
Change(*g_PointedToMemory);
Print("Pointed To After", *g_PointedToMemory);

std::cout << "--------------\r\n";

// The alternative Print
PrintPtr("Unallocated Before as Pointer", g_Unallocated);

std::cout << "--------------\r\n";

Print("Unallocated Before", *g_Unallocated);
Change(*g_PointedToMemory);
Print("Unallocated After", *g_Unallocated);

}

And now lets run the program again...

Yes, it explodes, with the exact same runtime error... yet the text said "null values are being handled".  And it is just totally untrue, the truth is that pointers like this simply mean you can pass something which points to the memory you want to use (passing a pointer is like passing a small number of bytes - depending on your architecture - rather than passing a reference to the memory a pointer maybe anywhere in the addressable memory space, and it does not have to necessarily be a pointer from "new" or "make_shared" (etc) it can be any memory...

So, we could make this PrintPtr function print our allocated "g_value" from the earlier pieces of code, by making the call with a dereferencing:

I am going to plug on with this book, I hope to find another hour tomorrow to start going through their advice for speeding up modern C++.  Unfortunately, right now, I am surprise and somewhat dismayed with this miss-leading stance, and I have three other pages folded down to follow up on this exact theme, the text taking you slightly off of course or making statements which are not correct C++ nor correct thinking.

Wednesday, 26 December 2018

Deep Thought about a new Laptop

Its the day after THE day we wait for all year, yes today is Boxing Day and so yesterday was Christmas Day and I've had a fab day, thank you for asking!  Lots of food, a little drink, a fair amount of chocolate, a home made cheese cake to die for and about an hour in the freezing air enjoying the hot tub with all the bubbles on... Hands down has to rate as one of the Best Christmas Days for me ever... and I hope yours were all as good.

Now, I know you're all asking, why have I been so quite on the old blog and YouTube Channel?  Well, you know moving house completely threw things off, but also, in case you missed this news, I'm changing jobs.

Yes, after 14 years I'm leaving where I've been and I'm going to be doing something else.  Quite what I can't tell you very much about, probably for a fair while.  But what I can tell you is, I start the new role on the 2nd January, the team I have been assigned to is brand new and we travel to the US in mid-January to cement the bootstrapping of the new project.

And, this trip brings me to your sage pages.  You have all likely seen my post about my once great and mighty custom built Laptop and you may have also spotted my frustrations (admittedly eight years down the line) getting a replacement battery.

But it's Boxing day there are sales, there's a trip abroad breaching the surface waters, I am frustrated with the older machine (which I still love and am using right now)... Time for a new Laptop.

My requirements are a solid machine for programming, a good screen, RAM upgrade possible and definitely both SATA and M.2 SSD internally stored, I like lots of storage and I like to be able to replace it.  A replaceable battery, preferably as a pack is preferable and all of this as a known brand, so the battery would be on the market a fair while.

This means, GPU is not a concern really, it would be nice, but it's not a major concern, I've been down the gaming laptop whirlpool before it only ends in miserably depleted batteries and headaches replacing them.

The storage requirement I have also precludes eMMC or anything else which is fixed or soldered.

My first thought is for a quality machine, from a name, like Lenovo, Acer, Asus or Dell.


My very first laptop, way back in time was an IBM Thinkpad 360 a 486 SX 25Mhz, rocking 4MB of RAM and a 20MB hard drive running DOS6.22 and Windows 3.1.


I loved that machine, but lent it my brother when he started his business - and he had it stolen during a burglary never to be replaced... Grrr... But I loved the sturdy build on that machine, the keyboard was adorable, if I recall correctly it was a tiny form factor, like a 10" screen that did 640x480.  It travelled brilliantly in North Africa in the late 90's with work, and handled the Turbo Pascal and Turbo C programming I threw at it.  But due to the seeming prohibitive pricing and the changing of the keyboard I've never returned to the Thinkpad range, that thinking may change.


My second laptop is a veritable work horse Dell Inspiron 6400.  I say "is" because we still have this machine, it was purchased when I worked for "The Lordz Gaming Studio" as my workstation and a gaming machine... At the time, when on mains power, it ran beautifully (though I have a tale to tell when it was on battery)... And today we still have it, it's had more RAM and several larger hard drives over time, but we still use it daily as the general house laptop running Lubuntu (the wife still hasn't realized she was covertly switched to Linux about two years ago).


Wanting to get a brand name, I started where I left off and went to look at the Dell site, which  immediately splits Laptops by "Work" or "Home"... Clicking Work, foooooorget it, they immediately class you as requiring a Core i7 and price me out of the market, it's actually quite shoddy, in work I would say you need to pick the number of real cores to hyper threads, rather than limit the model number and so I know many of the Core i5 4 core 8 thread processors are being denied to me.  I therefore beat a hasty retreat to "Home".

The next thing I noticed is that all the Dell units seem to ship with 8GB of RAM.  Not unexpected 2019 is hurtling towards us at a rate of knots, but still it made me think about upgrade pathways, so set about finding tear down videos and images of inside these machines and was struck by how hard it is to pin down what you're going to be buying against what is on the screen, you can't tell, there's a skewing of the model number to contents within throughout their range, there's identifying model numbers which bear little to no relationship to the contents of the machine and worst of all many of the teardowns listed themselves as applicable to multiple models in the same lineage.  Telling me what?  Well, nothing firm, just that there's options beyond options but many of them look sort of fixes at birth for the boards within.  I saw one out of four units come up as having two RAM slots, many others had only one.  There was also no help trying to ask their online chat, they didn't know, they could only read what was listed with the product "it has 8GB of RAM"... yes, but can I put another one in?

However, I persevered and even started from a £679 machine to spec it up, however, I immediately saw that this whole line only had SSD storage a M.2 2280 no SATA storage option.  Another tear down video later and I confirmed this machine had only the M.2 slot, no internal SATA.  Which put it out of the running.  But it had given me an idea of a base processor I was going to aim for elsewhere, I was edging towards the Intel Core i5-8250U (6MB cache, up to 3.4Ghz).

This processor ticks many of my boxes, it seems to keep the base chasis of a machine at a low price, it has four real cores with hyper threading and a decent boost frequency.  Base frequency is lacking, but TDP is very desirable and it's battery life I'm concerned with.  And the units in built GPU (an Intel UHD 620) supports upto 4096x2304, and 4K maybe somewhere I go (I already have a 2K monitor as my daily driver).  It also supports up to 32GB of DDR4.

Similar processor counts, with slightly more cache, in the i7 range push prices up between £120 and £200, I could seriously live with this machine for 5 years and put that extra money into extra RAM and storage to add to a lower priced machine... So, the i5 is winning out and I'll carry on to other sites.

Now the Boxing day sales have kicked off, so I'm looking at the major high-street names and online retailers.  Like Amazon, AO, Ebuyer and CurrysPCWorld.

Whilst on Amazon and AO I saw a series of Acer and Lenovo units which were very low prices, like £150-£180.  The Lenovo IdeaPad 120 and HP Stream spring back into my mind, nice little units, too low-power for my linking and they're storage seems to be exclusively eMMC and tiny so not of use to me, but nice looking units, especially the Lenovo kit.

The only machine from this pass which really came to hand was the HP Pavillion kit, I saw a few others, but the Pavillion got really close.  It's a brand name, the hard drive is a 2.5" SATA and can be changed.  The RAM can also likely be upgrades with standard SODIMM modules.  However, hard data is hard to get and the price is still just shy of £700.  There's also no clear data on whether it can take both an SSD in M.2 form AND the SATA drive, there's a series of HP support images they themselves provide, but they're generic and clearly not specific to the unit I was looking at.  A minor annoyance is the charger also seems to be a custom pin type to HP, rather than a generic/universal USB/C type.

Without that firm information, with the higher price, I can't justify the risk of just not knowing, I need to know, so the HP now falls to the way-side.

A lot of web pages and a fair few Intel Ark comparisons later and I'm heading to the Lenovo site.

Now the Lenovo site immediately wants you to go down a series channel, so you're looking at the E, the T, the L and X and Yoga lines, there's options all over the place.  My immediate realization was the market (or maybe "experience" is a better description) experience of the Lenovo units, I was easily and quickly able to start see tear downs and information.

I started on the X and T series, quickly finding them drool worthy, but massively too expensive.  The E series came into view, I wanted to keep to 14" screen and so the E4XX series.  I'm an Intel bod, so E480.

So, a starting price of £431.99 today, with the Boxing Day 10% off.

The 2-3 week shipping missive has be worried, it may arrive as I'm on a jet half-way across the Atlantic and if we are in that situation, the machine has to be worth the wait.  But if I can pick an off the shelf version "Ready-To-Ship", I'm happy to get it sooner and upgrade it with parts myself.

The immediate problem in the "Ready-To-Ship" types are the processors and screens, they all seem to be Core i3 and the screens are 1366x768 (hello, I have 2012 on the phone for you Lenovo).  They're therefore straight out.

However, I can immediately see the memory customization options...

I can see 4+4 and 4+8, which is screaming that there are probably two memory slots and a quick google about tells me there are.  This means the 4GB (cheapest) configuration needs checking for a price to upgrade to a total of 8GB or 12GB by adding an additional 4 or 8 stick myself.

If we buy the 12GB configuration straight from Lenovo, then it'll cost an additional £111... Can we get a single 8GB stick cheaper?.... Why yes, a Crucial 8GB stick of the exact same spec RAM is only £49.99, less than half price.  Worth the risk of popping a brand new machine's case open?  Certainly.

But if I got the machine at 4GB spec, that's how I would fly with it, I'd order the additional 8GB, but park opening the machine until after my travel... Still, worth knowing.

What about storage? Well, the motherboard takes an M.2 SSD 2820 and a 2.5" SATA and we can elect to ship with one or the other... 

Playing the same game as with the RAM, we can see a cost of £85 for a 128GB M.2... Can I beat that?.... Yes, a WD Blue M.2 at 250GB is only £44.48.  That's twice the space for half the cost.  Lenovo, that is a pretty badly inflated price there.

We have to have the SATA mechanical drive, so our only question is RPM or the amount of space?  And, I think for now 500GB at 7200RPM would be fine and we can upgrade later (like a 2TB 2.5" SATA is only like £70).

The E480 is looking pretty darn good to far, we can get it in the i5, we can up the screen to HD and it has an HDMI out.  It's a USB-C type standard charger, we are all pretty much there.

It can't all be roses and light, what's the catch?  The battery.

Yeah, we're right back to where I came in on this whole problem of a new machine, changing the battery.  The battery on this is an internal unit, you can get to it, you can change it, but not easily and not on the road.

So, research time, can I find the exact battery type and can you buy them after market?


Yes, I can and Yes you can.  So we're onto a bit of a winner here.

Linux will have to come later, we're probably going with the lowest RAM and disk, to up the CPU and screen, lets price up where we might stand at the end of this glorious research...


That's not a bad price point, lets say we have the RAM (£49.99) and 250GB SSD (44.48) after market, we add £94.47 to that base price.  Giving us £607.99.

That's more machine, with head-room on the RAM expansion and storage for future proofing, 

You know what, I may reach out to Lenovo, see what they can do to assist this project?... If only to improve that shipping time.

Tuesday, 18 December 2018

Xelous Clothing Range

I was just wondering, whether these folks have realised that this overweight English and decidedly unstylish fellow typing this missive has been "xelous" since 1996...


Not the best search results for them from my point of view.



P.S. I am also NOT the Orc Warlock... Alliance represent!