C++14 std::memset Not Working?

When I say not working, I do not mean the function does not work, of course it works.  What I mean is that left to it's on devices the GCC compiler can actually chose to not perform a memset in certain circumstances, you can read more detail about this in the official bug list.

So, what's going on?  Well, I have a piece of code which uses a common buffer, each function locks a mutex (std::lock_guard), fills out the buffer and then transmits over USB to target devices, crucially I then want to clear the common buffer.

The purpose of this architecture is to keep sensitive information being transmitted through the buffer present for as little time as possible (i.e. the time between filling out and transmit, ~35ms) rather than filling out, transmitting and leaving the data in the buffer; as the time between calls may be anything up to 5 seconds, plenty of time for someone to halt the program and inspect memory dispositions.

In pseudo code therefore our sequence looks something like this:

COMMON BUFFER [256]
CLEAR BUFFER

USB CALL:
    FILL  BUFFER
    USB TRANSMIT
    std::memset(BUFFER, 0, 256);

In debug this worked no problems, all my soak testing worked, I was quite happy.

Until one switches to release, whereupon the GCC compiler can opt to optimise away calls which it defines as not having any noticeable effect.

In the case of the memset call here, to the compiler, locally sees nothing inspect the BUFFER variable after the memset is performed, it therefore decides nothing is bothered the buffer is cleared or not.

Of course we know the next call to any call, or the same call, will rely on the BUFFER being empty when it enters.

There are therefore two problems, if one had this code:

USB CALL:
    std::memset(BUFFER, 0, 250);
    FILL  BUFFER
    USB TRANSMIT
    std::memset(BUFFER, 0, 256);

You would of course pass all your tests, even in release, the buffer would work as expected, you could even assume no data is being left in RAM between transmit calls; you'd be dead wrong, but you can assume anything you like really.  No, what's happening here of course is that the lead-in memset is clearing the buffer before use, but the sensitive data is left in the buffer between calls as the latter memset is still being optimised away.

The code remains vulnerable to introspective intrusion attempts.

The real solution?  Well, I've gone with:

USB CALL:
    std::memset(BUFFER, 0, 250);
    FILL  BUFFER
    USB TRANSMIT
    std::memset(BUFFER, 0, 256);
    auto l_temp(buffer[0]);
    buffer[0] = buffer[1]
    buffer[0] = l_temp;

Whatever you do to solve this, don't spend a whole day as I just have.

In conclusion, of course the call works, it's the compiler which is optimising the call away.

Software Engineering : "warning: defaulted and deleted functions only available with -std=c++11"

warning: defaulted and deleted functions only available with -std=c++11 or -std=gnu++11

When you're using std=c++14??!?!?!  What the heck is going on here?

I've noticed this strange bit of behaviour with g++ and warning when compiling with multiple cores... I've stripped this example back to the bare minimum, so lets just define what we're building and then we'll look at the strange warning which comes out, first of all, we need to compile two files, completely independent of one another:

Main.cpp

#include <iostream>

int main ()

{

std::cout << "Hello World" << std::endl;

}

This is our first file, the other has to be a class:

Data.h

#ifndef DATA_HEADER

#define DATA_HEADER

#include <string>

namespace Xelous

{

class Data

{

private:

std::string m_Data;

public:

Data(const std::string& p_Data);

const std::string& GetData() const;

};

}

#endif

And the code for this class looks like this:

#include "Data.h"

namespace Xelous

{

Data::Data(const std::string& p_Data)

:

m_Data(p_Data)

{

}

const std::string& Data::GetData() const

{

return m_Data;

}

}

This code so far is all fine, however, in our data header we don't want the default constructor, so we're going to just delete the default constructor, like this:

#ifndef DATA_HEADER

#define DATA_HEADER

#include <string>

namespace Xelous

{

class Data

{

private:

std::string m_Data;

public:

Data() = delete;

Data(const std::string& p_Data);

const std::string& GetData() const;

};

}

#endif

Fairly simple stuff so far, and no problems, no errors... However, I always build with "pedantic", especially as I get closer to release and I'm looking at code, so lets see my build file for the above project:

CC=g++

STD=c++14

WARNINGS=-Wall -Wfatal-errors

PEDANTIC=-pedantic

OUTPUT=example

MAIN=main

DATA=data

CompileMain: ${MAIN}.cpp

${CC} -std=${STD} ${WARNINGS} ${PEDANTIC} -c ${MAIN}.cpp -time

CompileData: ${DATA}.cpp

${CC} -std=${STD} ${WARNINGS} ${PEDANTIC} -c ${DATA}.cpp -time

Lets just stop there, and look what we have in the build file so far, we have the compiler, the STL version to use, the warnings, we're using pedantic and I'm only compiling the two files, there's no linking going on.

I always prefer this so I can trap individual compile errors or problems, speeding up the over all development... Once I'm happy both classes or files are clear I can then combine them into their link, by adding:

Link: ${DATA}.o ${MAIN}.o

${CC} -o ${OUTPUT} ${DATA}.o ${MAIN}.o -time

This will link everything up...

There isn't a problem with this code, we can save the file now and call "make" on those three targets and it will work fine... Let us just complete the make file:

clean:

rm -f ${DATA}.o

rm -f ${MAIN}.o

rm -f ${OUTPUT}

clearscreen:

clear

all: clean clearscreen CompileData CompileMain Link

These last three targets are our clear, a simple clear screen and then the "all" target, 

So, we can now "make all" and see this output:

Building one after the other is fine, however, if I use "j2" that is to make it build data and main on different cores at the same time, then link the result of both you get the above error:

Both compiles are being sent to the compiler with c++14, when we're using serial single focus compilation there is no warning, yet with two the warnings pops out?

I've actually run out of time on this very busy Sunday to look into this any further at the moment, I'm therefore going to schedule this for tomorrow and let it loose on the world...

Software Engineering : C++14 Factory Create Pattern for GCC

In my previous Software Engineering item, I covered a factory create pattern, to prevent programmers using my classes with their vanilla constructors, forcing them into using a smart pointer in the hope that they would never get a memory leak from using my classes (assuming my classes clean up cleanly internally).

This pattern can be seen here, but is specific to the Microsoft STL implementation of the Memory classes.

On Linux however, with GCC, you have to make a different class a friend, this is "__gnu_cxx::new_allocator", which like "_Ref_count_obj" is an implementation of a counter to references to the allocated class type.  It is also templated, and used exactly the same way as the Microsoft version.

Use it like this:

You can see my whole screen, with all the code here:

Click to Zoom in

On the left is the example code, on the right you see me invoke g++ with C++14 to compile and then run the example program.

The code clearly shows the private constructor, it's a trivial class, if any of that confuses you then this maybe too advanced an item for you to pick up on just yet, but do come back and learn this, as it's very useful when you've built up your C++ confidence.

GNU C/C++14 Installation & Codeblocks 16.01 from Source (Command Line)

In yesterdays post I explained a C++14 user was having instant issues with the vanilla install of gcc/g++ on Ubuntu 14.04 LTS.

Getting a C++14 Compiler
So, here today are my command-line steps to update the GNU Toolchain v5.x.

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install gcc-5 g++-5

If you already have compiler alternatives you may need these lines.

sudo update-alternatives
sudo update-alternatives --remove-all gcc
sudo update-alternatives --remove-all g++

But, everyone will need to swap the default gcc and g++ command-lines to the new paths to make them the defaults.

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 20
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-5 20
sudo update-alternatives --config gcc
sudo update-alternatives --config g++

These last two --config commands are only needed if you have multiple alterantives, so don't worry if it tells you it's failed as you only have the one.

Now, if you perform g++ --version, you should see it's a version 5.x series compiler.

Codeblocks 16.01
Older versions of Codeblocks may start to error on the code completion with some of the C++14 specific commands.  So, we need install some prerequisites, and then build the latest 16.01 version from source.

sudo apt-get instal gtk+-2.0 automake libtool libwxgtk2.8-dev libwxbase2.8-dev

wget http://sourceforge.net/projects/codeblocks/files/Sources/16.01/codeblocks_16.01.tar.gz

tar -xvf codeblocks_16.01.tar.gz

cd code*

The next step is interesting, we need to iteratively check:

./bootstrap

Running this will show you anything wrong with your machine environment, any missing dependences etc... However, once bootstrap runs cleanly, you can continue below.

./configure
make
sudo make install

The make step takes quite a time, the more cores & RAM you have the better, on an 4 core (8 thread) machine with 8 GB of ram, I've found it takes about 10 minutes.  On a single core machine as the poor VM I had was assigned, it took a lllooooottttt longer.

Once complete we needed to use the text editor of our choice, in sudo mode....

sudo nano /etc/ld.so.conf

And to this we need to add the line:

include /usr/local/lib

Save the file, exit the editor and run:

sudo ldconfig

Once this was complete, we can run up Codeblocks, and see it's version 16.01.

And we can further see the nice new C++14 options in the build options:

Even the code highlighting recognises the make_shared operation:

Voila, if this helped, do check out my other posts, and code, leave a tip with my new tip jar!  And I'll be making a video of this one soon, as it's so useful.

WarThunder on Linux

Yes, it is official, WarThunder is available on Linux!  This was a bit of a secret as I had access to the game on Linux with thanks to Gaijin.

http://warthunder.com/en/news/2608-war-thunder-is-now-available-on-linux-en

There were a few bugs, and communication was sadly pretty poor as rather than reviewing the game in any fashion one was simply playing and reporting bugs.

The experience however is identical to the Windows and Mac release, the graphics being rendered in OpenGL (which you can even do on Windows and you always do on Mac all the time) so the porting of the game was essentially down to their moving their implementation.

With libraries like SDL this is an easier task then before, however, WarThunder is now, without doubt, the most complex and detailed game I've seen on Linux. 

And it marks a massively interesting change in the fortunes as game developers look at the platform.

The Witcher 3 and other AAA title games are also appearing for the platform, however, WarThunder with both it's land, air and soon to be sea game modes is far above those scripted screen play type games.

I'm certainly interested in how head-tracking and complex flight stick controls now start to improve on Linux, with this fantastic game now on the platform.

glfw v3 on Debian/Mint/Ubuntu

Please Subscribe now to help me reach 1000 subs :)

boost libraries and -Weffc++

I've got a peeve for the day, I've come to use the boost libraries, specifically signals2, in a little project and I wanted to knock up some code to demonstrate what I was up to quickly... However, I've just sat with Code::Blocks spewing beeeellllions of errors and warnings at me.... All from code which looks fine.

The warnings and problems are all coming from Boost... Don't include boost and all is well... but I know boost is fine, I know thousands of people use it, I know its tested... why... oh why is my project cocking it up?

Well, it seems I use -Weffc++ in my compiler scripts, by automatic and default, I like the effective warnings... The problem is, this switch breaks using boost... Boost clearly does not listen to Scott Meyer.. grrr.

So folks, if you want to use boost, don't use the g++ switch -Weffc++

Guide To Building a Raspberry Pi C/C++ compiler on Ubuntu

These are personal notes on setting up a Cross Compiler for C/C++ on my Ubuntu 12.04 machine, so I can build programs for the Raspberry Pi.

Of course my reason for this is that my PC's are far more powerful than my Pi, even compiling the most trivial program can result in waiting a duration or two on the Pi.  So, I want to employ my heavy PC metal to generate the ARM architecture code for me.

The first steps were to install all the pre-requisites, I'm assuming you have Ubuntu 12.04 32bit LTS installed fresh and clean (either as a real machine, or as a virtual one) and that you know how to use your Linux Machine, here we go.

sudo apt-get update

sudo apt-get upgrade

sudo shutdown -r now

Reboot... Log back in etc....

sudo apt-get install subversion bison flex gperf build-essential texinfo gawk libtool automake

Download ncurses tar ball from "http://ftp.gnu.org/pub/gnu/ncurses" extract it and then within its folder:

./configure

sudo make install

Download crosstool-ng from "http://www.crosstool-ng.org", extract its tar ball and from its folder:

./configure --prefix=/opt/cross

sudo make install

Now we need to add this built item to the path:

PATH=$PATH:/opt/cross

And now we need a space to do the building of the cross compiler:

cd ~

mkdir Pi

cd Pi

/opt/cross/bin/ct-ng menuconfig

Goto "Paths and misc options" and then turn ON "Try features marked as EXPERIMENTAL".

Goto "Target Options" and into "Target Archetecture" and select "ARM".

Go into "Target Options" and set Endieness to "little".

Go into "Bitness" and set it to "32bit".

Goto "Operation System"

Go into "Target OS" and select "Linux".

Goto "Binary Utilities" and then select "Binary Utilities" latest version (2.22 as of writing).

Goto "C Compiler" and enable "Show Lincro Version" these are the 4.7+ versions supporting C++11, and enable C++.

Exit and save.

And now we kick off the building of the tool chain:

/opt/cross/bin/ct-ng build

The output for me ended up in "~/x-tools/" as per the paths selected in the paths & misc options.

I now pronounce you GNU

I'm struggling with a pronunciation... how to pronounce GNU?  Their website tries to explain, but it has only left me more confused:

"The name “GNU” is a recursive acronym for “GNU's Not Unix!”; it is pronounced g-noo, as one syllable with no vowel sound between the g and the n."

I've always (since about 1996) pronounced GNU as "GaNoo", the involuntary vowel after the 'G' being natural for me as a native English speaker.  The involuntary vowel makes the strange contrast of the consonants there work in English.

I've hard some people pronounce it as "GæNew", "Ger~Noo" and heard them give up totally and call it "G.N.U".  I have always known what they're talking about, and they've always known what I've been talking about.  After all accents are so varied in Britain you never can be sure what accent, or language, you'll get out of someone so we accept a certain amount of variance to get the job done.

So it pains me to see from the site of GNU that I've been saying it wrong all these years... "one syllable"... I've definitely been using two... and no vowel sound between g and n.... gah, I can't do that, I simply can't... I'm not a linguist, and my English brain says "stick an 'a' sound in there it'll be fine".

There's a great British tradition of co-opting a phrase to suit our tongue, it drives other lingual structures to distraction... Ever been to France and tried to order a coffee in perfect French but with a rich Lancashire accent?  I've seen someone try and get stared at like a they were trying to show a dog a card trick, that involuntary head twist from the French as you accidentally drop the feminine instead of the masculine for some object or other... I drive the French mad, I once had a woman scream at me (in French) about my pronunciation of "5".. not Five... Cinq... "Sank... Sanque... Sink... Five love, five" (hold up the digits of one hand.... much quicker).

So, with GNU I'm going to stick with GaNoo.  Lets hope you still pronounce Hurd, Hurd...