Over the Fence.... And Far Away....

No, I'm not talking about horses this time.... Lets talk engineering, what is a fence?  A memory fence or barrier is used to order reads and writes around certain operations called Atomics.

This ordering ensures that both communication of the value from the register in the processor is correct between all the cores/threads running and the cache within the chip as a minimum, and may even leave to a full flush to the main memory (though that is far longer in cycle).

These can take tens to hundreds of cycles to leap the value over the barrier.

In C++ the most common two, or certainly which I encounter most often, are fetch_add and fetch_sub, using them to control everything from the shared count on a resource to the control gating on fixed sized "high performance" containers.

And there in lies the rub, these operations cost a lot of cycles just to increment or decrement a counter in many of my use cases, so why use them?

Well, that barrier within the chip, between the memory is very expensive if we compare it simply with the very cheap increment or decrement of a register itself, just the value in the register on the chip can change in a single operation instruction; sure it took others to load the register and it'll take yet more to store the result off, just as it would with the atomic; but on top of that you have no overhead in comparison with the atomic....

Until... Until you try to synchronize that flat increment or decrment, sure then that code is going to be far faster, however, it's not thread safe, not at all, the atomic already is (when ordered correctly)...

In order to protect a flat operation one therefore has to wrap a separate lock, or mutex, around it which is far far more costly than the atomic operation.  This difference is called the "contention cost", the contention cost of an atomic is simply in the number of steps, lets look at code:

The atomic addition, the CPU itself will execute

lock xadd [or similar]

This itself is a single instruction, it may result in multiple cycles of the CPU to complete, but it is a single instruction.  It ensures unique ownership of the cache line (usually 64kb) within which this variable resides, and means if you perform an operation anywhere in that cache line you will be making optimal operations.  As the CPU can perform all the atomic updates in that 64kb block without having to fetch another, this is really useful when there are a few cores (2-8 on average) accessing an area of memory and event holds up when scaling out to more cores.

A mutex however, has to be controlled wholly separately from the increment, so we may end up with C++ such as this:

std::mutex lock;

uint64_t count { 0 }; 

{

    std::lock_guard<std::mutex> lockGuard { lock };

     ++count;

} 

The execution here will have to acquire the mutex in a harsh manner, internally this is an atomic; if the pathway here is lock-free, then the atomic operation underlying the mutex is the only added cost.  However, and this is a HUGE however, if there is contention, someone else already has the lock then this lock-guard has to spin wait... And it's the contention, the other thing having the mutex locked, which adds the cost.

So you're essentially gambling on whether you have a value not contested before the lock or not, and in both cases you take on the cost of an atomic operation; so for my codebase and it's uses across sub 32 core machines means that an atomic is much more efficient in most all my use cases.

A mutex however is far more useful when protecting more than a single register, when protecting a whole block of memory, a shared physical resource (like a disk) or just a complex structure you can only use a mutex around it.

All this sort of eluded me earlier this evening, I was in the middle of a technical conversation and I bought up the atomics backing a shared_pointer in C++ and immediately just sort of lost it, my memory drifted far far away and I have to admit to waffling some.

I even forgot about weak_ptr deriving from a shared_ptr and it's uses to "lock" a new copy of the shared_ptr and so passing ownership by weak pointers.

But it came from a very specific conversation about Unreal engine, about TSharedPtr... Not a structure I myself have used, and for the life of me I could not think why not, I just knew not to having been told...

And of course here I sit a few hours later and I know why, it's not thread safe... TSharedPtr in Unreal is not threadsafe, and why is it not?  Well because it does not protect its internal reference count with an atomic, no it's just a flat inc and dec of a count integer register "for performance purposes".

So sure if you're inside one Unreal system, on one thread, then yeah you can use the TSharedPtr, but it's utility is much reduced to my eye, and you would want to perhaps look at other ways to hold your resources in that thread, even in thread local storage rather than in the engine heap.

The moment that TSharePtr crosses a barrier, then you're far far away from thread safe.

So what do you use a TSharedPtr for?  The documentation says "To avoid heap allocation for a control block where possible"... Yet it contains an explicit reference count, which is in a control block, and underlying it is a flat "operator new" and it uses only default deletion via the flat operator delete.... So my non-Unreal expert brain says "Why use it at all".

Hence when asked earlier today my memory was over the fence and far far away.... Of course now, it's all returned and... yeah I just sat through two hours of TV mulling this over.... and had to come write something down before I went a little bit odder than usual.

Tomorrow let me regale you with my story about forgetting how to load a crash dump and the symbols files and explain myself properly, despite doing that particular duty about a hundred thousand times in my life.  Hey ho, a technical conversation in which I fouled up, one is only human.

C++: I was today years old when I learned this about shared_ptr

I was on a review for a colleague and there was this piece of code which stood out to me, I could not figure out what was being achieved.  Without being specific it was essentially

static std::unique_ptr<B> Create(const std::shared_ptr<A>& referenceToA);

Just the function prototype didn't sit right with me and the use case was even more strange to my eye for the "B" structure here being created has the shared pointer in the members.

struct B
{
    B(const std::shared_ptr<A>& referenceToA)
        : mMyA(referenceToA)
    {
    }

public:
   static std::unique_ptr<B> Create(const std::shared_ptr<A>& referenceToA)
    {
        return std::unique_ptr<B>(new B(referenceToA));
    }
};

private:

    std::shared_ptr<A> mMyA;

And my head just lot it.  To my eye this API, the create function here, is given a constant reference to the shared_ptr and what can be constant about a shared_ptr?  Well the internal reference count, or so I thought.  I believed this was a mistake, I believed the compiler would throw this one out and say "No, you can't change the reference count of this const object".

Never had I ever thought that the shared_ptr is actually referencing some other controlling block elsewhere.  My understanding was therefore flawed and I'm happy to admit naive.

So what would happen here?  Well, the B constructor, actually copies the shared_ptr control block, it therefore does and can increment the reference counter to the shared_ptr.

As counter intuitive as the const is therefore.  We aren't actually saying the shared_ptr itself is constant, rather the reference to it is, we should be reading the parameter type as std::shared_ptr<A>const &  referenceToA.

I felt rather silly for not realising this earlier, not least as I did once write a C program to switch out the qualifiers on a function call to make things like this stand out in formatting my code!!

But it has slipped my mind! Okay! I'm old, shut up.

Here is the full code of what I put together to understand this: 

#include <memory>
#include <mutex>
#include <queue>
#include <string>
#include <cstdio>
struct A
{
A()
:
mIndex(GetNextIndex())
{
}
~A() = default;

const int mIndex;
private:
static int GetNextIndex()
{
static int index{ 0 };
return ++index;
}
};
struct B
{
public:
B(const std::shared_ptr<A>& parent)
: mParent(parent)
{
}
std::shared_ptr<A> mParent;
static std::unique_ptr<B> Create(std::shared_ptr<A>const &  referenceToA)
{
return std::make_unique<B>(referenceToA);
}
~B()
{
mParent.reset();
}
const int& GetParentIndex() const { return mParent->mIndex; }
};
int main()
{
std::shared_ptr<A> original{ std::make_shared<A>() };
printf("Original %i: %i\n", original->mIndex, original.use_count());
auto createdB{ B::Create(original) };
printf("Copy %i: %i\n", createdB->GetParentIndex(), createdB->mParent.use_count());
printf("Original %i: %i\n", original->mIndex, original.use_count());
}

However, I have to say, I still don't like this; clever as it is, there's a hidden copy going on, in the B constructor the copy of the shared_ptr control block and it's reference count incrementing from 1 to 2.

It does quite neatly move ownership of the pointer, but the const and the reference just make my brain spin.  So where this can only gain 10/10 plaudits for C++ smarts, it only gets a mere 3/10 on the maintainability ladder for me, and only when we have the const& together, when written as at the top of the page this drops to 1/10.

Performance is also a factor here, if we wanted better performance we would have to consider whom owns the original A here.  If no-one and it is always added into the shared member in B, well move it... Create once and move it, this code doesn't communicate that as an option, but it certainly could be.

Software Engineering : C++14 Factory Create Pattern for GCC

In my previous Software Engineering item, I covered a factory create pattern, to prevent programmers using my classes with their vanilla constructors, forcing them into using a smart pointer in the hope that they would never get a memory leak from using my classes (assuming my classes clean up cleanly internally).

This pattern can be seen here, but is specific to the Microsoft STL implementation of the Memory classes.

On Linux however, with GCC, you have to make a different class a friend, this is "__gnu_cxx::new_allocator", which like "_Ref_count_obj" is an implementation of a counter to references to the allocated class type.  It is also templated, and used exactly the same way as the Microsoft version.

Use it like this:

You can see my whole screen, with all the code here:

Click to Zoom in

On the left is the example code, on the right you see me invoke g++ with C++14 to compile and then run the example program.

The code clearly shows the private constructor, it's a trivial class, if any of that confuses you then this maybe too advanced an item for you to pick up on just yet, but do come back and learn this, as it's very useful when you've built up your C++ confidence.

Software Engineering : C++14/make_shared & Factory Create Pattern

I have a C++ issue, it's not often I can pick fault, but this is one of them... For years, I've used the factory create pattern; or at least my take on that pattern.  

Whereby I make the constructor private, or the whole class "Uncopyable/Uncreatable", and then I add a new static function which returns the class wrapped into a smart pointer.

By doing this, I get the advantage that no-one can use outdated "new" calls on it, and I drastically reduce the potential for third party memory leak creation/exploitation.

I can still do this with C++14, however, for a while now authors have recommended using "std::make_shared".  Their points are valid, but they break my factory creation pattern.  Which has much more to offer me than just using "make_shared" or whatever smart pointer creation template.

Lets take a look...

So, we have the class, but we're creating and deleting it in the old style, anyone forgetting to perform that delete will leak memory.

There, we can implement creating the class instance with a smart pointer...

A big change next however, we make the constructor private, to absolutely stop other programmers creating instances of our class with "new" and so totally negating anyone creating the class and potentially forgetting to delete it.

So we offer the programmer the static create function, which returns the shared pointer wrapped class... At least, that was the idea, as we can see, it utterly fails, because the std::make_shared template class is not a friend of our "foo" class.

Finally, we have to take a step back, to use the create, and hence enforcing our whole factory create pattern, we have to return this smart pointer wrapped instance after using "new".  This is fine, it is the same as "std::make_shared", but feels like a retrograde step.