Showing posts with label help. Show all posts
Showing posts with label help. Show all posts

Saturday, 9 December 2017

Start C++?

I've been writing C++11 or better code, well since 2010, as I started doing so with the TR1 as was.  We're nearing eight years since then, and it starts to show.

So, where would I recommend starting to learn modern C++?  Well, if you've never programmed before, don't start by learning C or C++, go learn Pascal, or Python, or something else which is more friendly.  I started out in Pascal, for a good three years, before I started to work in C and later moved into C++ (in 1998) so if 20 years of C++ teach me anything, it's don't try to learn if as your first language.

Once you have a concept of how to program, then start to learn C++, and I would recommend finding someone - hopefully like me - and asking them.  An hours chat with them, to help swap what you know with what they know, is a good start.

Saving any such friends, YouTube, watch CPPCon, BoostCon, watch talks about programming C++ but most importantly get a development environment and cut some code, if you must go for a community edition of Visual Studio 2017, but otherwise get a Linux machine up and running and have a play.

If you work in an office where there's a C++ guru, ask them, if they're worth their salt they'll be more than happy to take you through a few things.

And failing that, I include a set of programs below, one through three, these are the most simplistic C++ programs I would recommend you start off working with, and if you want to know more comment below.


So, open an editor, and write this C++ code, then save the file as "main.cpp":

#include <iostream>

int main ()
{
   std::cout << "Hello World";

}


If you are in Visual Studio, you run that file, if you're in linux close the editor after saving and lets use the gnu g++ compiler (install it in Ubuntu say with "sudo apt install g++"), and you compile this into a program like so:

g++ main.cpp -o example1

Your program output will be called "example1", and you can run that program, and it'll say "Hello World".

The code itself I want to only explain the first line... the include, this tells the compiler to include some code for you to use, and "iostream" is the input/output streaming functions for you.

int main is the first function the program starts to run from, int is the return type, meaning integer, but in C++ we don't need to return a value here - if anyone argues with you about this, they're wrong.  The name "main" is the a special name and the compiler makes sure your program starts with this function entry point.  The empty brackets show we're not passing anything into the function, there are no parameters.

The braces mark out the body of code for the main function, so it starts with an open brace, contains lines of code and then ends with a close brace, and yes I call them "braces" not "curly brackets" :)

The one and only line of code we've got left is the output call, this is streaming the value on the right of the "<<" chevrons to the standard character output stream... "std::cout"... I'll reiterate 'standard character output stream".   And the value we're streaming is a string (note the quotes in the code) "Hello World".

This is the only line of code with the all important semicolon ending, this is used to tell the compiler that we're done with out line of code.

Go, try this...


The next most basic program for C++ noobies to learn, is to maybe output some more kinds of data.... After say, asking the user something...We've seen cout, how about "standard character input"?...

#include <iostream>
#include <string>

int main ()
{
    std::cout << "What is your name? ";
    std::string name;
    std::cin >> name;
    std::cout << "\r\n";
    std::cout << "Oh Hi " << name;
}

We're introducing a new header, the "string" header helps us store strings of characters in the type "standard string"... "std::string"... Did you spot that?... This is our first variable, and it has the name "name" so we can refer to it later.

As ask the user a question by sending characters out "<<" to the output stream, and then we read them back in ">>" from the character input stream, and we read them into the "name" variable we just created.

Next we output a carriage return "\r" to move our carat back to the left of our screen, and we move to the next line "\n".

And finally we output a piece of text AND our input variable!

You can play with this code with other variable types, other than string... Try "int" to read in a whole number, of "float" to read in a floating point number.


But our third program, will involve some actual processing.... Lets average the age of a set of people we ask the user to input...

#include <iostream>

int main ()
{
    int TotalAge (0);

    int age;
    std::cout << "Enter the name of person 1: ";

    std::cout >> age;
    TotalAge = TotalAge + age;
    std::cout << "\r\n";

    std::cout << "Enter the name of person 2: ";
    std::cout >> age;
    TotalAge = TotalAge + age;
    std::cout << "\r\n";
    
    std::cout << "Enter the name of person 3: ";
    std::cout >> age;
    TotalAge = TotalAge + age;
    std::cout << "\r\n";


    float Result = TotalAge / 3;

    std::cout << "The average Age is: " << Result
}

Here we have much more code, we ask for each persons age, adding it to the total, then we calculate the resulting average age and print it out.  Take a moment to look at this....

What do you see?  If the first thing you see is that there are three sets of the same code in there, then you have the seed of a programming within.  If however you just see a mess, then maybe C++ isn't for you.


Thursday, 2 November 2017

C++ : Ignored qualifiers (-Wignored-qualifiers)

What is an ignored qualifier?  Well, in a class, lets have a student class:

#include <string>

class Student
{
    public:
        const std::string Name;

    private:
        bool m_Present;

    public:

        Student(const std::string& p_Name)
            :
            Name(p_Name),
            m_Present
        {
        }        

        void SetPresent() { m_Present = true; }

        void SetAbsent() { m_Present = false; }
};

Now, we may want to access the present flag, therefore provide a function to do so:

    bool Present() const { return m_Present }

This function is telling the user of our class quite a bit, it tells them that the return type is boolean and that the calling of this function makes no changes to the class  contents (const trailing the parameter list).

However, for me, this isn't quite right, I believe that we want to inform any user that the boolean returned is also constant, it does not change unless you alter the internal value with the "Set" functions, therefore I prefer and like to see code stating:

    const bool Present() const { return m_Present; }

This is perhaps overkill and most of the time completely acceptable code to present, however, some might prefer not to see it, specifically anyone defining "-Wignored-qualifiers" as with the "const bool" the const here is technically superfluous, the return type is boolean and a new instance of it, it is not a reference to the internal value, if it were the function may look something more like this:

    const bool& Present() const { return m_Present; }

Now we are intrinsically returning a reference to the internal boolean, or even:

    const bool* const Present() const { return &m_Present; }

For speed of operation we may directly drive the reference back as a constant boolean constant pointer.

I find this much more informative to the user, they know our intent, the code though more verbose communicates its meaning much more clearly.

As ever, yes I have seen questions asked of interfaces where "bool X()", or "bool X() const" is provided but then programmers have asked "How do I change X", with the const return, ignored qualifier or not, the know not to ask this function to change X and can look up elsewhere in your API.

Wednesday, 10 May 2017

Development : Python, MySQL and Protocol Buffers Build

Today I've come to a totally virgin installation upon a server, this was for a work group I've got to head up whom are looking at pushing MySQL with Python.  And things initially went wrong...

I stipulated they had to use Python3 and thought everything else would be fine for them to install with Pip3, so...

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python3
sudo apt-get install python3-pip
sudo apt-get install mysql-server

Everything looked fine, their user could install packages through Pip3 and they had Flask and a few other things and were flying.  I used the mysql client on the command like to test I could add a few items and also ran...

sudo mysql_secure_installation

To do the basic hardening of the database, so everything was fine... Right?....RIGHT?!?!

No, not exactly.... "I can't get mysql.connector".... Came the cry.

And they were right, it reported a myriad of build issues and could not install.  I took a look... NIGHTMARE!

It appears the installation of mysql.connector via Pip3 depends upon Protocol Buffers from google for the latest version of mysql.connector... Which the Pip install didn't sort out, at least not easily... Luckily I run a whole gaggle of virtualized machines, so I could quickly spool up a new instance and try a few things out...

This is the script I came up with....

cd ~
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install -y python3-pip git autoconf automake libtool curl make g++ unzip
sudo pip3 install flask
hostname -f > hostname.txt
sudo apt-get install -y mysql-server
sudo mysql_secure_installation
sudo ldconfig
cd ~
git clone http://github.com/google/protobuf.git
cd protobuf/
./autogen.sh
./configure
make
make check
sudo make install
sudo ldconfig
cd python
sudo python ./setup.py install
cd ~
sudo ldconfig
sudo pip3 install mysql-connector --install-option='--with-protobuf-include-dir=/usr/local/include/google/protobuf' --install-option='--with-protobuf-lib-dir=/usr/local/lib' --install-option='--with-protoc=protoc'

Lets go through this step by step...

cd ~
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install -y python3-pip git autoconf automake libtool curl make g++ unzip

This is just the basics, so our installation will now depend upon git, autoconf, automake, libtool, curl, make, g++ and unzip, most all your standard distributions will contain packages for these, we use -y just to skip any permission requests.

sudo pip3 install flask
hostname -f > hostname.txt

The next step is simply a couple of items for our project, we're going to use Flask to provide a restful interface, and the "hostname.txt" is simply to remove our need to call "hostname" again later.

sudo apt-get install -y mysql-server
sudo mysql_secure_installation

Our next step is to install and secure the MySQL service.

sudo ldconfig
cd ~

Generic code now, to simply reload the library list and change to the home directory.

git clone http://github.com/google/protobuf.git
cd protobuf/
./autogen.sh
./configure
make
make check
sudo make install

This is the build of protocol buffers from google, so we pull it from their github repo, we move into that folder, prepare and configure the build, then make the whole build.  By far this is the LOOOONGEST instruction, on a single core 1GB equipped virtual instance this took around 45 minutes.

Once complete we simply need to reload the libraries again...

sudo ldconfig

However, protocol buffers are still not installed within Python, so we are still in the "~/protobuf" folder, we now need to go deeper, into the python folder and perform the setup installation under python...

cd python
sudo python ./setup.py install

When complete we again need to reload the libraries...

sudo ldconfig

And the final, secret sauce, is to actually install mysql connector through pip3 with protocol buffers...

sudo pip3 install mysql-connector --install-option='--with-protobuf-include-dir=/usr/local/include/google/protobuf' --install-option='--with-protobuf-lib-dir=/usr/local/lib' --install-option='--with-protoc=protoc'

This is a single command, spanning one single line.

And voila, once complete you get to use mysql.connector in your python code...

import mysql.connector
import gc

con = mysql.connector.connect (user='whatever', password='something', host='localhost', database='yeahyeah')

con.close()
con = None

gc.collect()

You can find out more about why I nullify and garbage collect a connection in my previous post.

Thursday, 27 October 2016

Administrator : Linux Network File System (NFS) Mounted Drives

Over the next few days I'm planning to bring you at least three videos about sharing files between different systems, specifically Windows and Linux... Today the easiest (at least for me) Linux to Linux sharing.

For this you will need SSH access and a user account on the remote system, and sudo (root) rights to both machines.  I'm running Ubuntu machines here, both for the client and the server, which variant (32/64) makes no difference.

The Server
sudo apt-get update
sudo apt-get install nfs-common nfs-kernel-server

We need the nfs-kernel-server here, and it will run as a service, once it's all installed we need to make a folder, I create them like this, making it owned by myself:

sudo mkdir /media/xelous
sudo chown xelous /media/xelous

Then I edit:

sudo nano /etc/exports

And I add to it:

/media/xelous     150.0.8.*(rw,no_root_squash,async)

This is the local folder we're mounting, and we're making it available to ALL the machines on the "150.0.8.1 to 150.0.8.255" range of IP addresses.

Saving this file, I then need to restart the whole machine, or just the service:

sudo /etc/init.d/nfs-kernel-server restart

You can then run:

showmount -e

To see the local mount you've just created, if you have an issue, and it doesn't show up, check the above again... because it does work, honest.... The most common problem is permissions on the folder you've created, sometimes on systems you are not the administrator on, it's best to share a folder from your /home directory.

The Client
The client is a simpler installation:

sudo apt-get update
sudo apt-get install nfs-common

Then you can check the remote mount, lets assume the server is on IP 150.0.8.40:

showmount -e 150.0.8.40

You should see the remote mount you created on the remote machine:

Lets create a folder locally, into which we'll mount the remote folder:

sudo mkdir -p /media/remote
sudo chown xelous /media/remote

Now, I happen to be the user "xelous" on both machines, but change your username for the local or remote machines... Mine is not best practice here, as they just have different passwords....

To mount the remote folder locally:

sudo mount 150.0.8.40:/media/xelous /media/remote

So, this is mounting the remote to the local, on the local machine I can then just hop into that folder and work, knowing all the files are trickling out over the network and into that remote machine.

This is very useful if you're going to run a thin client system, or are working on a machine with no, or read-only, local storage.

Why does this exist?
The driver behind this was my main development machine running out of disk space, and my not being allowed to install a new drive... yes, go figure (don't worry, I have asked the fair fellows of IT for access to my BIOS again - Yes, I'm still on a machine with a BIOS not UEFI, don't laugh).

So, with my workstation critically low on disk space, where was I going to put everything?... Well, on another Linux machine I have on the network of course, a big fat server with a slow CPU but oodles of storage.

Sunday, 16 October 2016

Software Engineering : RegEx in C++ 11/14 with STL

I want to show you how the STL regular expressions in C++ work.... Note, complete source code & makefile at the bottom.

First we'll need a make file, I'm using Ubuntu Linux with GNU G++ v5.4.0, opening a terminal I get a text editor up and we create this makefile:

CC=g++
STD=c++14
WARNINGS=-Wall -Wfatal-errors
FLAGS=-pedantic
OUTPUT=application
FILES=main.cpp

all:
$(CC) -std=$(STD) $(WARNINGS) $(FLAGS) $(FILES) -o $(OUTPUT)
clean:
rm $(OUTPUT)
And I save that as "makefile"... Next we need a simple "main.cpp" file to test this with, so go a head and write:

#include <iostream>
#include <string>

int main ()
{
std::cout << "Hello World" << std::endl;
}

Save that and we can be in the folder and simply type "make".  Everything should complete cleanly, and you can then type "./application" to run the resulting "Hello World" application.

Now lets go back into the main.cpp, and we'll write a function... I'm not going to show you this as proper C++, I am not going to teach you about classes, so just go with me, we'll write this function above the "int main()" we just created, and then we'll define a function to split a string into words whenever it finds a space....

#include <regex>

std::vector<std::string> SplitString (const std::string& p_Source)
{
std::vector<std::string> l_result;
// The actual regular expression
std::regex l_regularExpression ("(\\S+)");
// Process the whole source string through the filter
auto l_regularExpressionResult = std::sregex_iterator(
p_Source.begin(),
p_Source.end(),
l_regularExpression);
// Use the result iterator to get all the individual strings
// into the result vector of strings
for (auto i = l_regularExpressionResult;
i != std::sregex_iterator();
++i)
{
auto l_item = (*i);
std::string l_TheString = l_item.str();
l_result.push_back(l_TheString);
}
// Return the result
return l_result;
}

Lets just take a look at this working, into your main and do this:

int main ()
{
const std::string l_SourceString ("Mary Had a Little Lamb");
std::vector<std::string> l_words = SplitString(l_SourceString);
for (auto i = l_words.cbegin();
i != l_words.cend();
++i)
{
std::cout << (*i) << std::endl;
}
}

We can save, exit and build the program again, running it we see this:

Mary
Had
a
Little
Lamb

So what did our new "SplitString" function do?  Well, lets first of all hope you're comfortable, with STL iterators, because we use one to go through the source string and then another to go through the expression result.

Our important lines of code are, std::regex l_regularExpression ("(\\S+)");  where we define the regular expression string, no I'm not going to teach you all the ins and outs of creating those strings, this expression however just gets individual strings.

The next important line is: auto l_regularExpressionResult = std::sregex_iterator(  where we are going to use the sregex_iterator constructor to actually apply the filter we created on the previous line, and we apply it to the span of the whole source string "begin()" to "end()" on the std::string::iterator there.

We could try to use the std::string::const_iterator too, by simply substituting with "cbegin()" and "cend()".

The final parameter is passing the actual filtering regular expression into place.

The result, and we don't need to worry about the type as we're leveraging auto there, is a copy of the iterator.  Depending on the STL implementation you have will define when the processing takes place, some versions will process as you iterate over the sregex_iterator, making you process the input on the fly, whilst others pre-process everything, holding off your code moving to the next line of code (when you step through) until the complete source has been processed through the regular expression.  This can be a performance trap for some, as they either think it will process, when it does not, or it does not until you iterate, and confusion ensues.  Especially when you are writing cross platform code and the platforms express different behaviours.

The last important piece of code is actually going through the result to see if there is anything in the resulting iterator.

The awkward piece of using auto shows up here, because on some platforms when you try to iterate through the result and get each string you might want to do "(*i).str()" rather than assigning the dereference (*i) to an auto first.  However, some compilers (especially when using -pedantic, GCC on this one) don't like this, so to make the code more maintainable and pre-empt it being on any platform where the dereference of the iterator is reported to "not contain a definition for "str()", I simply assign the dereference to an auto called "l_item" and then use "l_item.str()"... That's a lesson in maintainable code right there folks.

That is a very basic introduction to regular expressions, you can see why I have gone through this below.

Right now through, lets use a more complex regualr expresion, and avoid the complexity of the interator stuff, lets just validate a string as a UK Postcode:

const bool ValidatePostcode (const std::string& p_UKPostcode)
{
std::regex l_Validate ("^([A-PR-UWYZ0-9][A-HK-Y0-9][AEHMNPRTVXY0-9]?[ABEHMNPRVWXY0-9]? {1,2}[0-9][ABD-HJLN-UW-Z]{2}|GIR 0AA)$");

return std::regex_match(p_UKPostcode.c_str(), l_Validate);
}

This might look a little cramped, but I never wanted to make a mistake with the reg-ex.  This isn't a perfect solution btw, I'm still writing a test routine to check it against a full list of UK postcodes online, I think it will let some stranger codes through as valid, but they are edge cases, this will work for 99.5% of addresses, and 100% of those I've tested so far.

There you go, good luck!


=== WHY DOES THIS EXISTS ===
Today I've been using regular expressions in C++, some might consider this dark magic, however, I assure you it is all above board, the problem was the validation of a UK postcode, there was a quite terrible function:

bool Validate(char *Code);

Defined, which had all manner of hackery and trouble within, not least it could not handle some London Postcodes, we'll come back to postcodes later, however I replaced all the functionality of this code with two lines of code... Literally two, it went from around 500 lines of un-maintainable junk, to the two lines of active code to manage which you see above, I in fact could have placed the regular expression string into our master list of "strings" to yet further minimise where constants are defined, but I left that to him, left him a small victory to coerce acceptance of my drastically demonstrating his not thinking about the code changes needed, and spending all week on something which took me two lines and about 10 minutes to make sure the regex was right!

Handing it back to the owner, after my peer review, I think they wanted to cry, instead they rushed off to our common Director, avoiding all code managerial level input from fellow programmers, and said I had "shown them up by using a third party library".

I had used STL, something we use elsewhere, I had also followed the coding standards which exist, so the function had become:

const bool ValidatePostcode(const std::string& p_UKPostcode) const;

This, I think you must agree, is more informative as to what it does, it tells us we can't edit the values, we're still passing everything by reference but we're not changing the type of our system string handing from "std::string" to "char*" and we also define that the function changes nothing in the class it is within with a trailing const.

All these rules are in the coding standard, folks before you go around a peer to complain; a more senior peer at that; please check you are in fact on the right track.

So, having validated my changing the function prototype, I had to explain why I had used a third party library (as all such libraries need formal evaluation)... "Regular Expressions are in the standard library".... Was my simply reply... "Only in the latest technical release!"... Was the mouth frothing reply from the hurt chap.  "No, they've in C++11, we use STL all over the code, it is formally evaluated and signed off by everyone, including yourself".

The guy looked extremely crest fallen, and whatever his motivations for having a go at myself, I realised he just didn't know, he'd not read the books I had, he's not used the code as I have, and he'd simply always used regular expressions from third party sources, and that's fine, but please folks just check  your coding standard and have at least a look on google, before you go shouting to those above in an unprofessional manner.


---- THE COMPLETE SOURCE (main.cpp) ----

#include <iostream>
#include <string>
#include <regex>

std::vector<std::string> SplitString (const std::string& p_Source)
{
std::vector<std::string> l_result;
// The actual regular expression
std::regex l_regularExpression ("(\\S+)");
// Process the whole source string through the filter
auto l_regularExpressionResult = std::sregex_iterator(
p_Source.begin(),
p_Source.end(),
l_regularExpression);
// Use the result iterator to get all the individual strings
// into the result vector of strings
for (auto i = l_regularExpressionResult;
i != std::sregex_iterator();
++i)
{
auto l_item = (*i);
std::string l_TheString = l_item.str();
l_result.push_back(l_TheString);
}
// Return the result
return l_result;
}

const bool ValidatePostcode (const std::string& p_UKPostcode)
{
std::regex l_Validate ("^([A-PR-UWYZ0-9][A-HK-Y0-9][AEHMNPRTVXY0-9]?[ABEHMNPRVWXY0-9]? {1,2}[0-9][ABD-HJLN-UW-Z]{2}|GIR 0AA)$");

return std::regex_match(p_UKPostcode.c_str(), l_Validate);
}

int main ()
{
const std::string l_SourceString ("Mary Had a Little Lamb");
std::vector<std::string> l_words = SplitString(l_SourceString);
for (auto i = l_words.cbegin();
i != l_words.cend();
++i)
{
std::cout << (*i) << std::endl;
}

// Postcodes
std::cout << "--- Postcodes ---" << std::endl;
std::cout << ValidatePostcode("NG16 5BP") << std::endl;
std::cout << ValidatePostcode("NG10 1NQ") << std::endl;
std::cout << ValidatePostcode("Robert") << std::endl;
std::cout << ValidatePostcode("FP52 JTY") << std::endl;
}

---- makefile ----

CC=g++
STD=c++14
WARNINGS=-Wall -Wfatal-errors
FLAGS=-pedantic
OUTPUT=application
FILES=main.cpp

all:
$(CC) -std=$(STD) $(WARNINGS) $(FLAGS) $(FILES) -o $(OUTPUT)
clean:
rm $(OUTPUT)


P.S. Yes this will all work with "STD=c++11" in the make file!