Using Flash Drives in ZFS Mirror

This post comes from an idea I had to allow me to easily carry a ZFS mirror away from a site and back again, we didn't need much space - only 5gb - but it had to be mirrored in triplicate, one copy to stay locally, one going into a fire safe on site and the third to be carried by the IT manager off-site each evening.

The trouble?  A near zero budget, so for a little over £45 we have a 14GB ZFS mirrored pool, across three 16 GB USB Flash drives and one three port USB 3.0 hub.

It was perfect for the task at hand, extremely portable, and cheap, I thought the same approach may help anyone trying to get to learn a little more about ZFS, a student or even someone using a laptop as a small office server - as the laptop literally has its own battery back-up system built in!

It's not the fastest solution, its in fact extremely slow, but as an entry step it's perfect.

See the full video below, throughout the commands I list were in use...

Commands:

Listing Disks by ID...

ls /dev/disk/by-id

Listing Disks to a file for use in a file script as you see me using...

ls /dev/disk/by-id -1 > disks.txt

------------------

To install ZFS on Debian/Ubuntu linux:

sudo apt-get install zfsutils-linux

------------------

To remove & purge ZFS from your system:

sudo apt-get purge zfsutils-linux

(and you will be left with "/etc/zfs/pool.cache" to remove or back up yourself).

------------------

Command to create the pool...

sudo zpool create <Name> mirror <DiskId1> <DiskId2> etc...

The name we had here was "tank", if you already have data on these disks you will need to add "-f" to force this change through.

------------------

Command to make a file executable - like our sh script:

sudo chmod +x <filename>

------------------

Zpool Commands:

sudo zpool status

sudo zpool import <name>

sudo zpool scrub <name>

sudo zpool clear <name>

You will want to "import" if you completely remove ZFS or move one of your sticks to a new machine etc, simply insert the disk and import the pool by name.

Scrub will be used whenever you return a disk to the pool, remember the point here is to allow you to replicate the data across the three sticks and be able to remove one or two to safe keeping, be that an overnight fire safe, or taking a physical copy with oneself.

Clear is used to remove any errors such as the Pool becoming locked out for writing - which it may if a drive, or all drives are removed - you simply clear the current problem with any pool.

Summary:  Remember this is NOT the optimum way to run ZFS, this is actually extremely slow, you are replicating each write over your USB, you can only cache so much in the RAM, but it is not a performance piece, this is about ensuring one replicates data for safe keeping, a small office or your dorm room server setup could be completely provided by a laptop in this manner, it has it's own battery backup, it is quite (if you get the right machine) and really this is a very cheap way to play with ZFS before you move onto other bigger hardware options.  Plus, I find the best way to learn about technology is to break it, even a little, and so constantly breaking down your pools by pulling USB sticks out of them is an excellent opener to recovering your pools.  Play about first, don't put anything critical on there until you're really happy with the results.

For an excellent post covering creating ZFS pools, cheak out programaster's post here: http://blog.programster.org/zfs-create-disk-pools

And for the official ZFS documentation you can check things out with oracle here: https://docs.oracle.com/cd/E26505_01/html/E37384/toc.html

Oh, and Happy New Year... I guess I made it to 100 posts this year...

Server Admin : How Good is your Backup?

How robust is your back up solution?  Go on, be honest with yourself, how good is it?... Because I've seen a whole host of them and, at this very moment, this is the screen up on one of my servers....

Yes, my raid 5, just a test raid 5 with three really bad recycled SAS drives in it has failed; this doesn't surprise me, but it does delay me because I now have to rebuild the data... However, I know my data is good.... Lets see how good my back up is.

This back up is coming from a DD created raw image of the virtual disk, stored to and soon lifted from my NFS accessible ZFS mirrored back up server.

Therefore you would be right to ask, why are you rebuilding the virtual RAID disk in the above screen shot?  Well, I'm going to test my back up strategy!

I popped the known bad disk and the good disks out, replaced all three and I'm able to test a restore to a new virtual disk set, I have a USB boot drive ready, this is a test.

This kind of test, a real live restore, is sorely missing from so many enterprise set ups, so ask yourself is your back up going to work?

Windoze Security Loop Hole

This is an example of why I hate Windows...

In a curious case of a security loop hole, in the office, we have a supposedly locked down security situation, none of us are local administrators on our machines, and neither do we have access to any of the very useful parts of our machines.

This is a real pain, and one whereby we often have to call up on the IT Administrators to come and physically, or remotely in a remote desktop session, enter their password for us.

I personally disagree that educated users such as myself have to put up with this situation, I agree totally with data privacy and integrity, however, I wholly disagree with locking people out of things on their machines, such as defragging, or emptying the temporary folders... Or in the case of a programmer, not being able to empty Prefetch or write an ISO to an SD Card.

Anyway, today, I had to write an ISO to an sdcard, the result... I called IT and asked them to run the program for me....

So, just to be clear, I'm logged in as myself:

I am unable to access parts of the system, like the Administrators desktop folders...

I get IT to log the ISO image writer as their elevated user, and the loop-hole begins, you see the program has a standard windows open dialog.  And this will work with any standard windows open or save-as dialog, in any program... The program is running as Administrator at this point.

When I select to browse to the file to open, the default folder is the administrators folder by name...

However, because these dialogs all use explorer under the hood, and it's all integrated, they do far more than select a file for you, they let you create folders, browse things and even launch programs...

Yes you can launch a program from a save-sa, or open-sa, browsing dialog!

Lets try to run a command prompt...

Oh, look, it's running as Administrator...

And now I can see the Administrator account directories, which were hidden from be in my own logged on Explorer window..

And I can clear the prefetch folder in windows...

This is clearly wrong, but it's all caused by windows, so what's going on?...

Well, instead of asking the current session (logged in as regular old me) to start the new application instance of Command Prompt, it's asking the application owning the browsing dialog, so command prompt is started under and inherits the user credential level of that program, not my whole session.

What should have happened, well, I believe windows, starting a new program from an elevated user like this should have re-prompted for the user's password again.  And indeed, trying to start certain files from the launched command prompt it does go back to the session level to ask for the credentials to start the application with.  But not asking and just starting the new application is a problem.

Solutions I can think of include, setting the administrator level account to timeout its password every minute, so one reduces the amount of time a regular user has an unaccredited ability to launch programs.  And within a minute the administrator could have started anything the user wanted and left.

A better solution however, might have been to have a user elevation level which could give access only to what the regular user wanted, permissions to use peripherals perhaps, rather than start applications.  And the Administrator should not have just started the application as themselves, but should have started the application under themselves as the Hardware only user.

There are other solutions, and I'm sure many I'm not even going to think about, because I don't use Windows systems.  If I want security, I simply use Linux and set things up correctly.