“…I am running Ubuntu Linux.  I want to wipe my hard drive.  How do i do that?…”

I’m assuming that you don’t have another computer you can plug that hard drive in? Additionally, you cannot wipe a hard drive if you have no OS to run the software. And finally, I assume by wiping, that you are referring to writing 0′s and 1′s via several passes over the hard drive.

I have come across an Open Source solution that does this, it’s called Darik’s Boot And Nuke.

In general, you download the image from here: http://www.dban.org/download, burn it to a CD and boot off that CD. Follow the prompts and select your appropriate configuration options.

Note: If there is data on the hard drive that you wish to keep, make yourself a seperate backup on some other media BEFORE you wipe the drive.

But, what if there were no backups? What if your hosting provider cannot restore data at their end? To be blunt, you’d be back to square one! Developing a whole new site or blog from the beginning! That’s a chilling thought, to lose everything and start again.

For peace of mind and data (intellectual property) , today’s post will highlight some of the steps we’ve taken to fully automate the backup process. Hopefully this will help many of you who may encounter the same issues, or are simply looking for a proactive, automated backup system for your web sites, blogs, ecommerce sites, etc.

We’ll need 5 things to ensure this system works:

• The remote host (your web hosting server).
• The local host (your Ubuntu or other Linux based desktop).
• The open source Rsync package.
• OpenSSH.
• Cron.

Let’s start with our desktop, which is the ‘localhost’. In my case the desktop is Ubuntu Linux 7.10, but this can be any Linux based system. This could also be another Linux server, if you tweak this a bit more. ;)

I know ‘cron’ is enabled (because it’s part of the default installation) of my Linux desktop. I also know SSH is installed (because it’s installed by default and I’ve used it), but I’m not sure if ‘rsych’ is there and if it works over SSH.

Side note: For those not familiar with Rsync, “rsync is an open source utility that provides fast incremental file transfer. rsync is freely available under the GNU General Public License and is currently being maintained by Wayne Davison.” Source: http://samba.anu.edu.au/rsync/

To see if rsych is installed, use the following terminal command:

apt-cache search rsync

If you see it’s installed, to determine if rsync works over SSH, open a terminal and type the following command (substituting your correct information):

rsync -avz -e ssh Your Remote Username@Your Remote Server Host:/The Remote/dir /Your Local/dir/

Here is what the switches mean:

a: Use ‘archive’ mode.
v: Use ‘verbose’ output.
z: Use ‘compression’ during file transfer.
e: Specify the ‘command’ to run. In this case SSH.

In my case the command could look something like this:

rsync -avz -e ssh backupadmin@ubuntulinuxhelp.com :/backupdir/daily /home/ubplay/sitebackups

After entering the above command, I’m prompted to enter the password and the file transfer begins.

In my case this is simple because the hosting provider uses ‘The’ industry standard software (Linux) as the standard applications, openssh, rsych, cron, etc. And my local Linux system already had the tools installed. Now that I’ve determined it works, cron can automate the system. However, before moving to cron, make sure your server is configured to backup the files and databases on a daily (or other) schedule.

If you’re using industry standard hosting services, you’ll be on a Linux box using cPanel. Personally, I’ve tried several others including Plesk, ISPConfig, etc, however in my opinion, they don’t have the amount of flexibility or options that cPanel does. In terms of a LAN however, in my opinion nothing beats Webmin. Webmin has the greatest flexibility and options. However, I’m going off topic here, back to the subject at hand!… Log into your hosting control panel and use the interface to configure your scheduled backups to occur during low-traffic periods. Make a note of the directory the backups are saved to. WHM/cPanel is great for this as it’s configured via a simple GUI, and is easy to use. :) In my case the server backs up the web site files and databases and stores them in /backupdir (so that my cron job can download any files in this directory later). For privacy issues, I’m not going to post the script as it contains a username and password among other “exposures”.

Before moving to cron itself, I needed to configure a script that will rsync over the SSH connection. Here are some example I found on the rsync site: Rsync Examples. Another great resource we found is here: resync-incr. On this site you’ll see another methodology and example scripts. And finally another great backup scripting resource here: Backup Script. I’m sure some of you have other great sites and resources listed, please comment below and add them. :)

After you’ve set up your script, however you want it (there are hundreds of ways!), use cron to run it. Setting up the cron job is not very difficult:

0 2 * * * /home/ubplay/cron/rsync-ubuntulinuxhelp

This (above) downloads the backup at 2am every day. Remember to ensure that your server has finished creating its backup by this time. Otherwise you’ll not be downloading the files you expect. In my case I use nano to create the file called “rsync-ubuntulinuxhelp” placed in the …/cron director. The file named rsync-ubuntulinuxhelp contains the actual bash script. To create the cron job itself (that calls the script), complete the following in a terminal:

sudo crontab -e

and use the following parameters:

* * * * * path to script/command to be executed and script/command
- – – – -
| | | | |
| | | | — Day of week (0 – 7)
| | | ——- Month (1 – 12)
| | ——— Day of month (1 – 31)
| ———– Hour (0 – 23)
————- Minute (0 – 59)

(‘*’ means ‘every’).

Side note: to view your existing cron jobs, in a terminal, type:

sudo cron -l

to delete a cron job:

sudo cron -r

As usual, I hope this helps some of you!

Enjoy :)

[tags]linux, ubuntu, automatic, backup, website, cron, rsync, how to, openssh, save website[/tags]

It left me wondering as to the status of some of the software I have used in the past. Now that I’ve garnered much more experience, I was intrigued as to how (if possible) I could provide this business limited services. If not, could I at least access the data on the failed drive?

Here’s the scenario. I have an older 10GB hard drive that had failed and I’ve stored away (for the last 2 weeks). I remember there was almost no data on it. I wondered if I was able to get the data off that drive, it had Ubuntu Linux 7.10 on it (ext3). I already had one backup of the data prior to the failure so simply stored the drive on the shelf for no particular reason. If I was unable to get the data off the drive, I wouldn’t have cared because I already had a backup. I just want to see if I could do it. And, if I could, share this information with others – to help them.

Back to the present. I plugged the drive back into a working system and sure enough, I cannot access it. It powers up and spins, but the GUI cannot access it. I read some years ago that one tactic is to place the drive in a freezer for a couple of hours or so before trying to reclaim data. I kept the PDF I read back then, and it was called “200 Ways to Revive a Hard Drive“.

While freezing the drive might enable access, I still might need some applications to actually grab a copy of the data. I though, I’d only have one shot at booting up and accessing the drive, so assumed the best course of action was to create an image of the drive and work with that. Creating images in Ubuntu Linux is not a difficult task to do and can easily be performed on the command line ( in terminal), using the following commands:

sudo mkdir /mnt/image
sudo mount -t ext3 /dev/hdb1 /mnt/image
sudo dd if=/dev/hda of=/mnt/image/10gdrive.raw

So I was set to go. I had the following ready:

The existing PC with another working 10GB hard drive in it (which I formated to ext3, the same as the failed drive), to be booted up with an Ubuntu Linux LiveCD. This is important because I wanted to create the image of the failed drive and needed to put it on the working drive. If the working drive were in use, then that would not be possible, therefore booting off the LiveCD would provide this functionality.

I plugged the failed (frozen) hard drive and turned on the system. After the LiveCD boot completed, I was able to use the above commands to initiate the image creation (of the failed drive). Will it run? If it does, will it remain operating for the period of the image creation?

Well, it did last the surprising hour and a half it took for the image to be made. So I suppose, the hard drive failure must have been just something simple that prevented reading of data. I now had the image to work with and see if I could get any data.

I powered down the system and removed both hard drives. The “bad” hard drive can be used as a paper weight :) and the good “image” drive I set aside for the next step.

I plugged a new hard drive into the system, actually it was the original one that already had Linux on it (Ubuntu Feisty), and plugged the imaged drive into the second ATA connector and powered up the system.

However, let’s stand back for one moment so that I can explain the next steps (at least what worked for me). A couple years ago I was introduced to Sleuthkit which is a series of command line scripts used for data forensics. Also Autopsy, which provides a GUI interface and aff (which needs afflib). Why aff? I’m not overly trained in the technical aspects, but I do know that the use of other compression formats requires Autopsy to have the file decompressed first, but using the afflib was the best (at least that’s what was in my notes at that time. I don’t remember where I originally read this), as it allows Autopsy to use the compressed file. Also, I want to see if I can burn a 10GB hard drive to a DVD because:

“…AFF implements the LZMA compression system, which can produce disk image files that are dramatically smaller than other tools on the market today. Compared with the Expert Witness format used by EnCase and other tools, AFF images are typically 30% to 50% smaller…”

To clarify, Install Sleuthkit, Autopsy and aff compression, so you can work with the image. The disk image is currently in raw format so aff format gives us something we can work with and burn to DVD. The command to install the tools is:

sudo apt-get install autopsy afflib

Note: I didn’t have to install Sleuthkit because it gets installed automatically when you install Autopsy.

Now that we have the required tools, lets convert the image to something burnable to DVD:

sudo afconvert -X9 -o /root/10gdrive.aff /media/dvdrecorder/10gdrive.raw

Here are what the switches in the above command mean:

-X9 Sets the compression to 9 (the maximum).
-o Output to a file.

Note: “afconvert –help” will give you the switch option explanations.

What this command actually does is convert the image back to a compressed .raw and send it to the DVD. The data did fit, but only because there were hardly any files on the original bad disk, so the empty space was compressed to virtually nothing. (At least that’s how I interpreted it). If I messed something up, I had the backed up image on DVD.

The rest was a simpler matter of using Autopsy and following the instructions.

The PC now has the following:

A working hard drive, using Linux (Ubuntu).
The “imaged drive” plugged into the PC as well.

To start up Autopsy open a terminal and type: “autopsy” and you will see a screen that shows you this:

Just follow the instructions and you’ll see:

I ignored the Javascript warning because security was not an issue. Remember, I just wanted to see if I could do this and share what I learned.

Moving on…

Oh… I need to add a “host”…

Simply follow along and play with the tools. You’ll be surprised what you can see! ;)

Needless to say this was a good exercise for me. The real motivation for me to do this (and document my “journey” in this blog), is to see how much I can learn. That’s also what Opensource is about right? The more I play with Linux, the more I learn and the greater the confidence level.

And… just a side note, for those readers not aware. A very good (and free) trouble shooting tool I also use is UBCD (Ultimate Boot CD). It’s a troubleshooting, fixer upper, data scanning, high speed formating, disk testing… the tools go on and on. Very cool, try that one out too, as you won’t be disappointed.

On a lighter “fun” note, we can still put those dead drives to good use! ;)

Hard Drive Speaker System.Build a 15,000 rpm Tesla Turbine using hard drive platters.
Pulling apart a desktop hard drive to get rare earth magnets.
Hard drive platter clock.
Hard Drive Fridge Magnet.

Hopefully some of you will find the above information of help. :)