Felipe Alvarez's blog

I got hired at my old job. I’m very happy for that. They’re happy I came back! Saves them money and time on training, and the off-chance of hiring a ‘lemon’ that quits after 4 weeks. I’ve entered into a Digital Forensics project. I need to develop a tool that will automate some of the labourious tasks involved in digital forensic evidence acquisition. I’ll probably need to brush up on GNU Bash scripting, and PHP.

It’s been very warm here, but not too hot, which I like. Yesterday was a bitch, though, felt like 30°C. Fortunately, today was overcast, and so helped to alleviate some of the scorching.

Health

1. Drink plenty of water.

2. Eat breakfast like a king, lunch like a prince and dinner like a  beggar.

3. Eat more foods that grow on trees and plants and eat less food  that is manufactured in plants..

4. Live with the 3 E’s — Energy, Enthusiasm and Empathy

5. Make time to pray.

6. Play more games Read more…

Flight was good. Good food, Good service. I’m very happy with korean Airlines, and I’d travel with them again, just maybe not from Toronto to Brisbane (Brisbane to Toronto, however, was flawless, arguably the best flight I’ve ever taken in my life).

These are pictures of the free Hotel room Korean Air set me up with. I had to wait 16 hours for my next flight. They provided me with a free lunch coupon, but I got hungry by 7:00 AM and just looked for a decent local joint to satisfy my kimchee and bibimbap craving.

Excuse the poor quality pics – they were taken with my laptop’s webcam.

Previously we have an example on regular expression, but It doesn’t shows the power of square brackets ( [ ] )

Let say you want to search for string fprintf, vprintf and sprintf using grep, usually what you do is

egrep "fprintf|vprintf|sprintf" *.c

You may be ask why don’t just uses the word “printf”? If uses the word printf, it will return all of them but also include printf itself. But in this case i don’t want to grep other printf besides f,v,s printf. Thats the square brackets comes in to lessen your trouble.

egrep "[sfv]printf" *.c

It simply return the result with any character specified in [ ] with word printf concatenated.

The square brackets can be used with other RE symbols, here is another example, let say I want to gets all lists with words start with a character “a to f”, I can do this

egrep "^[a-f]" com-book.txt

It is case sensitive, I want all a to f including the upper case A to F.

egrep "^[a-fA-F]" com-book.txt

Copyright: ByExample.com

UPDATE: I changed ‘>’ (erase file, then write to file) to ‘>>’ (append to file). This avoids you overwriting your, or other peoples’, public keys.

Setting up password-less logins is both dangerous, and mighty. It allows one to authenticate to an OpenSSH server without typing in a password. Authentication is gained via knowledge of a private key.

Generate a Public/Private Key Pair

$> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/felipe/.ssh/id_rsa):
Enter passphrase (empty for no passphrase): <ENTER>
Enter same passphrase again: <ENTER>
Your identification has been saved in /home/felipe/.ssh/id_rsa.
Your public key has been saved in /home/felipe/.ssh/id_rsa.pub.
The key fingerprint is:
d7:79:c3:01:ce:90:71:a2:a2:3d:83:26:fb:9a:1f:5b felipe@linux.local

You will then find two files inside your directory. Keep them safe, secure, and secret. The public key (the one with .pub at the end) can be widely disemmindated. It represents the antonym of secrecy and privacy. The private key, however, must remain private and secret at all times.

Copy the PUBLIC key to a remote OpenSSH server

You must copy your public key to a remote host. The host will verify that you own the private key by encrypting a “challenge” and forcing your ssh client to decrypt it. If successful, you are authenticated, and admitted entrance. A password isn’t required.

$> cat /home/felipe/.ssh/id_rsa.pub | ssh felipe@remote-host.com \
"cat - >> .ssh/authorized_keys"
felipe@remote-host.com's password: <PASSWORD>

This copies your public key the authorized_keys file (NB: authorized_keys2 is deprecated and no longer recommended for use. OpenSSH checks both).

Testing Phase

‘logout’ or ‘exit’ and try:

$> ssh felipe@remote-host.com

It should not ask you for a password. You should automatically be logged into the remote system.

Works with scp and rsync too!

‘scp’ and ‘rsync’ both use a ssh client at the backend, and so will also authenticate automatically utilising your public and private key pair. Try:

$> scp file_a felipe@remote-host.com:file_b

This should transfer without pausing to ask for your password. Likewise try:

$> rsync -r /backups/2010/Jan felipe@remote-host.com:/backups/2010

This should backup your entire directory to remote-host.com without pausing to ask for a password. You can put a line similar to this one in a shell script, and run it with cron once a week or so. It will automatically backup your system, using OpenSSH, and proven secure and safe method for authentication of human and machines across an untrusted public network, away from curious eyes.

For this setup, I used 4 320GB sata 300 hard drives. This array is not configured for booting, just for redundant storage. My four drives are:

• sdb
• sdc
• sdd
• sde

1.  Ensure all partitions on the drives are erased

There are a few ways to do this. I just overwrite the first million bytes with zeroes.

$> dd if=/dev/urandom of=/dev/sdb bs=1M count=1

Another way to do it, is using fdisk, like this

$> fdisk /dev/sdc
d (deletes a partition by its number)
1 (partition number)
w (writes changes to the disk)
q (quit without saving)

For best results, one should remove all partitions from all the RAID members.

2.  After all partitions have been erased from all members, we must create RAID partitions.

We can use fdisk again, like this:

$> fdisk /dev/sdd
n (this makes new partition)
p (primary (not extended))
1 (number 1)
start: <press enter>
end: <press enter>
t (selects partition type)
fd (0xFD is the symbol for Linux RAID partition)
w (writes changes to disk)
q (quits without saving changes)

Repeat this process for all your RAID members. Do not format these disks. We will first build the array, then format the array.

3.  Tell mdadm to create an array with 4 members

$> mdadm --create --level=5 --metadata=1.2 --raid-devices=4 \
 /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

• creates the array from available members
• select the raid level (we want RAID5, but 0, 1, 10, 5, 6 are available)
• metadata ensures we have the most robust and up-to-date RAID system
• raid-devices select 4 devices for our array. We could have done 3 devices, and one spare. A spare will automatically rebuild if any live members fail or die.
• /dev/md0 is the array
• sdb, sdc, sdd, sde are the partitions that will be a part of this array

Now that you’ve created it,  you don’t need to assemble it. In case you need to, however, this is how you can do it.

$> mdadm --assemble <ARRAY> <DEVICES> ...

$> mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

4.  mdadm is now creating and initialising the drives.

You can check progress with

$> cat /proc/mdstat

It is a good practice to check your array every now and again. See a detailed report with:

$> mdadm -vD /dev/md0

Which does a –verbose –detail ‘ed check of your array /dev/md0.

5.  Partition and Format the Array

You can partition the array with your favourite program. Don’t partition the drives! Partition the array /dev/md0! I use fdisk:

$> fdisk /dev/md0
n (new partition)
1 (number)
start: <press enter>
end: <press enter>
w (write changes to disk)
q (quit without saving changes)

Next you format the partition with your favourite filesystem. I like ext3. My distribution ships with a shortcut program called mkfs.ext3. You may require mke2fs, which by default create an ext2 filesystem. Add option -j to create a journaling ext3 filesystem. Type man mke2fs for more information.

$> mkfs.ext3 -v -L ADD-A-LABEL /dev/md0p1

Where ‘p1′ is the first partition on the array. The array may still show ‘rebuilding’ bur it is usable. It will not be fully redundant, however, until rebuilding status shows 100%.

6.  Create or Edit /etc/mdadm/mdadm.conf and /etc/fstab

It should read something like this:

#/etc/mdadm/mdadm.conf
DEVICE /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
CREATE owner=root group=disk mode=0660 auto=yes
MAILADDR root
#MAILADDR xxxxx@domain.com
ARRAY /dev/md0 metadata=1.2 num-devices=4 devices=/dev/sdb1,/dev/sdc1,/dev/sdd1,/dev/sde1

Your /etc/fstab should include a line similar to:

#/etc/fstab
# automount /dev/md0 raid partition
/dev/md0p1  /mnt/mountpoint  ext3  rw,user  0 0

My /etc/fstab reads:

/dev/md0p1 /media/leopard ext3 rw,user,noacl,noatime,nodiratime,noauto 0 2

noacl,nodiratime,noatime should improve performance of ext3. noauto prevents filesystem from being mounted automatically, just in case (I’m paranoid). The ’2′ at the end makes fsck scan the drive after 31 or so mounts. See man mount for more options.

Update: There is another terrific guide explaining how to modify and grow a RAID1 array with 2 (or more) disks to a RAID5 array.

I will never again buy a RAID Controller card from Promise again! They claimed to support GNU/Linux, and they don’t. They said they didn’t have drivers for Windows 7, and then suddenly they magically appear on their Downloads page. We weren’t even notified.

I’m going to go with mdadm and try my luck with software RAID on Mint. If I must buy another card, for whatever reason (namely software RAID is much too slow, and hardware RAID will offload the work to the card itself) I’ll go with Adaptec.

I just ripped some of my CD’s to MP3, but I was just curious what OGG would do for me. I had never actually compared the two encoding formats, side-by-side, but today, I was simply stunned.

A song compressed with MP3 (VBR 128Kbps Normal Quality) was around 5.1 – 5.8 MB. It sounded good, but ‘clearly’ inferior to the actual CD Quality sound.

The OGG rip (VBR 128Kbps), on the other hand knocked my socks off! It was around 3.0 – 3.1 MB and sounded ‘nearly’ as good as the original CD!

I hesitated, at first, to rip them all to MP3, in case I wanted to share them (!gasp!) with others. However, now that I can see a 17% – 20% compression gain using OGG over MP3, I no longer feel that way. I wholeheartedly endorse the use of OGG Vorbis for ALL compressed lossy compression.

Most [good] audio/multimedia players already support OGG (except, MS programs, obviously!) so you should have no problem listening to them.

If you have a portable media player (PMP) without native OGG support there are two options

1. Contact the manuafaturer and demand (request?) that they support OGG in future versions of their players
2. Ask them to create a firmware update to include OGG support on currently supported players
3. Install Rockbox: a Linux-based GNU open-source free software suite which allows many major PMP’s to play a huge variety of free and proprietary (i.e. non-free, patented, or otherwise ‘encumbered’) formats, such as OGG. It also allows you to play wide variety of video formats, as well. It included a bunch of interesting features such as backlight dimming, battery-saving features, audio enhancement features, and plenty of games (plays DOOM too!)

I’m flying back to Brissy soon, it feels good. I miss so many people back ‘home’. Some I’ve spoken with many times since I came back to Canada, some only just a few. I appreciate the friends I have, and I’m thankful for them.

I have yet to meet many of my Canadian friends. Many Hamilton friends I have not yet seen, and long to see. I shall see them soon, with luck.

I have been enjoying this time off very much. I have done nothing but slack off, eat like crap, sleep in ’till noon everyday, go out with friends, watch movies, go to bars, go to cottage. No snow shoveling, no work, no classes, no trips to grocery store. Just bliss.

I have enjoyed my time here, but I miss Brissy. It will be good to be back — warm too!