Ubuntu RAIDbox: Part 3

In general, this part of the instructions assumes that you are using storage devices (disks and CF) that contain nothing that you care about. If this is not the case, please either first copy the data you want to keep to a safe location, or learn how to adapt these steps to preserve what you have.

Boot your RAIDbox hardware and launch the Ubuntu installation process.

  1. Choose Manual Partitioning
  2. For each disk displayed, delete every partition.
  3. Once complete, partition your CF card as a single ext3 partition with / as its mount point.
    1. Move the cursor to the single row underneath its drive descriptor, e.g. :
      IDE1 master (hda) - 8.0 GB TRANSCEND
              pri/log    8.0 GB     FREE SPACE
    2. Press ENTER
    3. Select Create a new partition and press ENTER
    4. Press ENTER to accept the default size, which is all of your disk space.
    5. Select Primary and press ENTER
    6. If Use as: is not Ext3 journaling file system, then
      Select Use as: and press ENTER
      Select Ext3 journaling file system and press ENTER
    7. If Mount point: is not /, then
      Select Mount point
      Select / – the root file system
    8. If Bootable flag: is not on, then
      Select Bootable flag: and press ENTER.
    9. Press ESC to return to the partition menu.
  4. For each hard disk, first create a 384 MB RAID partition at the end of the disk:

Move the cursor to the single row underneath its drive descriptor, e.g. :

SCSI4 (0,0,0) (sda) - 500.1 GB ATA ST3500841A
         pri/log  500.1 GB     FREE SPACE

Press ENTER

Select Create a new partition and press ENTER

Press BACKSPACE until the partition size field is empty. Then type 384 MB and press ENTER

Select Primary and press ENTER

Select END and press ENTER

Select Use as: and press ENTER

Select physical volume for RAID and press ENTER

Press ESC to return to the partition menu.

After all disks are complete, fill out the remaining space with another RAID partition:

Move the cursor to the row underneath its drive descriptor that shows FREE SPACE, e.g. :

SCSI4 (0,0,0) (sda) - 500.1 GB ATA ST3500841A
         pri/log  499.7 GB     FREE SPACE
        #1 primary  378.4 MB   K raid

Press ENTER

Select Create a new partition and press ENTER

Press ENTER to accept the default size, which is the rest of your disk space.

Select Primary and press ENTER

Select Use as: and press ENTER

Select physical volume for RAID and press ENTER

Press ESC to return to the partition menu.

Scroll back up to the top of the Partition disks page, select Configure software RAID and press ENTER.

If asked to Write the changes to the storage devices and configure RAID, select Yes and press ENTER.

To clear out results from previous attempts, select Delete MD device and press ENTER. Delete all devices until there are no more defined.

Once all MD devices are deleted, select Create MD device and press ENTER.

Select RAID5 and press ENTER.

Press BACKSPACE until the default number of active devices for the RAID5 array is deleted, then type 4 and press ENTER.

If the default number of spare devices for the RAID5 array is not 0, press BACKSPACE until the number is deleted, then type 0 and press ENTER.

One by one, move the cursor to each of the 4 lines that ends in 1 (e.g. /dev/sda1) and press SPACE to select it. When done, press ENTER.

Repeat the above steps, but this time, select all the lines that end in 2 (e.g. /dev/sda2) which should be the only 4 options remaining.

Select Finish and press ENTER.

When the partitioner is done, it’ll take you back to show you the results of your RAID work. From this point on, leave your actual disk partitions alone and only work with the RAID5 device entries. e.g.

RAID5 device #0 - 1.1 GB Software RAID device
      #1   1.1 GB

and

RAID5 device #1 - 1.5 TB Software RAID device
      #1   1.5 TB

Select the line under the smaller RAID5 device and press ENTER.

Select Use as: and press ENTER

Select swap area and press ENTER

Press ESC to return.

Select the line under the larger RAID5 device and press ENTER.

Select Use as: and press ENTER

Select Ext3 journaling file system and press ENTER

If Format the partition: is not set to yes, format it, then select it and press ENTER.

Select Mount point: and press ENTER

Select Enter manually

Change the default to /raid and press ENTER.

Press ESC to go back.

When back at the Partition disks screen, review your partitions one final time and then scroll down to select Finish partitioning and write changes to disk, then press ENTER.

The rest of the installation process is just like a normal Ubuntu installation, except with more simultaneous disk activity. When your system boots, Ubuntu will be entirely on the CF card, and your swap partition will be on one raid array, and a giant chunk of storage will be mounted at /raid.

The only thing that remains is to get /var off the CF as soon as /raid is mounted, because CF cards can only take a certain number of write cycles, and we don’t want logging to kill our boot disk.

Once your RAIDbox boots for the first time, log in as your main user account. If you run the command ‘df -h’ then you should see something like this:

Filesystem            Size  Used Avail Use% Mounted on

/dev/hda1             7.4G  660M  6.4G  10% /

varrun                188M   44K  188M   1% /var/run

varlock               188M     0  188M   0% /var/lock

udev                  188M  112K  188M   1% /dev

devshm                188M     0  188M   0% /dev/shm

lrm                   188M   34M  154M  19% /lib/modules/2.6.22-14-generic/volatile

/dev/md1              1.4T  198M  1.3T   1% /raid

As you can see, our CF disk is mostly underused, and we want to keep it that way to ensure a long life. More on that later.

Ubuntu RAIDbox: Part 2

With the hardware assembled, the next step is to install Ubuntu Server 7.10. I did not want to use an optical drive, but if you want to hook up an optical drive and install from the CD, you can just insert Ubuntu Server 7.10, configure the BIOS to boot from the optical drive, reboot, and go to Part 3.

For the adventurous, here’s how to PXE boot Ubuntu Server 7.10 on your RAIDbox (or any PXE-capable machine, for that matter) from another Linux server on your LAN.

Pick a Linux machine on your LAN to act as the PXE server for the PXE boot process on your RAIDbox. I chose a server running 64-bit Ubuntu Server 7.04, but I anticipate any recent version of Ubuntu would have the same setup process. Here’s everything you need to do on the PXE server:

Download (or copy from another machine) the .iso file for Ubuntu Server 7.10.

Install dnsmasq and prepare the tftpboot directory:

sudo apt-get update
sudo apt-get install dnsmasq
sudo mkdir /tftpboot
sudo chmod 755 /tftpboot
sudo chown nobody: /tftpboot

Run these commands to extract needed files from the Ubuntu Server 7.10 image:

sudo mount -o loop /location/of/your/ubuntu-7.10-server-i386.iso /mnt
sudo cp -a -r /mnt/install/netboot/ubuntu-installer /tftpboot
sudo cp -a  /mnt/install/netboot/pexlinux.0 /tftpboot
sudo cp -a -r  /mnt/install/netboot/pexlinux.cfg /tftpboot

Edit /etc/dnsmasq.conf. The lines you add will look something like this. You need to provide an IP address that you want your RAIDbox to have, you need to specify the IP address of the PXE server you’re configuring, and you need to provide the IP address of the gateway you use to get to the internet, which is usually your LAN’s regular DHCP server. You must also list the MAC address of the network device you’ll use on the RAIDbox when PXE booting. (This makes sure no other devices on your LAN connect to the PXE server.) If your PXE server also has a human-readable name on your LAN, you can list it here. Otherwise, leave it blank.

dhcp-range=IPADDRESS.FOR.YOUR.RAIDBOX,IPADDRESS.FOR.YOUR.RAIDBOX,12h
dhcp-host=MAC:ADDRESS:OF:YOUR:RAID:BOX,IPADDRESS.FOR.YOUR.RAIDBOX
dhcp-boot=pxelinux.0,HUMAN_READABLE_NAME_OF_YOUR_SERVER,IPADDRESS.OF.YOUR.SERVER
dhcp-option=3,IPADDRESS.OF.YOUR.GATEWAY
enable-tftp
tftp-root=/tftpboot

Here’s how it might look:

dhcp-range=192.168.0.240,192.168.0.240,12h
dhcp-host=aa:bb:cc:dd:ee:ff,192.168.0.240
dhcp-boot=pxelinux.0,presence2,192.168.0.34
dhcp-option=3,192.168.0.10
enable-tftp
tftp-root=/tftpboot

(Re-)launch dnsmasq:

sudo /etc/init.d/dnsmasq restart

Ubuntu RAIDbox: Part 1

I needed a RAIDed NAS box that I could rsync to for backing up Linux filesystems. While there are commercial offerings for around the same price, none of them provided native Linux filesystems or allowed SSH access and/or rsync, so I decided to roll my own. This is one way you can make a cost-competitive and low-power RAIDed NAS box in Ubuntu that does ssh/rsync, nfs, and samba. All prices listed are approximate, since I bought more than I needed initially (see below for explanation), but prices do include shipping.

You can probably do it cheaper if you have different requirements. I wanted to reduce the number of moving parts and keep power consumption low, so I went with a passively-cooled mini-ITX motherboard and a CF boot disk. You could easily save about $300 by using more conventional motherboards, but keep an eye out for how much it costs to run the box constantly. My design only has 6 moving parts: 4 SATA HDDs and 2 cooling fans for the HDD enclosure. When off, the power brick consumes 4W (there are likely better models available). On boot, it spikes to 133W while the HDDs power on. At idle, it consumes 67W, and when the RAID is under heavy access, it consumes 77W.

Once all your parts are in, assemble the system. The only tricks involve the power converter board and the hot-swap enclosure. Stick the unprinted part of the power converter motherboard into the PCI Express slot on the motherboard, to keep it away from metal. I also recommend that you attach your enclosure’s topmost SATA drive slot to SATA connector 1 on your motherboard, following the progression down to the bottommost SATA drive slot connecting to SATA connector 4 on your motherboard. This way, if, say, /dev/sda1 fails, you’ll know it’s behind the topmost door in the enclosure.


(*) I ordered more parts than I needed to, and if I avoided the waste with a second RAIDbox, I think the price would be closer to $900. Specific areas of waste were due to cabling and necessary PSU upgrades.

Chimerism

Chimerism is the fascinating state of a fully-formed organism having two distinct sets of DNA.  Basically, during development, multiple (usually just two) zygotes fused and developed into a single embryo. The allocation of the DNA can happen at a high level, such as the top half of the body is DNA1, and the bottom half is DNA2. In addition to, or instead of that, the DNA allocation within individual organs can be a fairly even mix of the two sets of DNA or it can be split essentially down the middle.

The incidence of chimerism among humans is estimated to be very low, but here’s the spooky part: Consider how rarely people would ever get a DNA test to see if they had more than one set of DNA. Maybe you are a chimera and don’t know it.

We have technologies to test for known genetic diseases, but if you’re a chimera, you might provide a mouth swab to test for a disease, and get a negative result, except it’s only negative for that set of your DNA. Maybe the other set of your DNA has the bad genes but wasn’t tested. False positives could occur, too, of course. I also wonder if chimeras could be vulnerable to compound diseases that involve the two sets of DNA interacting in an incompatible way.

As we enter an era when DNA testing for medical and legal purposes seems only to be on the rise, it’ll be interesting to see if chimerism is discovered to occur at much higher frequency than previously estimated.  If so, chimerism may need to be the first DNA test done before doing any further DNA tests on an individual.

Stigmata and Demonic Possession

My understanding of the phenomena is that they only seem to happen to Catholics.  If they do happen to other Christians, they do not happen in nearly the same frequency as with Catholics.  They also seem to not happen to non-Christians at all, as far as I know.  So what does this mean?

If the phenomena are real, it implies that Catholics are uniquely vulnerable to demonic possession and more likely to be given Jesus-like wounds.  What would let demons more easily possess Catholics?  Why would Catholics be chosen to receive stigmata?  Why don’t all Catholics realize they’re in such danger?  Why do the same things not happen to everyone, or at least to non-Catholic Christians?

If the phenomena are fake, it implies that some elements within Catholic culture are more likely to believe in things like demonic possession and stigmata, otherwise the phenomena would be classified as mental illness and/or self-mutilation and/or Munchausen syndrome, be treated, and never be talked about as supernatural.  Why would some Catholics tend to believe supernatural explanations for these phenomena?

I  don’t have an answer, so I’m curious to know what the rest of you think.

Warranty Madness

I got a deep fryer for Festivus and just hooked it up on Sunday, upon which it immediately made a loud “pop!” noise and blew the breaker it was on.  After much troubleshooting, I determined that the electronics module had sacrificed itself in the name of some unknown cause.  This particular model is modular, and the electronics module is a small part of the overall unit.  I called the manufacturer, T-Fal, to return the defective bit under warranty, and I was told that the warranty “only covers the entire unit, not individual components.”  I asked if the rep realized the absurdity of that, since warranties are usually restrictive in the opposite direction.  I was told, though, that if I wanted to replace just the failed component, I could always just buy one.  Nice!

The reason I was given for the whole “all or nothing” stance is that they’d be doing what amounts to integration testing on the whole unit before sending it back, and if they couldn’t fix the problem, they’d send me a new unit.  Thanks, but I think the mechanical components are fine, based on my previous assembly, and all I need is a replacement electronics module.  My end result will be the same: a working unit.  The bottom line is that I just don’t want to pay more than I have to for shipping to get it.

I’ve been on the other end of this warranty issue, too, and I use the same philosophy.  On rare occasions when there have been component failures with computer systems that I sell, I’ve always complied with customer requests to just send back individual components.  If I know full well that one stick of RAM is bad, why in the world should I make the customer send a 35-pound computer back just so I can swap it for them?  It’s only when the problems couldn’t be remotely diagnosed or addressed that I’ve needed the entire system back.

A supervisor is supposed to call me back in 1-2 days to either try to convince me why I should pay extra for shipping, or to listen to what I want and make it happen.  In the meantime, let’s all think about what’d happen if your aftermarket car stereo’s warranty required that you send them your entire car so that they could do proper integration testing on a replacement unit, or if your rechargeable battery warranty required you to send them any and all devices you used with the batteries so they could test the replacement batteries.

Buying Online: Convenience vs. Security

I run an online store, and one of the features we offer is shipping to an address that isn’t the customer’s billing address. I did this for a couple of years without verifying that the shipping address was connected to the card holder, until some limo driver in Ohio defrauded us and other online stores by ordering items as one of his previous customers. He knew the real billing address because he picked the victim up at his house, and he used his own home address as the shipping address. Unfortunately, not all fraudsters are as dumb as this man, or the problem might weed itself out genetically.

Nowadays, we call the credit card company to verify that the customer’s shipping address is registered as an alternate shipping address with their account. This ensures that the actual card holder(*) has provided the address, rather than just someone who knows the card number and the real customer’s billing address. Ironically, even though I always offer to provide all the information and only ask for a yes or no response on the validity of the shipping address, some credit card companies won’t even talk to me because I’m not the cardholder, citing “security reasons.” It’s for “security reasons” that I’m doing this in the first place. Thankfully, these short-sighted policies are rare.

I know of no way to automate this, and it’s not clear that it’s even possible without each credit card licensee wanting to provide this service and agreeing on a common API(**). This means that the process is not convenient for the store, and it’s not convenient for customers who have no alternate shipping address on file and are asked to add one. Most large online stores don’t go through the pains of manually doing this verification, preferring to combat fraud through some combination of automated heuristics that don’t work all the time, random manual auditing, and after-the-fact actions like litigation. This means that all someone needs is your address and your card information, and they can probably order whatever they want and have it shipped wherever they want. It gets worse when the fraudster lives near you, since it’d pass any distance-based heuristics for guessing whether or not a shipping location is ok.

Think about who could have your address and your card information: cab companies, limousine companies, Lowe’s/Home Depot delivery, moving companies, or a cashier at a restaurant who copied your card information and looked up your address. “But Bob,” you say. “This is FUD! If it’s so easy to get this information, why hasn’t everyone become the victim of credit card fraud?” First of all, lots of people do fall victim to it. Credit card fraud is big business on all sides of the issue. Having said that, there’s a relatively small percentage of people who have access to your full credit card information, since it’s in the best interests of the business owners to make sure they minimize the number of eyes that can see it. I suppose it’s also fairly easy to catch most people who commit credit card fraud, especially when they can’t resist ordering lots of items and helping investigators to narrow down the search, so perhaps the failures of individuals sustain the notion amongst most would-be fraudsters that it’s too much risk for too little gain. It may also be the case that everyone has had someone try to commit fraud using their credit card, but some combination of store action and credit card company heuristics caught it before it happened.

Whether or not you choose to worry about this is up to you, but if you want to help stores out that do take the extra steps to make sure items bought with your card can only be sent to addresses that are associated with you, it’s easy to add an alternate shipping address to your credit card account. Just call up customer service, tell them you’d like to list an alternate shipping address, and they’ll know what to do because it’s far more common than you might have known. Then when people like me go through the manual process of calling up to make sure it’s your alternate shipping address and not some random place where a fraudster decided to hang out and wait, your order will go through without further intervention on your part.


(*) or at least someone who has access to so much sensitive information that they seem to be the actual cardholder

(**) This is one of the rare times when I think government regulation would help.

Acronyms FTW

I learned recently that the acronym FTW usually means “For The Win” and not “Fucks The World” as I had thought. Strangely, the alternate version is very similar in context, but I now know why utterances like “Ubuntu FTW” didn’t quite seem to fit the character of a typical Ubuntu supporter.

Wind Turbine Research

We occasionally get very strong winds, but it’s often dead calm. I was toying with the idea of getting a good wind speed gauge to get some idea how much electricity I’d get out of various wind turbines, but then I came across the Beaufort Scales which offer a decent heuristic for wind speed and cost nothing. It’s by no means as accurate as a wind gauge, but it’s close enough to get some idea of whether or not a wind turbine would be feasible.

Today, we have 25mph-32mph winds, by my rough estimation. If I got a low-end $538 turbine (and the cost doesn’t include shipping or installation) it seems that I’d get between 250W and 500W(*) right now. At 9.5c/kWhr in my area, $538 would buy 5,663kWhrs (5,663,000 Whrs) of electricity. Doing the math, it’d take the turbine between 1.29 and 2.58 years to produce that amount of electricity if the wind were always as strong as it were today. That isn’t really feasible, given that wind speed here is usually low. I guess I’ll just have to wait for nanosolar to sell to the general public.


(*) The specs say it can do 400W max, but the chart indicates 500W max. It’s not clear which to believe.

Bad Behavior has blocked 223 access attempts in the last 7 days.