gyttja

CrashPlan on QNAP

David — Sun, 09 Oct 2011 20:09:41 +0000

First up on my QNAP: offsite backup of my data in “the cloud”. QNAP’s recommended and integrated cloud backup provider was a bit too expensive for my taste, especially since I plan to backup nearly 1TB of data. After some serious googling I found the perfect candidate: CrashPlan.

CrashPlan’s software–written in Java–has two components, a server and a GUI client. This makes it a perfect candidate for QNAP. Get the server running in headless mode on my QNAP, then connect to it via the GUI client from my desktop machine.

First up with the QNAP, install Optware IPKG.
Then follow Cokeman’s excellent guide for CrashPlan on a QNAP.
Installed CrashPlan for OSX and modded the ui.properties file to servicePort=4200 as in CrashPlan’s docs (to edit I used TextMate with custom command-line alias “mate”):
$ mate /Applications/CrashPlan.app/Contents/Resources/Java/conf/ui.properties
Added a tunnel to my QNAP in SSHKeyChain (an excellent program to manage SSH connections and tunnels):

SSHKeyChain CrashPlan tunnel configuration
Open CrashPlan on Mac. Because of the ui.properties configuration change, you will now be connecting from CrashPlan’s Mac GUI to the CrashPlan engine on QNAP. Go through the CrashPlan setup, configure backup options, done!

Connecting to headless CrashPlan engine via OSX GUI

Auf Wiedersehen Frankenstein, hello QNAP

David — Sat, 08 Oct 2011 21:36:58 +0000

My homemade server running Ubuntu started to experience strange hardware issues (random crashing, etc). Over the course of a year I replaced and fiddled with memory, motherboard, and hard drives yet still I had random crashes (would even randomly crash running memtest, so I ruled out a software issue). Since the server was my home NAS, I began to rethink the idea of maintaining unreliable hardware (and throwing money away left and right).

Out with Frankenstein, enter the QNAP TS-459 Pro II Turbo NAS. Fitted with four WD 2TB drives in RAID-5 my new QNAP has been extremely reliable, quiet, energy efficient (measured 39W at load!) and space-saving! Frankenstein was a huge server tower, the new QNAP is a tiny 18x18x24cm block that rests nicely under a cabinet. And quiet enough to be unenclosed: fan noise is unperceptible, yet faint hard drive noise can sometimes be heard if the room is totally silent. The QNAP is connected to a CyberPower UPS (as are my internet modem and router). The UPS and NAS fit nicely under a cabinet in the living room:

QNAP TS-459 Pro II Turbo NAS and CyberPower DX 600E UPS Green Power

The QNAP’s web GUI is less than perfect (I find it slow, illogical, and limiting) and as I subsequently found out, the command-line configuration is also less than perfect. But the reliability, quietness, and “it just works”-feel of the QNAP so far has me quite happy. That said, going from a completely configurable linux server to a more closed system like the QNAP was going to take some getting used to. I plan on writing a series of posts documenting the changes I make to my new QNAP to customize it to be my perfect little AFP NAS.

Stay tuned, more to come!

Reduce the size of MySQL ibdata1 on OSX

David — Thu, 18 Nov 2010 16:59:35 +0000

So I finally figured out why my TimeMachine backups were becoming bigger and bigger, 3-4GB backing up every day when I come home from work… It seems that my MySQL database file (/usr/local/mysql/data/ibdata1) keeps getting larger and larger, unnecessarily, even if I deleted databases, tables, etc. It seems even if I only update a few rows in a table in a small database, the ginormous idbdata1 file grows and then get marks as a candidate for backup into TimeMachine. Ugh.

I did some digging and found this interesting tutorial on how to clean up InnoDB storage files. Here I’ll explain what I specifically did on my OSX 10.6.5 machine with MySQL v5.1.38.

If you’re not a cowboy, stop MySQL, backup all files, then start MySQL again (I used the System pref to stop/start MySQL, feel free to use the command-line instead):
```
$ sudo cp -R /usr/local/mysql/data /usr/local/mysql/data.bak
```

Export all data from MySQL:

$ mysqldump -u root -p --all-databases > alldatabases.sql

Drop databases in MySQL (except “mysql” and “information_schema”):

$ mysql -u root -p
mysql> show databases;
mysql> drop database XXXX;

or use this great one-liner to delete all databases, modded a bit so it would work for me:

# measure twice, cut once... make sure we are deleting what we should be deleting
$ mysql -u root -p  -e "show databases" | grep -v Database | grep -v mysql | grep -v information_schema | awk '{print "drop database " $1 ";select sleep(0.1);"}'
# now delete them
$ mysql -u root -p  -e "show databases" | grep -v Database | grep -v mysql | grep -v information_schema | awk '{print "drop database " $1 ";select sleep(0.1);"}' | mysql -uroot -ppassword

Stop MySQL again
Add the following to the [mysqld] section in /etc/my.cnf:
```
[mysqld]
innodb_file_per_table
```

Remove the files:

$ sudo rm /usr/local/mysql/data/ibdata1
$ sudo rm /usr/local/mysql/data/ib_logfile0
$ sudo rm /usr/local/mysql/data/ib_logfile1

Start MySQL again
Reload the databases from the sql dump-file:
```
$ mysql -u root -p < alldatabases.sql
```
Verify that your database(s) are working properly

Delete the backup

$ sudo rm -rf /usr/local/mysql/data.bak

Done!

After this modification my TimeMachine backups are much more reasonably-sized. Very nice!

Furthermore, I noticed the same infamously large ibdata1 file on our continuous-integration build server at work: it was 40GB! I applied the same modifications as above. That server runs Ubuntu 10.10, the MySQL files are located in /var/lib/mysql/data, but otherwise the steps are pretty much the same. Even build-server bongo seems snappier now, and the total size of the MySQL data folder is 1GB (insane that there was 39GB of “dead” data in the ibdata1-file…)

Very happy.

AirPrint on Ubuntu

David — Thu, 11 Nov 2010 22:12:49 +0000

How to print from iPhone/iPad (iOS 4.2) via Ubuntu

I just noticed that with the latest 10.6.5 update, Apple at the last minute disabled support for printing from iOS 4.2 devices to shared printers in OSX. Instead of trying to overwrite new drivers with prerelease versions, I thought that there must be someone, somewhere out there who has figured out how to do this using the fine, free tools available to us. I like open standards, open tools, open source.

From what I understand AirPrint supports printing two ways:

Via officially-supported HP ePrint printers that advertise themselves on your subnet
Via shared printers from Mac/Windows (this is the part that apparently got axed, at least in the latest 10.6.5 update… don’t know how Windows will fare)

I don’t have a new HP ePrint printer, but I do have a wonderful Ubuntu 10.04 server running avahi (open source Bonjour/mDNS responder)… I wonder if there’s a way to have my server act as an AirPrint device and then send the print job to my networked printer?

Enter this post. Excellent! Set up a Bonjour service, point it to a shared printer, done!

Here’s what I had to do in Ubuntu to get printing to work, YMMV:

Install my printer (networked HP Color LaserJet 1515n) using the graphical configuration utility (System->Administration->Printing). Make sure the printer is shared.

Updated my /etc/cups/cupsd.conf configuration to allow network access (default Ubuntu configuration is for localhost only):

# Only listen for connections from the local machine.
#Listen localhost:631
#Listen /var/run/cups/cups.sock
Port 631
ServerAlias *

and

# Restrict access to the server...

Order allow,deny
Allow @LOCAL

Tested remote access to Ubuntu’s cups web GUI from my laptop (to make sure machines other than the Ubuntu server had access to cups): http://172.16.0.50:631/printers/
Tested that I could print remotely from my laptop to the new shared printer on Ubuntu (simple “Add Printer” in OSX and then print test page)

Configured avahi with a new printer service /etc/avahi/services/printer.service. As mentioned in the original link, rp and adminurl are the most important configuration bits.




  HP 1515n
  
    _ipp._tcp
    _universal._sub._ipp._tcp
    631
    txtver=1
    qtotal=1
    rp=printers/hp-cp1515n
    ty=HP 1515n
    adminurl=http://172.16.0.50:631/printers/hp-cp1515n
    note=HP Color LaserJet cp1515n
    priority=0
    product=virtual Printer
    printer-state=3
    printer-type=0x801046
    Transparent=T
    Binary=T
    Fax=F
    Color=T
    Duplex=T
    Staple=F
    Copies=T
    Collate=F
    Punch=F
    Bind=F
    Sort=F
    Scan=F
    pdl=application/octet-stream,application/pdf,application/postscript,image/jpeg,image/png,image/urf
    URF=W8,SRGB24,CP1,RS600

Printed a page from iOS simulator
Printed a page from iPhone 3GS with iOS 4.2.1:
Done!

I’m not sure if all printers will work out of the box with this configuration, but since my printer supports PostScript I assume it can rasterize pretty much anything iOS will send it. In any case, I didn’t have to configure any filters or print settings. It just worked. Hopefully Apple won’t further cripple AirPrinting by also “patching” iOS so that only HP ePrint devices are supported and it no longer recognizes Bonjour services with subtype _universal._sub._ipp._tcp. We’ll see what happens!

Now all I need is an iPad. And a reason to print.

Updated 2010-11-21: repaired some mangled XML due to WordPress’s less-than-nice HTML-parsing

Updated 2010-11-23: added post subtitle, added screenshot from iPhone