Cool Beans

So good:

% python
Python 2.5.4 (r254:67916, Nov 27 2009, 18:35:00)
[GCC 4.2.1 20070719  [FreeBSD]] on freebsd8
Type "help", "copyright", "credits" or "license" for more information.
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
>>>

I like paranoia in design. Well, I take that back. I don't like it when it inhibits programming experimentation and creativity, but I do like it when it comes to services, and most especially when it comes to backup.

I wanted to write about my experiences with consumer offsite backup services (e.g. Mozy, Carbonite, Jungle Disk) as well as the plain practice of having a redundant storage device onsite. But all that was side-tracked when I recently needed to quickly backup my servers, and discovered tarsnap.

Tarsnap was created by Dr. Colin Percival, the FreeBSD Security Officer. He also worked on the utilities portsnap and freebsd-update. All of these tools are run in the command-line, and greatly simplify the maintenance (and now backup) of unix systems.

The things I like about tarsnap are encompassed in its listed design features:

• It encrypts information before sending it to the Amazon Cloud (AWS), so if a person somehow gets access to the cloud servers, the information is unreadable. Even metadata is unreadable (filenames, sizes, names of the backups, etc.).
• It's easy to learn and use, and quite scriptable.
• It breaks your backup into variable-length blocks, and keeps track of these, so if another archive contains the same data, that same block does not get re-uploaded. As long as any backup references that piece of info, it'll remain stored and not be deleted. It's like storing incremental changes, but so much cooler.
• It's quite cheap. Especially if used for server backups, which typically won't take terabytes of space. 300 picodollars per byte transferred ($0.30/GB), and 300 picodollars per byte per month stored (again, $0.30/GB-month).

• The client code is open to peer review.
• It uses AWS! Geo-replication concerns are no longer a problem.
• It runs on almost any OS, yes, even Cygwin.
• You can secure the backup keys so that, if a person breaks into your system and starts deleting everything, they cannot also read or delete your backed up data. Even the backup key security is fascinating!
  
• Not to mention the author was surprisingly responsive via email to some questions I had about the web-based reporting and command-line options.
  

The tool has basically addressed almost every concern I've had about backups.

Most early "backup service" providers would simply give you space at a cost, but had very little to say about their data loss/breach liability or who had access to their systems. Others would claim their service is "secure using 128-bit encryption" but that only meant they installed an SSL cert for transfer; backups were still unencrypted on disk. Then there are those who tout The Cloud, and how much safer it is, without a hint about data geo-redundancy (or if they have more than one data center).

But with tarsnap, I just install it, create a key, split the keys to read, write, and delete keys (encrypting the read and delete keys), and with a command I'm securely backing up entire directories. Online. In the Amazon™ cloud.

tarsnap --keyfile /usr/tarsnap.key -c -f backup-2009-11-27 /usr/home /usr/local/etc /etc

How much easier could that be? And if your backups aren't gigabytes large, the small pre-payment of the online service could last a very long time.

My only concern is that the tarsnap server is, as of this time, a single point of failure. We have no option of having our own tarsnap interface to our own personal AWS accounts. So 1) we have to trust that it is indeed being sent to AWS by Dr. Percival (is that too paranoid?), and 2) we have to hope that the tarsnap server is fault tolerant and can be restored quickly. Granted, this problem exists for any online backup service unless you write your own. We depend on third-party uptime for any service, so it boils down to who's thought it through, and has addressed our backup concerns.

For now, I am glad "production" isn't the only place some important data lives. I am glad to not have to manually tar.gz files and move them to my workstation to be picked up by my desktop backup scheduler. With tarsnap, I was able to upgrade from FreeBSD 7.2 to 8.0-RELEASE and not worry (too much) about having to rebuild the server in case all failed. (I didn't need to.)

Online backup for the truly paranoid. Who backs stuff up who isn't paranoid?

There are lots of times I'm waiting in a line, in an office, at a transit station, or just generally without an urgent task but not enough time to dent my reading list or write some code. It's times like this that I often bust out my perfect mobile computer (a.k.a. iPhone) and hit various news apps like my RSS reader, CNN, USA Today, and AP Mobile.

But sometimes I don't feel like being inundated with stories on death and weather. Sometimes I want to find interesting stuff on the web without a concerted search effort. "Give me cool stuff, and don't make me think too hard about it."

Enter Prkl8.com. PrkL8 is a web-based content discovery service that can live in your Firefox toolbar and do just that: Give you cool stuff on the web. You can say you like or hate what it offered, and even provide public comments. It's a simple application that is your gateway to undiscovered interestingness.

It gets better: There's now a PrkL8 iPhone app.

I've been fortunate enough to participate in the private beta of this app. It's easy to get lost in the rabbit hole as it leads you to sites randomly or based on your areas of interest. Sign up and select categories, and you're on your way. Vote whether you like or hate what you were given, and it gets meshed into a huge customized learning engine.

Here's an excerpt from their site:

PrkL8 is a content discovery engine. What's that? Push a button and something cool pops up. Two other buttons let you tell us what you like and what you don't. The more you use those buttons, the better our suggestions get. Three buttons, that's all you'll need.

We are not a search engine. If you know what you're looking for, load up Google and get right back to work. We're more about the fun. We serve up rich, delicious web content without making you think too hard about it. You just push a button and BOOM! Something new.

Behold, the joy of personalized content discovery.

We're not going to help with your report. We're not going to settle any bets or debates. What we will do is show you cool new stuff that you might never have thought to look for. So come kill some time with us. Push a few buttons. Shake your Web Booty.

Lots of security-conscious users say we should use whole disk encryption. It prevents files from being read when you're not logged in. This is good protection if your computer is lost or stolen. If not used, tools like Knoppix, Slax, or DamnSmallLinux can easily be used to boot a fully functioning operating system (from, say, a CD or USB key) and view your files' contents.

Even deleted files are not truly deleted. They are not listed but also not fully removed from the disk. Typically, they exist without a file pointer to their location, so an undelete program or disk scanner can easily recover and/or view them.  The only remedy to this is to "wipe" them, and people don't typically expend the extra steps to perform a file wipe.

So your hard disk in its default state presents problems. Aside from oft-cited airport laptop searches, it's just not cool to have your files ready to be read by almost anyone with a USB key or bootable CD. Hence the push for encrypting everything on the whole disk. This is good for a few reasons:

• It's ubiquitous: You don't have to think about it and the drive is protected.
• There's little performance hit for doing this.
• It's free and easy to configure.
  
• The default state of files is secure, a.k.a. fail-secure.
  

But this is a bit of overkill, in my opinion. If you also like things like versioning, snapshots, and incremental backup (e.g. Apple's Time Machine), whole-disk encryption forces a choice: Do you want smart automatic backups? Or do you want to backup your entire disk volume (BIG!) each and every time?

I puzzled over this for a while, sticking with whole-disk encryption while not having a safe backup somewhere. I figured privacy and security were more important than the risks of data loss. (Huh?)

But with a new computer, I realized that automated unattended backups gave great peace of mind. How could I have incremental, optimized, regular scheduled backups while having some form of data privacy and security? One option was to create various encrypted volumes using Disk Utility or Truecrypt. But this was cumbersome, manual (needing me to remember to mount and unmount a volume all the time), and still didn't address my backup concerns.

I also realized that every single file on the computer does not need to be encrypted. Many files, like application libraries, everyday documents, publicly shareable media, etc. do not need high encryption. Just don't mix tax forms and healthcare PDFs with the latest lolcats jpegs saved to the Documents folder.

It was then that I happened upon Tao Effect Espionage.

Espionage is a customizable encryption tool that can secure areas you choose. It takes advantage of the sparsebundle and sparseimage functionality of OSX, but automates the process so you don't have to think about encryption. This forces you to keep your files organized, protect sensitive ones, and keep the rest of your system and non-sensitive files available for simple backups. Each time you want to access an important doc, a prompt gets your authorization, and you are in. Otherwise, it's fail-secure.

It even works on the Application-level. So let's say you want to password-protect and encrypt your email. This is not trivial in OSX. But with Espionage you can configure your Mail app to be protected, and it will unlock and lock the mail directories whenever the app is opened or closed!

Truth-in-advertising: There's a little trickery that needs to happen with Espionage and Time Machine. Espionage has its own backup functionality, and it's best to ignore protected directories in the Time Machine preferences. But I believe even with this slight config, the application is a great balance between keeping files secure, and keeping things simple, available, and smartly backed up.

In so many words of this article, it's easy to resist and do nothing. But don't do nothing! Keep your files both secure and backed up! You no longer have to choose one or the other; use Tao Effect Espionage.

This is supposedly an internal Netflix memo, but when posted on Slideshare it was discovered and spread virally. As you read through it, you will feel this coolness vortex pulling at you, making you wonder if it could ever be possible to have such a culture where you currently work. Such freedom, respect, fostering of innovation and excellence.

I especially like the sections on being "highly aligned and loosely coupled," which goes against anything I've witnessed before.

On August 19, the Silicon Valley Web Java User Group had the chance to hear Dan Allen (author of Seam in Action) talk about JSR 299: Contexts and Dependency Injection (CDI), and what's new in JavaServer Faces 2.0. Dan gave me permission to post the audio recording of this fascinating Java tech talk. This is quite a big change to JEE 6, and I can't wait till it's supported across the major Java containers.

There was so much new to talk about that we ran out of time. Hopefully I can also track down the slides so some of the code examples can be followed.

• JSR 299: Contexts and Dependency Injection for Java EE (90MB, 1 hour 34:35 minutes).
• JSF 2.0: What's new in JavaServer Faces 2.0 (15.9MB, 16:34 minutes).

Update Sept 10, 2009: Here are links to the slides so you can follow along.

• CDI
• JSF 2.0
  

Thanks to Google headquarters for a great venue, and to the SV Web JUG for organizing the event.

This title came from my misreading of Are you a language wonk?, a blog article about building a language. Sounds really cool, but I currently have no interest in creating my own language. The topic brought to mind a discussion I had with a very sharp former Cal student. He stated to all the newb programmers in the team, "Good programmers are language agnostic."

This got me to thinking about all the different religious battles I've witnessed (and been in) about what the best programming language is for the task at hand. Without delving into heuristics, bias, and conventions, here are a few things I think support mht's claim:

•  It is likely much easier for younger minds to switch-task and switch-context often.
• Having the ABC's in place, i.e. the theory (versus learning only by projects that require a certain job to be done) helps one be more platform-agnostic.
• We will, under stress or deadline, always resort to the path of least resistance. In programming, this is the path that we know best, the one where we are most practiced. (This is why it's good to regularly practice outside one's own comfort zone.)
• Learning on the job (also known as "experience") is not a substitute for personal interest and personal development in computer science.
• Mastery takes 10,000 hours. But master one language, and there is collateral benefit to other languages.
  
• Writing lots of code is the only way to mastery.

So think about your job. Are you tied to a single way of thinking/doing? Do you have strong opinions of what is "the best" without having fully understood the alternative languages? These are signs you are still on a monk's path to mastery, and would benefit from some agnosticism.