Cool Beans

So good:

% python
Python 2.5.4 (r254:67916, Nov 27 2009, 18:35:00)
[GCC 4.2.1 20070719  [FreeBSD]] on freebsd8
Type "help", "copyright", "credits" or "license" for more information.
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
>>>

I like paranoia in design. Well, I take that back. I don't like it when it inhibits programming experimentation and creativity, but I do like it when it comes to services, and most especially when it comes to backup.

I wanted to write about my experiences with consumer offsite backup services (e.g. Mozy, Carbonite, Jungle Disk) as well as the plain practice of having a redundant storage device onsite. But all that was side-tracked when I recently needed to quickly backup my servers, and discovered tarsnap.

Tarsnap was created by Dr. Colin Percival, the FreeBSD Security Officer. He also worked on the utilities portsnap and freebsd-update. All of these tools are run in the command-line, and greatly simplify the maintenance (and now backup) of unix systems.

The things I like about tarsnap are encompassed in its listed design features:

• It encrypts information before sending it to the Amazon Cloud (AWS), so if a person somehow gets access to the cloud servers, the information is unreadable. Even metadata is unreadable (filenames, sizes, names of the backups, etc.).
• It's easy to learn and use, and quite scriptable.
• It breaks your backup into variable-length blocks, and keeps track of these, so if another archive contains the same data, that same block does not get re-uploaded. As long as any backup references that piece of info, it'll remain stored and not be deleted. It's like storing incremental changes, but so much cooler.
• It's quite cheap. Especially if used for server backups, which typically won't take terabytes of space. 300 picodollars per byte transferred ($0.30/GB), and 300 picodollars per byte per month stored (again, $0.30/GB-month).

• The client code is open to peer review.
• It uses AWS! Geo-replication concerns are no longer a problem.
• It runs on almost any OS, yes, even Cygwin.
• You can secure the backup keys so that, if a person breaks into your system and starts deleting everything, they cannot also read or delete your backed up data. Even the backup key security is fascinating!
  
• Not to mention the author was surprisingly responsive via email to some questions I had about the web-based reporting and command-line options.
  

The tool has basically addressed almost every concern I've had about backups.

Most early "backup service" providers would simply give you space at a cost, but had very little to say about their data loss/breach liability or who had access to their systems. Others would claim their service is "secure using 128-bit encryption" but that only meant they installed an SSL cert for transfer; backups were still unencrypted on disk. Then there are those who tout The Cloud, and how much safer it is, without a hint about data geo-redundancy (or if they have more than one data center).

But with tarsnap, I just install it, create a key, split the keys to read, write, and delete keys (encrypting the read and delete keys), and with a command I'm securely backing up entire directories. Online. In the Amazon™ cloud.

tarsnap --keyfile /usr/tarsnap.key -c -f backup-2009-11-27 /usr/home /usr/local/etc /etc

How much easier could that be? And if your backups aren't gigabytes large, the small pre-payment of the online service could last a very long time.

My only concern is that the tarsnap server is, as of this time, a single point of failure. We have no option of having our own tarsnap interface to our own personal AWS accounts. So 1) we have to trust that it is indeed being sent to AWS by Dr. Percival (is that too paranoid?), and 2) we have to hope that the tarsnap server is fault tolerant and can be restored quickly. Granted, this problem exists for any online backup service unless you write your own. We depend on third-party uptime for any service, so it boils down to who's thought it through, and has addressed our backup concerns.

For now, I am glad "production" isn't the only place some important data lives. I am glad to not have to manually tar.gz files and move them to my workstation to be picked up by my desktop backup scheduler. With tarsnap, I was able to upgrade from FreeBSD 7.2 to 8.0-RELEASE and not worry (too much) about having to rebuild the server in case all failed. (I didn't need to.)

Online backup for the truly paranoid. Who backs stuff up who isn't paranoid?

There are lots of times I'm waiting in a line, in an office, at a transit station, or just generally without an urgent task but not enough time to dent my reading list or write some code. It's times like this that I often bust out my perfect mobile computer (a.k.a. iPhone) and hit various news apps like my RSS reader, CNN, USA Today, and AP Mobile.

But sometimes I don't feel like being inundated with stories on death and weather. Sometimes I want to find interesting stuff on the web without a concerted search effort. "Give me cool stuff, and don't make me think too hard about it."

Enter Prkl8.com. PrkL8 is a web-based content discovery service that can live in your Firefox toolbar and do just that: Give you cool stuff on the web. You can say you like or hate what it offered, and even provide public comments. It's a simple application that is your gateway to undiscovered interestingness.

It gets better: There's now a PrkL8 iPhone app.

I've been fortunate enough to participate in the private beta of this app. It's easy to get lost in the rabbit hole as it leads you to sites randomly or based on your areas of interest. Sign up and select categories, and you're on your way. Vote whether you like or hate what you were given, and it gets meshed into a huge customized learning engine.

Here's an excerpt from their site:

PrkL8 is a content discovery engine. What's that? Push a button and something cool pops up. Two other buttons let you tell us what you like and what you don't. The more you use those buttons, the better our suggestions get. Three buttons, that's all you'll need.

We are not a search engine. If you know what you're looking for, load up Google and get right back to work. We're more about the fun. We serve up rich, delicious web content without making you think too hard about it. You just push a button and BOOM! Something new.

Behold, the joy of personalized content discovery.

We're not going to help with your report. We're not going to settle any bets or debates. What we will do is show you cool new stuff that you might never have thought to look for. So come kill some time with us. Push a few buttons. Shake your Web Booty.

Stand By Me | Playing For Change | Song Around The World from Concord Music Group on Vimeo.

Time for a different kind of engineering. Not software, but beautifully constructed nonetheless. At first I didn't see what was going on with this video. But then I realized it's from worldwide asynchronous sources.

Jerry Cheung, co-founder of Outspokes, is one of those developers who makes you feel better about projects. No matter how harrowing the ordeal, how intractable the critical bug seemed, he would show up with a smile, a welcoming demeanor, and an patient attitude of figuring things out. I worked with Jerry when he was a graduating student in 2008 at UC Berkeley. He was one of our programmers at RSSP, and took primary responsibility for Ruby applications development. Even during the worst of finals and class projects, he'd still, much to my surprise, find the time and energy to whittle down the bug list. He really delivered. I was sorry to see him move on to the private sector.

Seeing one of his personal endeavors take off is something I can't help sharing. And believe me, it's not just because I've seen him work. It's because he's got this web application that blows my mind. Who knew you could do so much within your own website?

So if you're developing something cool that needs quick and relevant feedback, take advantage of this offer. Just check it out. A line of code added to your site and you'll have expanded your feedback engine by leaps and bounds!

 Get a $10/mo premium Outspokes account for free through January 2010!

Outspokes is a great new collaboration tool for anyone involved in creating or managing a web site. Whether you’re a freelancer with an outspoken client, a consulting firm dealing with design by committee, or just a remote team all producing the next great web application, Outspokes can help you communicate faster and more clearly.

Just follow @Outspokes and tweet the following:

Excited to get my free premium @Outspokes account, thanks to this tweet!
RT to get yours: http://bit.ly/3RSgrP

Then we’ll message you to confirm your account and get you on the premium plan!

What happens in January? If you love us, please stay on our paid premium plan. Otherwise, we’d be happy to provide you with our current free plan.

And here's a cool interview article by Serena Wu: Spotlight: Outspokes by Arthur Klepchukov and Jerry Cheung.

Lots of security-conscious users say we should use whole disk encryption. It prevents files from being read when you're not logged in. This is good protection if your computer is lost or stolen. If not used, tools like Knoppix, Slax, or DamnSmallLinux can easily be used to boot a fully functioning operating system (from, say, a CD or USB key) and view your files' contents.

Even deleted files are not truly deleted. They are not listed but also not fully removed from the disk. Typically, they exist without a file pointer to their location, so an undelete program or disk scanner can easily recover and/or view them.  The only remedy to this is to "wipe" them, and people don't typically expend the extra steps to perform a file wipe.

So your hard disk in its default state presents problems. Aside from oft-cited airport laptop searches, it's just not cool to have your files ready to be read by almost anyone with a USB key or bootable CD. Hence the push for encrypting everything on the whole disk. This is good for a few reasons:

• It's ubiquitous: You don't have to think about it and the drive is protected.
• There's little performance hit for doing this.
• It's free and easy to configure.
  
• The default state of files is secure, a.k.a. fail-secure.
  

But this is a bit of overkill, in my opinion. If you also like things like versioning, snapshots, and incremental backup (e.g. Apple's Time Machine), whole-disk encryption forces a choice: Do you want smart automatic backups? Or do you want to backup your entire disk volume (BIG!) each and every time?

I puzzled over this for a while, sticking with whole-disk encryption while not having a safe backup somewhere. I figured privacy and security were more important than the risks of data loss. (Huh?)

But with a new computer, I realized that automated unattended backups gave great peace of mind. How could I have incremental, optimized, regular scheduled backups while having some form of data privacy and security? One option was to create various encrypted volumes using Disk Utility or Truecrypt. But this was cumbersome, manual (needing me to remember to mount and unmount a volume all the time), and still didn't address my backup concerns.

I also realized that every single file on the computer does not need to be encrypted. Many files, like application libraries, everyday documents, publicly shareable media, etc. do not need high encryption. Just don't mix tax forms and healthcare PDFs with the latest lolcats jpegs saved to the Documents folder.

It was then that I happened upon Tao Effect Espionage.

Espionage is a customizable encryption tool that can secure areas you choose. It takes advantage of the sparsebundle and sparseimage functionality of OSX, but automates the process so you don't have to think about encryption. This forces you to keep your files organized, protect sensitive ones, and keep the rest of your system and non-sensitive files available for simple backups. Each time you want to access an important doc, a prompt gets your authorization, and you are in. Otherwise, it's fail-secure.

It even works on the Application-level. So let's say you want to password-protect and encrypt your email. This is not trivial in OSX. But with Espionage you can configure your Mail app to be protected, and it will unlock and lock the mail directories whenever the app is opened or closed!

Truth-in-advertising: There's a little trickery that needs to happen with Espionage and Time Machine. Espionage has its own backup functionality, and it's best to ignore protected directories in the Time Machine preferences. But I believe even with this slight config, the application is a great balance between keeping files secure, and keeping things simple, available, and smartly backed up.

In so many words of this article, it's easy to resist and do nothing. But don't do nothing! Keep your files both secure and backed up! You no longer have to choose one or the other; use Tao Effect Espionage.

Let the record show that Big Brother was established by us, the mobile camera-wielding public. In response to all the would-be police brutality videos captured by anyone with a phone, San Jose has started their own CYA program to document and record all interactions using TASER AXON video recording technology.

When we wonder how we got to a world where someone's always watching, blame ourselves. Think about it. How does someone monitoring you change your true behavior? Is this for the better? Welcome 1984.

If you cannot see embedded video, download it here: http://serve.castfire.com/video/181873/181873_2009-10-26-233050.mp4