ExpanDrive

If you’re happy using gmail.com as your user interface, then there’s no need to read further into this blog post. The Gmail web interface is the best I’ve ever used, but it’s not as good as a native, desktop app. If you wail on your applications like I do, then you need fast and local. Apple’s Mail.app isn’t prefect (no email client is), but it’s good enough. The interface is snappy, the search is fast and reliable, and I can drag and drop attachment files onto it.

I’m going to explain how to setup Gmail to work the right way with Mail.app and the iPhone. “Right”, of course, means “the way I want it to work.” In particular, this means that I want “deleting” something in my client to make it “archived” in Gmail for eternity. I almost never want an email to be deleted forever. I also want Sent Messages and Drafts to work in a reasonable way, but delete -> archive is the big one for me.

NB: This guide is a work in progress, a living document. Check back for updates. Feedback is actively solicited.

Setting up Mail.app

1. Follow the directions at the Gmail Help Center. Make sure you enable IMAP in the Gmail web interface before you play with Mail.app. Also, when Mail.app notices you’re setting up a Gmail account and offers to do it for you automatically, don’t fall for it. It will setup and POP3 account, which is worthless, and you’ll have to start over. Finally, note that every time you’re ever asked for a “User Name:” you’re going to have to enter your whole Gmail address username@gmail.com.

2. As always, the devil is in the details. In this case, the details are the “Mailbox Behaviors” panel for the account in Mail.app preferences. Google has Recommended IMAP client settings, an obscure Gmail Help Center page, and the Mail.app settings are unlinkably hidden by Javascript. I believe Google’s recommended settings are wrong. Well, I think their “Sent” recommendation is wrong, I agree with the other ones. Here is my recommendation.

If you match these settings then your Drafts, Sent, and Junk folders will automatically sync with the corresponding Gmail folders. Your local Trash folder will remain empty, and your deleted messages will get archived in Gmail’s “All Mail” folder.

If “Sent” is not checked, then Gmail store a copy in its Sent Mail folder, and Mail.app will store a copy in its Sent folder. These two copies will be different, and deleting one will not delete the other. The ”Delete sent messages when:” setting will only apply to the Mail.app copy. If that’s what you’re into, then go for it.

A few caveats

Gmail Folders in Mail.app

After following the above directions, Mail.app may or may not have figured out how to merge the the Gmail tags with the corresponding Mail.app folder. For example, the Gmail “Spam” tag and the Mail.app “Trash” folder. In the worst case, if Mail.app figures out nothing, then your Gmail setup in Mail.app’s main window will look like the image to the left. If this is the case, then click on a folder that Mail.app did not figure out for you. Let’s start with “Drafts”. Click on the “Mailbox” menu, pull down to “Use This Mailbox For”, and select “Drafts”. Repeat as necessary for “Sent Mail”, and “Spam”. Do not do this for “Trash”. If you do, then Mail.app will silently turn on “Move deleted messages to the Trash mailbox”__ and your archived mail will start getting silently deleted. Oofta. Also, for some reason my copy of Mail usually won’t merge “Spam” with it’s “Junk” folder. I’ve given up on trying to make it work, because it doesn’t really matter anyway.

You may have noticed that with the above settings you lose the Trash folder in Mail.app. That is, you will have no record of your recently deleted messages except for Gmail’s “All Mail” folder (Mail.app’s Trash will be forever empty). I’m fine with that, since deleted messages are the overwhelming majority of my “All Mail” folder. Another option is to make a costume tag in Gmail like “[Deleted Messages]”, and then set this to be the Trash folder in Mail.app. Then, when you delete the message Mail.app will remove all existing tags, and add a [Deleted Messages] tag. If you delete it againOther option. Make a [Deleted Messages] folder yourself.

Setting the “IMAP Prefix”

In the Mail.app sidebar your Gmail tags will appear as folders. The built in Gmail tags (All Mail, Spam, Starred, Trash) appear nested in a folder named [Gmail]. The rest of your tags appear as folders at the same level as the [Gmail] folder. Some people recommend that you set your IMAP path prefix (in Preferences -> Accounts -> Advanced -> IMAP path prefix:) to Gmail. I don not recommend this. It will make your built in Gmail tags appear at the top level, but it will make your personal tags disappear entirely. Also, if you fiddle with it Mail.app will re-download your entire mailbox. Fair warning.

Setting up iPhone Mail

First, and important FYI: there are two different and barely distinguishable types of Gmail accounts on the iPhone. If you go to “Mail, Contacts, Calendar” settings, then “Add account…”, and select “Gmail” then you will get an account type that I will call “POP3”, even though I’n not certain it is POP3. This potentially out of date video calls it POP3, though the obscure page calls it IMAP. Regardless of what it is, you can’t make it work the way we want.

Amazingly, if you let iTunes sync your Mail.app Gmail IMAP account to the iPhone, the iPhone will turn it into a maybe-POP3 account. Furthermore, if you have a mail account on the iPhone that looks like an account in Mail.app, but isn’t quite the same, then iTunes will be unhappy about syncing. So the first order of business is to stop iTunes from trying to sync your Mail.app Gmail account to your iPhone. Plug in your iPhone, find it in iTunes, click on the Info tab, scroll down to Mail Accounts, and uncheck your gmail account. Then, since you can’t plug in an iPod without it syncing, go delete the account from your iPhone.

Gmail non-IMAP “Auto-G” Type Account

The only way to get a Gmail IMAP account on the iPhone is to instead select “Other” at the account creation screen, and fill out the settings by hand. Instructions are on this page, or you can follow the above previously linked video. Note once again that you need to type out the full username@gmail.com each time.

After you’ve added your account information, quit the Settings app, then fire up Mail and let it download at least your folder structure. If you don’t do this, then the iPhone will let you fiddle with settings using only a fake INBOX folder that it assumes your account will have. Then, after it downloads your real folders, it will throw out the settings you put on the fake INBOX folder. I kid you not.

iPhone Gmail “IMAP” account

Once again, Google recommends that you keep a separate copy your sent messages on the iPhone on its Recommended IMAP client settings page, and I disagree. My settings are in the image below.

My recommended Gmail settings

Other notes

This is a lightly tested theory: if you enter username instead of username@gmail.com then you will download new messages ok, but your deletes and folder moves will not propagate to the server correctly. Oofta.

Heinous iPhone Bug: If you make a new account with the same settings as an existing account, then the client will merge the two accounts with the new one’s blank settings, and erase the data in the old one. If you delete one, then it will delete both.

Advanced IMAP Settings

Google Labs recently released Advanced IMAP Controls for Gmail. If you activate advanced IMAP controls then you can choose to make some folders “invisible” to the IMAP interface. Some people are really excited about this, because they don’t want to download their huge “All Mail” folder. Mail.app will only download it once anyway (and then you have an offline copy), and the iPhone only downloadss 50 at a time, so I don’t see what the big deal is.

The advanced controls also expose a completely incomprehensible feature to let you “turn off auto-expunge”. When this came out about 100 people reblogged it, and it’s pretty clear that no one knows what it does or how it’s supposed to work. According to the official Google post:

The IMAP protocol allows messages to be marked for deletion, a sort of limbo state where a message is still present in the folder but slated to be deleted the next time the folder is expunged. In our standard IMAP implementation, when you mark a message as deleted, Gmail doesn’t let it linger in that state — it deletes (or auto-expunges) it from the folder right away. If you want the two-stage delete process, after you’ve enabled this Lab, just select ‘Do not automatically expunge messages’ under the ‘Forwarding and POP/IMAP’ tab in Settings.

Um, ok. First off, what is “expunge” supposed to mean, and what does it mean to Google? Second, if deleted messages aren’t immediately expunged, then when are they expunged? Once a month? When you click the “expunge” button? (There is no expunge button). Third, that option to “Do not automatically expunge messages” followed by “When a message is expunged from the last visible IMAP folder: Archive the message” is pretty tempting. It’s a trap! As far as I can tell, with every other permutation of settings this will make deleted messages go to the Trash and then disappear forever.

I held out a long time before making the move to Gmail. I got an invite soon enough to get my second favorite username, but then I sat on it for years without using it. At first it was because I was too hardcore to let the gRobot read my email. Later, when I’d become less of an idealist and more of a pragmatist, it was because Gmail didn’t support IMAP. When IMAP support was added last year, I didn’t move because I was to lazy to migrate. Eventually my other mail server went down at an inopportune time, and I decided to make the leap.

Pages with potentially useful information

Deleting IMAP Messages

Apple iPhone and Gmail deleted messages

IMAP Actions

Gmail Archiving and deleting on iPhones

Advanced IMAP Controls

Advice that doesn’t quite work

FTP Drives:

They said we couldn’t do it. They said we shouldn’t do it. I said we shouldn’t do it. But we did it. Why did we do it? To prove how tough we are? To make a pile of money? Just because it was there? I can’t say for sure, but it’s done now. In version 1.3 ExpanDrive supports connections to plain ftp (and ftp/ssl) servers.

We’re calling it “beta” for now because we expect that there are still some bugs to shake out, but so far it looks pretty good. No, it looks very good. Good like streaming two movies over a local wireless connection at the same time good. That said, the ftp “standard” isn’t quite as standard as one might like if one were writing a filesystem for it, and there are a lot of different ftp servers out there. If you have an ftp server that ExpanDrive can’t connect to and you can give us a temporary login to that server, then we’ll send you a generous ExpanDrive coupon for your trouble. (And we’ll use the temporary login to try and make ExpanDrive work with your server.)

New trials for everyone:

We want everyone to love ExpanDrive as much as we do. That’s why we have a free 30 day trial period. For version 1.3 we’ve reset everyone’s trial, and we’ll continue to automatically reset the trial after every major version release. If you tried version 1.1 and it didn’t grab your fancy, then give it a try again. (We’ve also decrease the intrusiveness and frequency with which we nag you to register.)

Reconnect:

It’s not all about the new features. There also always needs to be “bug fixes and performance improvements”, and we’ve got those too. We’re mildly embarrassed at how far we let our SFTP reconnect feature slip. You know what I’m talking about: your server turned into a folder that didn’t do anything. We didn’t talk about it because we knew we were going to fix it any day now, and no one wants to draw attention to a feature that used to work. Well, I’m happy to say that not only did we fix it, we made it much better.

ExpanDrive used to guess when you’d lost a connection when it took a long time to hear back from a server. (“Hello? Server? Are you still there? Can you hear me, server? Hello…”) We now use OS X system notifications for network availability. ExpanDrive knows when the internet disappears and reappears, when your IP address changes, and when your computer is going to sleep or waking up. This means that server reconnect is faster and more reliable than it used to be, and also that you shouldn’t ever experience a “-36 error” just because you have a laggy connection.

Localizations:

ExpanDrive is currently available in German, French, Danish, and Romanian. Spanish and Japanese will be in our next release. (Many thanks to Jannis Leidel, Nicolas Doualot, CasperT, and Sebastian Pascu, ). We’d love some help with other languages!

Beta Updates:

Finally, we’ve got a lot of things cooking right now that are going to get released in the near future. If you’re like me, then you want to use the new stuff as soon as you can get your hands on it, even before the stodgy, uptight Magnetk QA department doesn’t want to give it up yet. If that’s the case then you can now choose to automatically update to the latest beta release. Our beta updates will contain no known bugs when they’re released, and they’ll be reasonably tested, but they’ll definitely be beta.

Download:

Get the latest version over at the ExpanDrive page or right here

Then send us a translation!

As Jeff mentioned previously, ExpanDrive 1.2.11 is ready for localization. There is a folder called ExampleLanguage.lproj in the Resources folder of the ExpanDrive bundle that contains the all the relevant .string files in English. All told, there are probably less than 1000 words.

At Magnetk, we love languages. Unfortunately, the only language that Jeff and I have studied formally is Latin, and Latin doesn’t even have words for most of the the things in ExpanDrive. So we’re going to have to depend on volunteers to help us make translations. We’ll happily send a 5-pack of licenses to anyone that donates a translation.

Additionally, inspired by Mike Lee, we’ll give a free site license to any reasonably sized not-for-profit or academic institution that would like to translate ExpanDrive.

Special thanks to Jannis Leidel for translating ExpanDrive into German translation.

We’re hardcore here at Magnetk. Scheme interpreters, language compilers, ray tracers (in our custom compilers), magnetohydrodynamic models, a Windows filesystem driver… We could go on. The thing is, there’s hardcore, and then there’s unnecessarily hardcore. Doing something hard is hardcore. Doing something just because it’s hard is unnecessarily hardcore.

Renderfarm with CUPS
Simon Bunker writes about how to us CUPS, the “Common UNIX Printing System”, to queue tasks for digital image render. Be careful, he warns at the end: “You could end up with a lot of scrap paper if you use the wrong queue!”

ICFP Entry in TeX
This year’s ICFP contest was to write a controller for a simulated vehicle. There was a two dimensional map with circular boulders and craters on it, and hostile creatures would chase your vehicle while your controller tried to drive it to the finish point. Steve wrote his controller in TeX. That’s hardcore because: “…TeX doesn’t even really support things as simple as floating-point multiplication or (even worse) division.”

Nicely done, guys. “Um, I’d write a scheme interpreter and then write it in scheme.”

Ever wonder how you get a new sport into the Olympics? The BBC seems to think that one way to do it is with cash. Big, million dollar stacks of cash. They claim that a Japanese cycling association paid $3,000,000 to cycling’s international governing body (the UCI) to get the “keirin” included in the slate of Olympic velodrome events.

I don’t really care about the scandal. I assume that everything about the Olympics preparation, from the venue to the events, is decided by bribery. It’s just a good segue for me to explain how awesome the keirin is. It works like this: the racers have to stay behind a pace setter (who rides either a bike or motorcycle) for most of the race. The pace setter starts out going medium-slow (about 20 mph), and over the course of about 8 minutes slowly ratchets the speed up to 30+ mph. The cyclists jockey for the prime position, right behind the pace setter. Then, at a prederminded spot about a half mile from the finish, the pace setter pulls off the course and it’s a free for all.

Check out this video (1:33) and you’ll get the idea. The action starts with the pace setter (in purple) pulling off the track.

The Japanese had to pay $3,000,000 to get that in the Olympics? Who wouldn’t want that in the Olympics? How much did Russia have to pay to get Rhythmic Gymnastics in the Olympics?

If you watch tv, then you’ve no doubt noticed the complete absence of new content which infests the summer months. For a little reprieve, check out Dr. Horrible’s Sing-Along Blog, a 15 minute, low-budget side project by Joss Whedon (the creator of “Firefly” and the “Buffy the Vampire Slayer”, and writer of the under-rated movie with the same name).

New episodes on Thursday and Saturday. Free until Sunday.

If you’re thinking in terms of a couple hundred dollars, your app probably isn’t even going to get listed in the App Store. The App Store isn’t going to be like VersionTracker or MacUpdate, where every piece of junk gets listed as it’s submitted.

-John Gruber, Friday, 27 June 2008

What. The. Fuck? http://xrl.us/kkkdy Productivity?

-John Gruber, Thursday, 10 July 2008

As you’re no doubt aware, a lot of talented, high profile programmers have announced that they’re moving to git for version control. Many of these programmers (for example, yours truly) have promised to post their impressions, their tips and tricks, their how-tos, and their workflows and the internet for the betterment of humanity.

You may have wondered why so few people have followed through with this promise. I’ll tell you: it’s because we still have no clue what we’re doing. We still pour over the cheat sheet and man pages every time we need to do something even remotely outside of our standard workflow (pull, commit -a, push). And when I say “we”, I don’t just mean those of us at Magnetk. Git is fundamentally different than subversion. It’s taking us some time to wrap our heads around it, and we’re all a little gun shy.

Well, not me. I’m not afraid of making mistakes and looking like an idiot.

Say you add git add a file or directory, and then, before you commit, decide “No, no. I did not want to add that file. I’ve changed my mind. Please do not add that file to the repository. I do not even what my coworkers to know that I thought about adding that file to the repository.” What do you do? git reset -- RegretableAdd.tex. The file will still be on disk, but it won’t get added to the repository when you commit.

If you delete or modify a file, and then decide that you want to get it back the way it was in the last commit, you can just do a git checkout fileToBeRestored.sty, and you’ll get it back. But what happens if you git rm it instead?

$ git rm DO_NOT_DELETE
$ # "I did not mean to do that. That was a mistake"
$ git checkout DO_NOT_DELETE
error: pathspec 'DO_NOT_DELETE' did not match any file(s) known to git.
Did you forget to 'git add'?

Probably not what you were hoping for, right? Can you get back DO_NOT_DELETE? In git, you bet you can.

$ git rm DO_NOT_DELETE
$ # "I did not mean to do that. That was a mistake"
$ git reset -- DO_NOT_DELETE
todo.rtf: needs update
$ git checkout DO_NOT_DELETE

And that’s all there is to it. More to come. The git-reset man page, unlike some of the other git man pages, is extremely readable. It contains sample workflows for such situations as “Undo a commit and redo”, “Undo a commit, making it a topic branch”, or “Undo a merge or pull”.

Corrections and additions are welcome.

“Anyone mind if I reboot Builder? It’s being a real bitch.”

http://www.thewebsiteisdown.com/salesguy.html