Fazal Majid's low-intensity weblog

Parallelizing the command-line

Sun, 28 Sep 2008 05:15:11 GMT

Single-thread processor performance has stalled for a few years now. Intel and AMD have tried to compensate by multiplying cores, but the software world has not risen to the challenge, mostly because the problem is a genuinely hard one.

Shell scripts are still usually serial, and increasingly at odds with the multi-core future of computing. Let's take a simple task as an example, converting a large collection of images from TIFF to JPEG format using a tool like ImageMagick. One approach would be to spawn a convert process per input file as follows:

#!/bin/sh
for file in *.tif; do
  convert $file `echo $file|sed -e 's/.tif$/.jpg/g' &
done

This does not work. If you have many TIFF files to convert (what would be the point of parallelizing if that were not the case?), you will fork off too many processes, which will contend for CPU and disk I/O bandwidth, causing massive congestion and degrading performance. What you want is to have only as many concurrent processes as there are cores in your system (possibly adding a few more because a tool like convert is not 100% efficient at using CPU power). This way you can tap into the full power of your system without overloading it.

The GNU xargs utility gives you that power using its -P flag. xargs is a UNIX utility that was designed to work around limits on the maximum size of a command line (usually 256 or 512 bytes). Instead of supplying arguments over the command-line, you supply them as the standard input of xargs, which then breaks them into manageable chunks and passes them to the utility you specify.

The -P flag to GNU xargsspecifies how many concurrent processes can be running. Some other variants of xargs like OS X's non-GNU (presumably BSD) xargs also support -P but not Solaris'. xargs is very easy to script and can provide a significant boost to batch performance. The previous script can be rewritten to use 4 parallel processes:

#!/bin/sh
CPUS=4
ls *.tif|sed -e 's/.tif$//g'|gxargs -P $CPUS -n 1 -I x convert x.tif x.jpg

On my Sun Ultra 40 M2 (2x 1.8GHz AMD Opterons, single-core), I benchmarked this procedure against 920MB of TIFF files. As could be expected, going from 1 to 2 concurrent processes improved throughput dramatically, going from 2 to 3 yielded marginal improvements (convert is pretty good at utilizing CPU to the max). Going from 3 to 4 actually degraded performance, presumably due to the kernel overhead of managing the contention.

Another utility that is parallelizable is GNU make using the -j flag. I parallelize as many of my build procedures as possible, but for many open-source packages, the usual configure step is not parallelized (because configure does not really understand the concept of dependencies). Unfortunately there are too many projects whose makefiles are missing dependencies, causing parallelized makes to fail. In this day and age of Moore's law running out of steam as far as single-task performance is concerned, harnessing parallelism using gxargs -P or gmake -j is no longer a luxury but should be considered a necessity.

Large sensor compact cameras finally on the horizon

Sat, 30 Aug 2008 17:37:21 GMT

I have stated on the record that my dream camera is a digital Contax T3 with an APS-C size sensor (or larger). Sigma launched the DP1, the first large-sensor compact this year, but it is a flawed camera, very sluggish, with a slow f/4 lens, and its Foveon sensor tops out at ISO 800, making it in practice a less capable low-light camera than my Fuji F31fd.

A few weeks ago, Olympus and Panasonic announced the Micro Four Thirds specification, which would allow for interchangeable-lens compact cameras with a larger sensor than the nasty tiny and noisy ones used on most compacts. Unfortunately it seems the whole misguided Four Thirds effort is destined to flounder, just as APS did compared to 35mm, despite the undeniable convenience. The 18x13.5mm sensor size has almost half the area of an APS-C sensor and all Four Thirds camera made so far have predictably poor low-light performance.

In a much more promising development, Samsung announced today that since it is finding it very hard to dislodge Canon and Nikon from their top position in DSLRs or even make a dent, they are going to create an entire new segment of professional quality compact cameras using the same APS-C sensors as their DSLRs, and due for 2010. Samsung uses the Pentax lens mount for its DSLRs, and has a long established relationship with Schneider Kreuznach. Pentax makes some very nice pancake lenses that combine high optical quality with small size. The only other company is Olympus, but the 25mm f/2 is saddled with the aforementioned Four Thirds sensor with all the limitations that entails.

At the same time, Thom Hogan has echoed rumors of an APS-C size Coolpix compact from Nikon. It looks like the big camera manufacturers can no longer afford to ignore the pent-up demand for this category, as demonstrated by the brisk sales of the DP1 (No. 49 on Amazon's Digital SLR chart).

r n m restaurant

Sun, 13 Jul 2008 06:39:19 GMT

I have just eaten what is hands-down my best meal of the year at r n m restaurant (their capitalization, not mine), on Haight & Steiner in the Duboce Park/Lower Haight district of San Francisco (not to be confused with the formerly raffish and now utterly commercialized Haight-Ashbury).

The restaurant is named after the chef-owner's father, Robert Miner, a co-founder of Oracle. The food was so good I am almost ready to forgive Oracle for their sleazy extortion tactics...

I started with the Parisian style tuna tartare with waffle chips, microgreens and a quail egg, a very classic dish (and one too often botched by careless chefs), given a little pep with a slight acidity. It was followed by an absolutely outstanding pan-roasted local halibut on ricotta gnocchi with asparagus and morel mushroom ragout, meyer lemon vinaigrette and mâche. The halibut was crisp outside, flaky inside. The ragoût was simply wonderful, a deep, rich and tangy broth, also slightly acidulated, with a generous helping of precious black morels. To top it off, the dessert, a Peach and cherry crisp with home-made blueberry gelato combined two of my favorite summer fruit in an unbeatable combination.

Be advised the parking situation in that neighborhood is particularly nightmarish, even by SF standards. If I had realized they offer valet parking, I wouldn't have had to park halt a mile away (after seeking a place in vain for nearly half an hour).

A contrarian take on Delicious Library 2

Mon, 02 Jun 2008 05:29:40 GMT

On Friday I yielded to the hype, and after cursory testing, I purchased a copy of Delicious Library 2. The clincher was the new feature that allows you to inventory your physical posessions like electronics or cameras, and publish them in HTML format for insurance purposes.

Unfortunately, after some slightly less cursory use of the product, it is deeply unsuited to this purpose. To think I actually upgraded my home Powermac G5 from Tiger to Leopard just to use this software...

For starters, on my dual 2Ghz G5, when running in a window on my secondary 30" monitor, the program is slow as molasses. With a library with only 3 items total, entering data fields is a one character per second tar pit. Moving the window back to the primary 23" monitor helped only a bit.

Secondly, the data model is simplistic. For all practical purposes, gadgets are treated just like books, with some repurposing of fields. The all-important serial number can't even be displayed in column view.

Third, even basic tasks are not handled properly. I have a Symbol CS1504 pocket scanner, which is one quarter the price or size of the Bluetooth scanner Delicious Monster sells, and has a 500 barcode memory, so you can actually use it away from your computer. Using my Python driver, I scanned some books' bar codes, dumped a text file of ISBNs and imported it into DL2. The import mapper allows you to specify which fields of the tab-separated text file go into which field of the DL2 data model. You would expect it to retrieve book detail automatically, but it does not do so. Worse yet, the "retrieve book details from the Internet" menu is grayed out when you select one of the imported books.

There are also some fit-and-finish issues. Right-clicking to get the context menu and selecting the "View in Amazon" option does not do anything. Perhaps this is due to the fact Camino is set to be my default browser, but the other way to view book details on Amazon works (hovering the mouse over the book cover thumbnail, then clicking on the overlaid eye icon that appears when you hover).

On the plus side, the HTML export works quite well, and the loan manager probably does as well, but given the shortcomings of the current version, I would not advise using it unless all you want to manage is books, CDs and DVDs, and you can afford to buy their expensive Bluetooth scanner to use in wireless semi-tethered mode.

Christopher Elbow chocolates

Wed, 14 May 2008 05:32:27 GMT

A few months ago, a new chocolate shop opened in Hayes Valley. Christopher Elbow chocolates is based in Kansas City, not a place that immediately springs to mind when the Great American Chocolate Renaissance is discussed. I had bought some of their products from Cocoa Bella, however, and knew they were good, if pricey.

They sell moderately expensive chocolate bars (the No. 10 41% milk chocolate with hazelnuts is pretty good), drinking chocolate, and bouchéees. The latter are a little too bleeding edge for my taste (spices do not belong in chocolate), but the Bourbon Pecan is to die for, a light and moist, pecan marzipan, almost creamy despite the deliberately roughly chopped texture, and topped with ganache. Not surprisingly, it is usually sold out at the other outlets..

The real draw, as far as I am concerned, is the hot chocolate. Dark, rich, creamy and thick, specially if you ask them to blend it with genuine praline, it is absolutely delicious. You can enjoy it in the twee little salon in the corner of the store before a concert at the nearby Symphony, or shopping in Hayes valley. If you are in the neighborhood, try also Miette Confiserie.

The value of over-the-counter service

Fri, 25 Apr 2008 00:42:05 GMT

My primary computer is a dual 2GHz PowerMac G5 until I can upgrade it with a Nehalem Mac Pro, most likely around the end of the year or early next year. I bought it in 2004, along with a 23" Apple Cinema HD (the old pinstripe plastic bezel kind with an ADC connector). Unfortunately, about a year ago the CCFL backlight on the monitor started turning pink from old age, and thus unusable in a properly color-managed photographic workflow.

I used that as an excuse to splurge on a humongous (and agoraphobia-inducing) HP LP3065 30 inch LCD monitor after reading the glowing reviews. The two features that sold me were the enhanced color gamut (the only way to improve that would be to get a $6000 Samsung XL30, something I am not quite prepared to do), and the fact it has 3 built-in DVI ports, so it can easily be shared by multiple computers (assuming they support dual-link DVI, which unfortunately my basic spec Sun Ultra 40 M2 does not). The fact it was 25% less expensive than the Apple 30" Cinema Display helped, of course.

About 6 months ago, I discovered there was a fine pink vertical line running across the entire height of the monitor, roughly 25 centimeters from the left. Since I primarily use that monitor for photo (the primary monitor for Mail, web browsing or terminals remains the Apple), at first I worried there was a defect with my camera. I managed to reproduce the problem with my MacBook Pro (they have dual-link DVI, unlike lesser laptops), and called HP support (the 3 year HP warranty was also an important consideration when I purchased).

My first support call in November 2007 went well, and the tech told me I would be contacted to arrange for an on-site exchange. This is a seriously heavy monitor and I did not relish the idea of lugging it back to FedEx, so getting premium support for a business-class monitor sounded an attractive proposition. Unfortunately, they never did call back, and as I had other pressing matters to attend to involving international travel, I just put it out of my mind (it is a very subtle flaw that is not even always visible).

I only got around to calling them back a few weeks ago. Unlike in November, I was given the run-around with various customer service reps in India until I was finally routed to a pleasant (and competent) tech in a suburb of Vancouver (the US dollar going in the direction it is, you have to wonder how much longer before HP outsources those call centers back to the US). The problem is not with Indian call centers, in any case, all but one of the CSRs were very polite (I suspect Indians learn more patience as they grow up than pampered Americans or Europeans would). The problem is poorly organized support processes and asinine scripts they are required to go through if they want to keep their jobs. In any case, the Canadian rep managed to find the FRU number and also told me someone would call to schedule an appointment. Someone did call this time, to let me know the part was back-ordered and they would call me when it becomes available.

This morning, as I was heading for the shower, my intercom buzzed. It was a DHL delivery man with the replacement monitor. I had to open the door to him in my bath robe. Naturally, nobody at HP bothered to notify me and had I left earlier, I would have missed him altogether.

One of the great things about Apple products is that if you live near an Apple store, you can just stop by their pretentiously-named Genius bars and get support for free (though not free repairs for out-of-warranty products, obviously). I now have a fully working HP monitor again, so I suppose I can't complain too loudly, but the Apple monitor with the sterling support looks like the true bargain in hindsight.

Backing up is hard to do (right)

Thu, 10 Apr 2008 23:45:47 GMT

You can never overstate the importance of backups. Over the last year I have put quite a bit of effort in making sure my data is backed up properly. The purpose of this article is not to describe backup best practices (that is a vast subject, there are other, better resources available on the web, and in any case there is no one-size-fits-all solution). I am just documenting my setup, the requirements that drove it, and possibly give readers some ideas.

The first part in planning for backup is to do an inventory of the assets you are trying to protect. In my case, in order of priority:

1.5GB of scans of important documents: birth certificates, diplomas, invoices, legal documents, bank statements, and so on. This data is very sensitive, and should be encrypted.
150GB of digital photos and scans
My address book, which lives on my laptop
My source code repositories
My personal email, approximately .75GB
The contents of this website, about 5GB
190GB of music (lossless rips of my CD collection)
My Temboz article database

Thus the total storage capacity required for a full backup is reaching the 400GB mark. This in itself precludes DVD-R or even tape backup (short of buying an expensive LTO-4 tape drive or an autoloader, that is).

The second step is to devise your threat model. In my case, by decreasing order of likelihood:

Human error
Hard drive failure
Software failure (e.g. filesystem corruption)
Silent data loss or corruption, e.g a defective disk
Theft
Fire, earthquake, natural disaster, etc.

Third, some general principles I believe in:

Do not use proprietary backup formats. The best format is plain files on a filesystem identical in structure to the original.
Do not rely on offline media for backups. The watched pot does not boil over, online data is much less likely to go bad without my noticing until it is too late.
A backup plan needs to be effortless to be successful. Plugging in external drives when backups are needed, or rotating drives between home and office is something I have tried, but not stuck to.
Backups should be verified — they should generate positive feedback, so that the absence of feedback can alert to problems
For all types of data, there should be one and only one reference machine that holds the authoritative copy. Multi-master synchronization and replication is possible using tools like Unison, but is much harder to manage and increases the risk of human error.

With these preliminaries out of the way, here is my system:

My primary backups reside on my home server, a Sun Ultra 40 M2 workstation, running Solaris 10. This machine is very quiet, so I can keep it running in the room next to my bedroom without disturbing my sleep. It is also relatively power-efficient at 160W with seven hard drives.
One of the seven drives is the 160GB boot drive, and the other six are 750GB Seagate drives configured in a 3TB ZFS RAID-Z2 storage pool.
With large SATA drives, reconstruction after a drive failure is long and the risk of another drive failing due to the stress of rebuilding is not negligible. RAID-Z2 can tolerate two drives failing, unlike RAID 5 which can only tolerate a single drive failure. This level of data protection is higher than RAID 1 since RAID 1 won't protect you if two drives that are the mirror of one another fail. You can get the same level of protection in RAID 6 or RAID-DP.
I have scripts to take ZFS snapshots daily, equivalent to the auto-snapshot service. The daily snapshots are kept for the current month, then I keep only monthly snapshots. Snapshots are the primary line of defense against human error.
Snapshot technology consumes only as much disk space as required to store the differences between the snapshot and current versions of a file, and is much more efficient than schemes like Apple's Time Machine where a single byte change to a mult0gigabyte file like a Parallels virtual disk image will cause the entire file to be duplicated, wasting storage. Because snapshots are taken near instantly and cost almost nothing, they are an extremely powerful feature of a storage subsystem.
I backup from my various machines to the Sun via rsync over ssh. An incremental backup of my PowerMac G5, which has most of the 400GB in my backup set, takes less than 5 minutes over Gigabit Ethernet, despite the ssh encryption.
ZFS is probably the best filesystem, bar none, but it is not perfect, as demonstrated by the Joyent outage and you still need another copy for backup in case of ZFS corruption.
Every night at 2AM a cron job on my old home server (2x400GB, ZFS RAID 0), that I now I keep at work, pulls updates from the Sun using rsync over ssh (the company firewall won't let me push updates to it from the Sun). Another cron job at 8AM kills any leftover rsync processes, e.g. if there are more data changes to transfer than fit in the 1-2 GB that can be transferred in 6 hours over my relatively pokey 320-512kbps DSL uplink (no thanks to AT&T's benighted refusal to upgrade its tired infrastructure).
My cron jobs use verbose output which generates an email sent back to me. I could suppress those messages, but then I would lose the ability to detect errors.
A last line of defense is to back up my server at work to a D-Link DNS-323 NAS box using rsync over NFS. This cute little unit holds two Western Digital Green Power 1TB drives in RAID 1, which slide right in, no tools required. It consumes next to no power or desk space. Since it runs Linux and is easy to extend using fun-plug, I could conceivably run the cron and rsync from there. As a bonus, the built-in mt-daapd server streams my entire music collection to iTunes over the LAN so I can listen to any of my CDs at work.
It can take a few days for this data bucket brigade to catch up with a particularly intense photo shoot, but it will eventually and is never too far behind. This provides me with near continuous data protection and disaster recovery.

Push recruiting

Thu, 20 Mar 2008 23:33:30 GMT

As I was debugging why feedparser is mangling the GigaOM feed titles, I found this easter egg on the Wordpress hosted site:

zephyr ~>telnet gigaom.com 80
Trying 72.232.101.40...
Connected to gigaom.com.
Escape character is '^]'.
GET /feed HTTP/1.0
Host: gigaom.com

HTTP/1.0 301 Moved Permanently
Vary: Cookie
X-hacker: If you're reading this, you should visit automattic.com/jobs and
apply to join the fun, mention this header.
Location: http://feeds.feedburner.com/ommalik
Content-type: text/html; charset=utf-8
Content-Length: 0
Date: Thu, 20 Mar 2008 23:36:17 GMT
Server: LiteSpeed
Connection: close

Connection closed by foreign host.

Knowing how to issue HTTP requests by hand is one of my litmus tests for a web developer, but I had never thought of using it in this creative way as a recruiting tool...

Macworld 2008 round-up

Wed, 16 Jan 2008 06:29:41 GMT

The MacBook Air was what I was waiting for (I pre-ordered the SSD version just before the online Apple Store buckled under the load). I have a MacBook Pro 15", and because of its weight I end up leaving it at work and not carry it with me at all times (the MacBook is hardly any lighter). Sure, the Air has drastically limited connectivity (the lack of Gigabit Ethernet is probably what I will regret most, even though I clocked my Airport Extreme at 90 true Mbps throughput). Other minuses include the glossy screen (instead of an anti-reflective one), the MacBook-like chiclet keyboard rather than the much nicer MacBook Pro keyboard), or the sealed non user-replaceable battery.

I suspect people deriding it are people whose main machine is a laptop. My main machine is a tower desktop, and no laptop is ever going to compete in terms of capacity and expandability. The drive on the laptop is merely a cache for the desktop where the real data lives. The compromises the Air makes are acceptable ones in exchange for a machine that is light enough for me to carry all the time. I was considering getting an Asus Eee PC prior to the show, and the MacBook Air is a vastly more capable and polyvalent machine.

Apart from that, the show was a relatively quiet one with few truly noteworthy new products. Here are the main highlights:

Matias did not have the Tactilepro 2.0 keyboard on display. I love mine (a version 1 with the ALPS keyswitch) and would like to get a spare, but apparently they have parted ways with the manufacturer of the new Matias-designed keyswitches and are working on a 3.0 version for later this year.
Fujitsu were demonstrating an ultra-small, bus-powered document scanner, the S300M. Unfortunately, once again for reasons due to licensing of the bundled software, they could not release a single SKU that would work with both PCs and Macs.
The German company Project Wizards was demonstrating Merlin, a project management program similar to Microsoft Project. The scheduling and load-leveling algorithms look at least as capable as Project 2000, and they told me the next version will allow team members to report on task advancement by simply contacting a built-in web server. Looks like a promising product.
Samsung showed the CLP-300 which they bill the world's smallest color laser printer. Indeed it looks roughly the same size as my monochrome HP LaserJet 1320, and much smaller than my bulky HP 2605dn, that's quite an achievement. I am wary of Samsung lasers since buying the CLP-500 for Kefta a few years back. The print quality was fine, but it was ludicrously slow, taking something like 5 minutes per color page to print. The CLP-300 seems reasonably fast, faster than the 2605dn at any rate.
Samsung was also showing off the gorgeous XL30 30" LED-backlit LCD monitor. LED backlight is more environmentally friendly, does not shift colors as it ages unlike a TFT backlight, and gives a wider color gamut. Unfortunately, its price is a princely "between $6000 and $7000".
Microsoft was showing off Office 2008, emphasizing ease of use and productivity rather than features for features' sake for a change.

They even set up a bloggers-only salon to curry favor, complete with Internet cafe and snacks.
I tried Nikon's humongous AF-S VR Nikkor 200mm f/2G IF-ED lens. Very heavy but impressive piece of gear.
Canon was showing off the new Flash-based HD camcorders they introduced at CES. They are not that much smaller than the HDV ones. The HV30 replaces the excellent HV20, but the only real improvements are 1080p30 mode and an articulating LCD.

A San Francisco local's advice to Macworld attendees

Mon, 14 Jan 2008 02:38:56 GMT

I have been living and working in downtown San Francisco for almost eight years now. Until a month ago, my office window (right) used to overlook Third Street and the Moscone center. San Francisco is a popular convention destination (one wonders why proctologists seem to prefer it to, say, Detroit) but Macworld Expo is definitely the biggest show in town. Restaurants and hotels are taken by storm, taxis become scarce, traffic gets even snarlier and the lines at Metron eateries cross the threshold of ludicrousness. So here are a few tips for Macworld attendees to have a better time and not caught in tourist traps.

Transportation

Driving in San Francisco is a non-starter. Traffic is horrendous, parking is scarce and you would lose far too much time just getting around. SF Muni is a pretty good public transport system (at least by admittedly paltry US standards) and their 1, 3 or 7 day Passport passes are good value.

Cars are mostly useless inside the city, but nice if you want to drive to make a Fry's run or a day trip to Marin across the Golden Gate. If you must drive, the friendly folks at Reliable Rent-a-Car will give you decent rates on Toyotas. Until I bought a car last month, they were my go-to place for when I needed a car.

Lunch

San Francisco has the best food in the United States, but you wouldn't know if from the overpriced eateries in a three block radius. The Firewood Cafe and Buckhorn Grill in the Metron are actually reasonably decent, but the throngs of convention-goers mean long lines. Mo's Grille has excellent burgers (I recommend the aptly named "Belly Buster"), and since access to it is a little tortuous, you have a fighting chance (it is literally just above the Moscone South).

Ranging a little further, Nova has decent burgers and a lovely lobster quesadilla, and the new Westfield Mall three blocks to the west has a decent food court. Some good local chains are Bistro Burger, S.F. Soup Co. or Café Madeleine (official birthday cake purveyor to Kefta).

That said, the best lunch experience is to take the historic F line streetcar to the Ferry Building Marketplace with its wide variety of gourmet food stores and eateries. I heartily recommend the clam chowder at Ferry Plaza Seafood (it used to be my Friday lunch of choice) or the eclectic fare at Boulette's Larder. Chocolates from Michael Recchiuti or fresh-pressed olive oil from Stonehouse make for great (and edible) souvenirs.

Staying hydrated is important when you expect to spend an entire day on the show floor. There is a Whole Foods store a mere block away where you can buy any required provisions.

Dining

Dining in San Francisco is an embarrassment of riches, it would be a shame to settle for overpriced hotel food. A word to the wise: most of the better places are hooked into the OpenTable reservation system which makes finding a good place with availability a much less hit-and-miss affair. This year Macworld coincides with the annual Dine About Town event where participating restaurants will offer specially discounted menus.

Equipment

Murphy's law will strike at the worst possible moment. If you need help with your Mac, the geniuses at the San Francisco Apple Store (or the smaller Chestnu St or Stonestown locations) can help. It's also good to keep in mind the Apple stores all offer free WiFi connectivity.

If you need commodity spare parts like a USB hub in a hurry, Central Computers is a mere block away and carries a wide assortment, albeit PC-centric.

If you are an attendee and have questions I have not answered, please feel free to email me, my contact info is at the right.

She said Yes

Sun, 14 Oct 2007 02:00:00 GMT

I flew into London today. I took the adorable Shaheen B. to dinner at The Ledbury.

Between courses, I asked her to marry me. She said Yes. The only plausible explanation for this lapse in judgement is temporary insanity. I couldn't be happier.

P.S. the ring is guaranteed to be De Beers-free and not a blood diamond, courtesy of the nice folks at Brilliant Earth, who by the greatest of coincidences are just across the street from my office.

iPhone first impressions

Mon, 09 Jul 2007 19:47:53 GMT

I thought I would escape the frenzy of iPhone hype by filtering out any mentions of it from my feed reader. In fact, I was quite resentful of the way the iPhone launch pushed out the release of OS X 10.5 Leopard to October 2007. On my way to my cousin's wedding on Friday June 29th, I passed by the San Francisco Apple Store and saw the line. It was actually fairly tame, as it only went halfway around the block (when the store originally opened, the line went all the way around and spilled over into Market Street).

Of course, when I came back, I had to see one. One of the petty annoyances with my Nokia E62 was how it would take several seconds for the address book to load. The iPhone, despite having a much more computationally intensive user interface, still manages to have lightning-quick responsiveness to user input. That itself convinced me to buy one.

The iPhone mostly meets or even exceeds the hype. The user interface is exceptionally good, let alone for a version 1.0 product. Some quick notes from a Nokia E62 switcher (my previous phone was also using Cingular/AT&T):

Email and web are very snappy. The SSL implementation on the E62 would take forever to negotiate with my home IMAP server (as in several minutes), the iPhone's is instant. The E62's web browser, despite being based on the same WebKit code base as Apple's Safari, could not run two concurrent AJAX XMLHttpRequest concurrently, Safari has no such problems.
The battery life is very short, well under 2 days, and it takes a long time to fully charge.
The glass screen does not scratch, but it does show fingerprints and smudges.
The virtual keyboard is surprisingly effective. This was the single biggest area where I thought it would fall short, but it actually performs far better than the E62's chiclet keys. Part of the reason is that the E62's keys actually wobble when you press them, which doesn't make for precise typing, and they are so tiny anyway that it's hard to type accurately without pressing other keys in the process. The iPhone's magnification effect as well as the fact you can slide your finger to correct a misregistered virtual keypress, makes for much faster typing. The predictive text engine is also far superior to schemes like Symbian's, or T9. T9 is unbearably annoying in the same vein as Microsoft Word's noxious autocorrect functionality or Clippy, I always disable T9 on any phone that has it, the iPhone's system is unobtrusive and eminently usable in comparison.
The sound quality on the iPhone is not at the same level as the E62, specially for the speakerphone.
No voice recorder. A rather silly omission.
The calendar does not support To-Do list items from iCal. This is ridiculous.
You cannot use iTunes music files as either the ring tone or alarm sound. This was probably to appease AT&T and the RIAA, who seem to believe they have a divine right to make you pay over again and again for the same music. Even if I were prepared to accept their racketeering and pay the obscenely expensive charge for a ring tone, I seriously doubt they would have what I used for mine on the E62, the finale theme from Sibelius' Kullervo op. 9.
The recessed phone jack is incompatible with most earphones like my ER-4P, but it works just fine with B&O A8, whose jack is actually a fairly thin molded connector wrapped in a rubber jacket that easily slips off to accommodate the iPhone jack.
Safari has no option to remember passwords for you, unlike the desktop version, and it does not recognize the standard http://login:password@site/ convention either, which makes logging onto Temboz harder than it has to be.
The Bluetooth functionality in the iPhone is pretty minimal, limited to using Bluetooth wireless headsets and not much more. You cannot beam business cards or photos. Unlike the E62, I cannot use it as a modem for either my MacBook Pro or my Nokia N800. Since there is no SSH client on the iPhone, this could bite me when I need emergency access. Then again, the $20 unlimited data plan for iPhones is half the price of my previous $39.99 unlimited data plan.
Not supporting Java or Flash is a feature, not a bug.
The camera, as could be expected, is mediocre. We all know the only purpose is to snap facsimiles of notes, billboards, flyers. whiteboards and the like.
The calculator is minimal. It does not support RPN and does not have either scientific or financial capabilities.
You can specify 24-hour time format, but there is no way to specify ISO date format.
The iPhone seems incompatible with my SendStation PocketDock Line Out USB, and thus I cannot connect it to my Ray Samuels Hornet pocket headphone amplifier and full-size Sennheiser and AKG headphones. It is also incompatible with Apple's own universal AV dock, and displays a warning message telling you so. Then again, since it is a GSM phone, the annoying pulsating buzz induced by GSM would make such an arrangement impractical.

Update (2007-07-13):

It must be the Friday 13th effect at work... My iPhone seems to have developed a defective proximity sensor. The phone works as a speakerphone, but no longer turns the headset speaker on when I bring it to my ear. Resetting and even restoring the phone does not help, it's probably a hardware issue.

Fortunately, the SF Apple Store Genius Bar let me in this evening without an appointment, and swapped it for a new one. This was the first time they had seen this particular problem, and they told me Apple's policy for the first month is to do full replacements and collect field failures for analysis. The repair process afterwards seems to be still up in the air. I would recommend they have swap or loaner units on hand, as people are less likely to tolerate not having a phone than not being able to listen to music for a week.

MacBook Pro 3G first impressions

Tue, 12 Jun 2007 02:35:19 GMT

I upgraded my MacBook Pro to the third-generation model so I can bump up my RAM to 4GB. Aperture and CS3 are very resource-intensive, and the 2GB upper limit of my first-generation MBP was somewhat constraining.

I just transferred my files over using Apple's migration utility and target firewire mode. The process, while not 100% automated (it did not transfer X11, for instance, or some of the preferences), is far smoother than any Windows equivalent. Here are my first impressions on the new model:

The new, environmentally friendly mercury-free LED backlight is definitely more blue in tone than the pinkish cold-cathode fluorescent backlight on the old model. The default ColorSync monitor profile does a good job of compensating for this, however. There is some vignetting on the 15" screen (darkening in the corners). I wonder how the 17" model fares, and whether they had to add additional LEDs for a more even backlight illumination.
This machine is fast. It blows my dual-2GHz G5 PowerMac out of the water in all benchmarks other than disk I/O. Unsurprisingly, it is also much faster than the first-generation machine, specially on graphics but also on disk I/O.
It does not heat up quite as much as the older Core Duo model, the heat level, while high, never reaches a potentially dangerous temperature. This is probably due to improved power management, as running two Parallels virtual machine will bring it up to the same pant-scorching levels as the Core Duo.

Adobe "Creative" Suite 3, a mixed bag

Fri, 11 May 2007 04:53:02 GMT

I installed Adobe Creative Suite 3 on my home PowerMac and my MacBook (the license allows you to install it on two computers as long as they are not in simultaneous use). The only real reason I upgraded is to get a native Intel version. I have barely started using it already and I already have peeves:

Bridge looks butt-ugly, is even slower than before and with a more amateurish interface than ever
The install procedure is incredibly annoying and Windows-like. There is no justification for an install procedure that chokes if the beta was not uninstalled officially (although I have to give some brownie points due to the fact the cleanup script is written in Python).
The icons are aesthetically bankrupt. What kind of credibility does Adobe think it has with creative people with such an astoundingly lackluster effort?
Barely installed and already in need of software updates. The widespread availability of fast Internet connections is no excuse for shoddy release management or a "we'll patch it post-release" mentality. Speaking of which, the only proper time to interrupt users with a software update dialog is as they are quitting the application, not by getting in the way of whatever task they are trying to get done by starting up the app.
Don't clutter my hard drive with legal drivel in twenty different languages. It's called "Creative Suite", not "Boilerplate Suite".
All the tie-ins to paid add-on services like Adobe Stock Photos or Acrobat Conferencing are incredibly obnoxious, just like those for MSN or .Mac.
JavaScript in Acrobat is a big security and privacy risk, and should be disabled by default.
On the plus side, thanks for making a "Design Basic" edition without all the despicable Flash garbage in it. I would actually pay more for the Basic version than for the supposedly premium one infected with Flash and Dreamweaver.

Update (2008-01-01):

It seems Adobe has also crossed a serious ethical line by building in spyware to track on whenever a user starts a CS3 application.

As far as I am concerned, this is the last straw and I will actively start looking for substitutes for Adobe products as soon as I return from my vacation.

Update (2008-01-02):

It seems Adobe does not collect the serial number after all. The apps should nonetheless never call on the Internet except possibly to check for updates. For people like myself who have static IPs, the IP address itself could be used to correlate the analytics with personal information.

Slava Rostropovich, 1927-2007

Fri, 27 Apr 2007 10:53:43 GMT

Legendary cellist and all-around good guy Mstislav "Slava" Rostropovich passed away in Moscow today. He was a friend and supporter of Shostakovich, Prokofiev, Britten and many others like Dutilleux, and many of the greatest works for cello of the 20th century, indeed of all time, were dedicated to him.

I had the opportunity to hear him conduct Shostakovich's "Babi Yar" a year ago (when I took this photo) and a few years earlier as a cellist the Dvořák Cello Concerto (sadly in replacement of the far superior Shostakovich First Cello Concerto that was dedicated to him).

The world of music has suffered a grievous loss. None of the current generation of cellists (Ma, Gastinel) is of the same caliber. As a conductor, his legacy is more mixed, as his Shostakovich interpretations often lack fire, but his Prokofiev cycle with Erato is marvelous, specially the Fourth in its original version.

Eery similarities

Sat, 07 Apr 2007 20:31:18 GMT

Is it just me or is the Finale of Sibelius' Second Symphony, Op. 43 (1902) eerily reminiscent of the first movement of Saint-Saëns' Third Piano Concerto, Op. 29 (1869)?

Acxiom acquires Kefta

Tue, 03 Apr 2007 17:00:29 GMT

I guess it's official now. Acxiom acquired my company, Kefta, last week. Acxiom is very discreet, but influential company, with a strong technical focus — how many public company CEOs do you know who are listed as inventors on their company's patents? The other founders and myself came to the conclusion a merger will allow us to serve our customers better, ramp up our sales to capitalize on an exploding market and enhance our infrastructure, something that would have beeen much harder if we stayed independent.

Due to confidentiality reasons, I cannot give much more specifics, but Kefta was my first startup. I thought I would have mixed feelings letting it go (I am staying on board, of course, but in a different role now). That has not turned out to be the case, however. We started in 2000, a mere two months before the bottom fell out of the market, and managed not just to survive, but to recover and thrive. At the moment, I am too excited considering the possibilities to wax nostalgic. There might be a few bumps down the road for this blog as I now have to extricate my personal web presence from Kefta's machines (my new hosting platform is a Solaris-powered Joyent accelerator).

Update (2007-05-16):

It seems I was acquired again. Once is good fortune, twice is negligence...

Yet another AppleTV article

Sat, 24 Mar 2007 09:56:33 GMT

Ever since my Panasonic PVR died and I switched to an Elgato EyeTV 250 for my PVR needs, I hardly ever use my 32" Sharp Aquos LCD HDTV, and do most of my watching on my Mac's 23" Cinema HD display.

To rectify this, I purchased an AppleTV yesterday at the San Francisco Apple Store, where they are prominently displayed, hooked up to Sony Bravia LCD TVs. While their choice of TV is questionable (remember, Sony is a four-letter word), the demo is effective for those who did not get to see it at MacWorld Expo 2007.

In all likelihood, I will cancel my Comcast cable subscription in a few days. The only TV shows I watch are:

Battlestar Galactica (iTunes season pass: $34.99)
South Park (iTunes season pass: $23.99)
The Simpsons (not available on iTunes yet)
Family Guy (although the show has become stale and probably on its way out)

I stopped watching live TV seven years ago when I bought my first PVR (a TiVo Series 1). My monthly Comcast bill is $56.20 (basic extended analog cable, no premium channels). Purchasing an iTunes season pass for Galactica and South Park would cost me just slightly more than one month of Comcast's "service". This also means the AppleTV will have paid for itself in less than 6 months (the famous "return on investment" or ROI metric used by IT departments to estimate whether a project is worthwhile or not). The Fox shows I can get over ATSC HDTV because I have an Elgato EyeTV 500 ATSC DTV/HDTV to Firewire tuner (broadcast flag free), and direct line of sight to Sutro Tower, where the San Francisco digital TV over-the-air signals are beamed from.

Of course, the satisfaction of firing the cable company, with its tendency to jack prices up much faster than inflation for ever degrading service, is in itself priceless. As a bonus, the iTunes shows are fully digital, and without ads.

The limiting factor is of course the abysmally slow standard of what passes for broadband in the US. Ironically, I left Europe for California in 2000 because I thought the epicenter of the Internet industry was here, but nowadays the US lags badly behind even formerly dirigiste France in terms of optical broadband and high speed DSL.

Broadband prices are much higher in the US — I pay $70 per month for 2.5 Mbps downstream and 384 kbps upstream, when in France I would get 18 Mbps for half that price (or 70 Mbps for the same price as in the US in the many areas that are getting optical coverage). This is despite the fact my former colleagues at France Telecom face labor costs and Internet transit costs double those of US carriers (the US' central role in terms of connectivity means US carriers can impose peering terms where non-US carriers pay the lion's share of the transoceanic cable costs, even now that Euro or Asian Internet traffic is beginning to eclipse US traffic). The reason for high prices is of course the coddling of the AT&T-Verizon-Comcast oligopoly by a FCC overly influenced by the doctrinaire Chicago School of economics, which refuses to accept even the theoretical possibility of a monopoly...

AppleTV is the second key product in Apple's digital hub strategy, and like the iPod, it is also available to Windows users. Apple did learn from its mistakes in the 1980s, where it lost potential dominance of the desktop PC market to Microsoft by having unrealistically maximalist designs on the market. In some way, this is akin to the virtualization phenomenon shaking corporate IT: like the browser or Parallels, iTunes is another middleware layer that makes the operating system almost irrelevant - Windows users can switch painlessly to the Macintosh, once they realize the elegance and simplicity of the iPod and AppleTV also apply to the Mac and they do not have to settle for the inferior Windows experience.

Now, AppleTV is a semi-closed environment like the iPod. I refuse on principle to buy low-quality, DRM-infested music tracks from the iTunes store. Switching to DRM-infested video tracks from the Apple store is not very consistent. For my defense, I must say:

Unlike music, video is something you see once and usually never again. Thus, the DRM restrictions are less onerous (still outrageous, but less unacceptably so).
There is no legal non-DRM alternative, unlike CDs for music.
Cable companies are really, really evil...

Last but not least, just as you can load your iPod with high-quality, non-DRM music ripped from good old CDs and SACDs, you can load video into iTunes from various sources other than Apple, such as the excellent Elgato EyeTV PVR software, a DVD ripper like Handbrake, podcasts and probably all sorts of other mechanisms in the future (I would be surprised if YouTube did not come out with an AppleTV compatible service soon). Since Apple refuses to license its DRM, that effectively forces other players to use non-DRM video. Who said two wrongs do not make one right?

In any case, I fully expect the AppleTV to be reverse-engineered and alternative operating systems made available for it, just as Rockbox provides FLAC support and gapless playback on the iPod, or how people managed to get Linux running on the original Xbox. Apple is probably not subsidizing the AppleTV the way Microsoft does with its game consoles, so they probably do not have a strong incentive to prevent repurposing with mechanisms like the encrypted boot loader on the Xbox. Less than a week after initial availability, there are already reports of people upgrading the internal hard drive...

Flat-panel HDTVs were the star of the 2006 holiday shopping season, thanks in no small part to free-falling prices. There is now a critical mass of people in the US who are starting to realize just how lousy standard definition TV is, like my friend and colleague Frank who can't bear to watch his TiVo Series 1 any more now that he has a humongous rear-projection 1080p screen, and is mulling building his own MythTV or Freevo box.

The iPod is already a mass-market phenomenon. I believe Apple has a real shot of taking a huge chunk of the cable companies' business away from them. Hollywood will be cheering, because Steve Jobs is one of them, and they can make much more profit from iTunes Store sales than from the crumbs the cable distribution monopolies grant them. Of course, there will be collateral damage like TiVo (not that I would particularly mind), and possibly NetFlix. Presumably Microsoft will do the same by adding equivalent functionality to the Xbox 360. Sony will try, but will fail utterly because of its insistence on polluting everything with proprietary yet unusable pseudo-standards and unredeemably horrid software. All in all, the television industry is in for some mighty interesting times.

Update (2007-03-24 10AM):

I have just cancelled my Comcast subscription. The guy handling the cancellation was actually very friendly, and we talked a little about South Park, TiVo, digital TVs and DVR options. They did not put any hurdles or unnecessary hoops to jump through in the cancellation process, you have to grant them that. Contrast this with scumbags like AOL who have been repeatedly been slammed by state attorney-generals for fraudulently keeping on charging users after cancellation. The cable company's pricing policies may be evil, but their customer service seems pretty good.

Glimpsed at Chicago O'Hare airport

Thu, 22 Mar 2007 17:09:08 GMT

Is Vista a piece of unalloyed garbage?

Sun, 18 Mar 2007 07:14:41 GMT

As far as I can see, the answer is yes.

About a month ago, my two-year old ~~Windows PC~~ game machine started crashing every two minutes in NWN2. This proved the last straw, and I decided to upgrade. One of the games I have, but seldom play is Oblivion, which is graphically gorgeous, but chokes on anything but the most powerful hardware at ordinary resolutions, let alone my Apple Cinema Display HD's 1920x1200, and cutting-edge video cards are no longer available for the AGP bus in any case.

I looked around for packaged solutions from systems integrators, specialized gaming PC companies like AlienWare, and Dell. Contrary to conventional wisdom, it is still much cheaper to build a PC from components than to buy one from a major vendor, $1500 vs. $2500 minimum. Part of the reason is that the vendors flag anyone wanting the absolute best video card as a "cost is no object" customer, add all sorts of expensive components that make no sense in a machine that will only ever be used for games, like fancy DVD burners or flash card readers to jack up the profit margins. As if anyone in his right mind would use a Windows computer for serious work like digital photography...

My configuration is the following: a relatively quiet Antec Sonata II case, an Abit KN9 Ultra motherboard, an AMD Athlon x2 5200, 2GB of Kingston DDR-800 RAM, a humongous nVidia GeForce 8800GTX video card, a 500GB hard drive and a basic DVD-ROM drive.

When it came to choosing the OS, after much trepidation I opted for Vista Home Premium because the 8800GTX is one of the few cards that support DirectX 10, which is a Vista-only feature. I knew Vista would embezzle half the processing power of one core in DRM code that is actually working against my interests, but then again nobody in his right mind would use DRM-ed formats, whether Microsoft or otherwise, to store their music library, so the damage would be limited. Also, Vista comes with "downgrade rights" which allow you to legally install the previous version of Windows.

Vista comes in an attractive copper-colored DVD that is actually quite elegant. Its color scheme is also far superior to the molten Play-Skool set monstrosity that is XP. When I started the Vista installer, I was pleasantly surprised by how quickly it dealt with hard drive formatting (the previous Windows I installed myself is Windows 2000, which will insist on a time-consuming full format instead of the quick format used by the XP or Vista installers). The good impression lasted for all of five minutes. After the inevitable restart to complete installation, the screen promptly dissolved into a scrambled red-and-white screen of doom (I did glimpse a blue screen of death shortly before it rebooted). The diagnostics were completely unhelpful, as could be expected. When the operating system cannot even install itself, you have got to wonder...

Dejectedly, I fished out a Windows XP install DVD. it would not accept the Vista serial number. So much for downgrade rights. Of course, since the package was now opened, no hope for a refund either. I ended up buying a copy of Windows XP, which installed without a hitch. Of course, I still had to install the video drivers, but it did not crash half-way through the install procedure. And Oblivion is now playable without agonizing stutters every two paces.

The 8800GTX is very recent hardware, which did not even have non-beta Vista drivers when I installed it, so I could understand the OS falling back to SVGA mode. There are no other really exotic components here, certainly nothing than XP SP2 could not deal with and therefore Vista should as well. The machine is also well within the recommended minimum configuration (although some experts now advise 4GB of RAM as a realistic minimum for Vista). Crashing during install, when a five year old OS like XP handles it just fine, is simply unacceptable in my book. Even Solaris 10 Update 3, an OS notorious for its limited hardware support, installed without a hitch. Despite the ten man-millennia Microsoft invested in this lemon, they apparently could not be bothered to test the installer.

Conclusion: unless you buy a computer with Vista pre-installed, avoid it like the plague until SP1 is out, just like Intel.

Post scriptum:

Actually, I would not even recommend a PC with Vista preinstalled, as it has terrible backward compatibility. It will not run Office 2000, which is what my company has, for instance. Joel Spolsky has an excellent article on how the new, bloatedly bureaucratic Microsoft lost its way by sacrificing backward compatibility on the altar of useless marketectures. Perhaps they are just trying to force-upgrade people to Office 2007. They should beware: unlike 2002, people have credible alternatives now.

Update (2007-08-30):

The paper about how Vista eats up CPU on DRM has been criticized by the generally reliable George You. My point about the inability to even install on a modern machine that XP has no problems with remains. In any case, having the operating system constantly eat up CPU on tasks I do not want it to, whether it is 7% or 100% of one core, is still morally no better than a parasitic botnet.