Dianna Wynne Jones: Dark Lord of Derkholm

The central premise of Dark Lord of Derkholm seems like such a natural hook on which to hang a comic fantasy that I’m surprised it hasn’t been done to death: there’s a big market for people who want to play at being a Frodo-style hero, triumphing over fearsome evil against long odds, so generic fantasylands hire themselves out for tours that provide mock heroic experiences.

Diana Wynne Jones is no run-of-the-mill comic fantasist, however, and while Dark Lord of Derkholm pokes some gentle fun at Lord of the Rings and its endless imitators, there’s much more going on here than parodying the standard episodic heroic fantasy. Jones’ characters are emotionally complex and we meet them in medias res with a lifetime’s worth of experiencing — notably including assorted rivalries and resentments — under their belts. Rather than let the reader sketch in the backdrop of his or her favorite fantasy novel, Jones provides a complex milieu that has its own unique personality, despite nods to some of the familiar genre tropes.

I enjoyed it quite a bit, but arguably there’s a little too much going on for a single volume. Post-Tolkein fantasies are often criticized for being padded with excess verbiage (after all, the gods of marketing decreed that they must all be trilogies at minimum). In contrast, Dark Lord of Derkholm often feels compressed, with perhaps a little too much elided. The book might have been stronger if Jones had juggled fewer balls — dragons & elves & dwarves & demons & griffins & horse-lords, oh my! And that’s just the mise en scène, the actual story involves court intrigue & rebellious adolescents with image issues & derring-do & temple intrigue & longtime married couples growing apart & parallel universes & … well, you get the idea.

I can’t decide whether Dark Lord of Derkholm would have been strongest cut to a single shorter volume, or expanded into two (I’m not sure there’s a natural breaking point, for one thing). Either way, I think it’s ill served by its title, which makes it sound much sillier than it is. I’d recommend it without hesitation to those who are already fans of Jones, but I think I’d still steer newcomers to Hexwood, or perhaps Howl’s Moving Castle.

Needs More Demons? Absolutely not.

Delia Sherman: Changeling

I enjoyed Delia Sherman’s young-adult fantasy Changeling quite a bit. It’s the story of Neef, who was kidnapped from the mortal world at birth to dwell in the fantastic “New York Between,” and raised as a sort of second-class citizen of Faerie. This is perhaps tired territory, but Sherman manages neat twists on some very hackneyed tropes. One element in Changeling‘s favor is that its faerie denizens draw from multiple mythologies and folklores (there’s a helpful glossary at the back). The faeries of Changeling are also not all sweetness and light; many of them are indifferent or actively hostile to mortal concerns, which feels more true to their folkloric origins than many modern interpretations (particularly those aimed at younger readers). I also appreciated the explicit nods to Kay Thompson’s Eloise, Kenneth Grahame’s The Wind in the Willows, and A Midsummer Night’s Dream (some of whose characters have cameos) as well as the implicit reference to E.L. Konigsburg’s From the Mixed-up Files of Mrs. Basil E. Frankweiler.

Neef’s literal status as a changeling can certainly be read as a metaphor for the sense of alienation and not-fitting in that every adolescent undergoes. The book worked best for me when that was the only obvious metaphor at work — the fashion vampires of New York’s theatre district and, particularly,the Dragon of Wall Street felt forced in comparison to Sherman’s otherworldly versions of Central Park and the New York Harbor.

But even though I thought the novel’s final third flagged a bit, on the whole I found it fast-moving, funny, and surprisingly fresh.

Needs More Demons? Not a bit of it.

Justine Larbalestier: Magic Lessons

I think it would probably occur to me to compare and contrast the first two volumes of Larbalestier’s “Magic or Madness” trilogy with the first two books of Scott Westerfeld’s “Midnighters” trilogy even if I didn’t know the two authors were partners. Many novels feature teenage protagonists simultaneously blessed and cursed with special powers, but Larbalestier and Westerfeld’s systems of magic evince a rare degree of both originality and logic. (They also jointly remind me of Alan Moore’s rigorous extrapolations of superpowers in works like Swamp Thing and Miracleman, and Steven Gould’s hard-nosed explorations of a special power in Jumper and Reflex.)

Magic Lessons continues the story of Reason Cansino as she grapples with the consequences of her new-found abilitites. I was braced for a let-down when I started the book. Part of the pleasure of the first volume ( Magic or Madness) was in puzzling out how Larbalestier’s system of magic works along with Reason, and I expected the second volume to be less surprising on those terms. Even the title seemed a bit lackluster. I had similar misgivings when I started Touching Darkness, the second of Westerfeld’s “Midnighters” books, and in both cases they were completely unfounded.

Magic Lessons makes it immediately clear that it’s not a sequel-as-afterthought, and that there are plenty of additional surprises in store. It starts, quite literally, with a bang, as mysterious forces lay siege from across the globe to the back door of a witch’s house in Sydney, and it never really lets up. Magic Lessons kept me wide awake on two unturbulent airplane flights — no mean feat, because climbing through the troposphere usually puts me out like a snuffed candle. Magic Lessons doesn’t end on a cliff hanger — it’s a satisfying read on its own. But I still can’t wait for the next one.

Needs More Demons? No. Amply supplied with demons.

Maureen Johnson: Devilish

Maureen Johnson’s Devilish commanded my attention as soon as I heard first of it (via Westerblog, of course). The potent combo of demonic subject matter, a Providence RI setting, and a cover that evokes one of my favorite Penelope Houston albums added up to a heaping helping of positive associations and I requested Devilish from the library tout de suite.

comparison of covers of Maureen Johnson's book Devilish and Penelope Houston's album Cut You

I found several little things to like a lot, like Johnson’s favoring of elegant, witty descriptions of things like clothes and sports cars instead of the more common brand-name dropping. I also appreciated protagonist Jane Jarvis’s refreshingly pragmatic response to purportedly supernatural goings-on. I’m definitely interested in reading more from Johnson.

This is apparently Johnson’s first foray into fantasy genre fiction, though, and it shows a bit. The novel opens with a prologue that suggests a combination of Bradbury’s The Halloween Tree and McKay’s Little Nemo in Slumberland, but it doesn’t quite live up to that promise, partly because the prologue gives away too many of the book’s best surprises to an attentive reader. And one of the main drivers of Devilish‘s plot is fundamentally a deal-with-the-Devil story, and that’s very well-worn territory for any fantasy/horror reader. Johnson strives for an original take on the trope, but some readers may find her gimmick (because there’s always a gimmick in deal-with-the-Devil stories) a little hard to swallow. Fortunately, it’s also a story about the parameters and limits of friendship, and it works much better on that level.

Needs More Demons? Yes. Or it needs the demons it already has to be a little more convincingly, you know, demonic.

Diana Peterfreund: Secret Society Girl

I’ve been on such a major Scott Westerfeld kick for most of this year that not only am I reading everything of his I can get my hands on, I’m subscribed to the Westerblog and I read some of the other young adult books he talks up there, too. Here’s one:

Diana Peterfreund’s debut novel Secret Society Girl wasn’t quite what I was expecting. I knew going in that it involved a very thinly-disguised Yale and a secret society clearly modeled on Skull & Bones. I didn’t expect it to involve anything supernatural per se, but partly because it was commended to my attention by a (mostly) Sci-Fi writer, and partly because of certain events in the book itself, I expected it to be somewhat less naturalistic and more conspiracy-oriented — a little more “Secret” and a little less “Society.”

Secret Society Girl had two major problems. I’m going to try to side-step spoilers in discussing them. The first is that that future events are often obvious to the reader long before they happen to protagonist Amy Haskel. In fact, one of the revelations planned for the next book (Secret Society Girl is plainly set up as a first-in-series) is telegraphed so thoroughly that’s hard to imagine that even Haskel will really be surprised when the penny drops. That has the side effect of making the book’s first third or so feel a bit sluggish, because it’s devoid of any real suspense.

Things improve considerably when the protagonist’s primary obstacle is finally introduced. The novel turns unabashedly feminist without being the least bit didactic. Unfortunately the dénoument is a bit of a letdown — a little too deus ex machina.

To balance my criticisms, I should point out that Peterfreund has a good handle on her authorial voice, a nice gimmick for chapter headings (each one starts with “I confess: …”) and the requisite ear for natural-feeling dialogue. There’s one sequence that struck me as jarringly over-written, but only one. And it wouldn’t surprise me a bit if Secret Society Girl gets optioned for film, or even made into one, in which case I can stop feeling guilty for lobbing a few pebbles at it.

Needs More Demons? Definitely, even if they’re not exactly literal. Haskel deserves more significant adversity to triumph over next time.

Stanislaw Lem: Mortal Engines

Stanislaw Lem is one of the many authors I’ve always meant to read something by. I’ve even picked up a handful of his books over the years with noble intentions of follow-through which have, to-date, gone unfufilled. So picking Lem’s Mortal Engine from the freebie box I’d commited to availing myself of only if I really read the books was a moderately acid test of my resolve. About as acetic, I’d say, as Coca-Cola syrup.

The stories in Mortal Engines can be be grouped into two rough categories. Many of them are fables of societies peopled exclusively by robots. Lem’s machine minds regard organic life as a bizarre and terrifying aberration, but they are themselves the polar opposties of Asimovian automatons: they’re more often ruled by emotion than logic. In spirit, Lem’s fables reminded me far more of Rabelais, Cervantes, and Swift than most science fiction: the societies he depicts are often pre-industrial, if not downright feudal, with “electro-knights” levelling “cyber-lances” at one another in duels of honor. The most striking attributes of these stories are Lem’s ability to look at humanity from an outside and contemptuous perspective (not unlike Swift’s Houyhnhnms) and the strangeness of his imagination: not only does Lem boldly modify virtually any noun with “cyber,” “electro” and myriad variants; but his inorganic life proliferates through the universe in a dizzying array of forms: intelligent crystals, gaseous beings, consciousness spread throughout cities.

Lem’s second mode is a much more conventional SF blend of action-oriented narrative with philosophical overtones. In one of the stories, his recurring space pilot character Pirx confronts a mining robot which runs amuck at a lunar base. Pirx worries not just about surviving encounter, but the moral dimensions of his struggle with the robot. Although these stories were supposedly more realistic, I found it harder to suspend disbelief in them as a 21st century reader , because the resources available to Lem’s moon colonists and other future denizens were often ludicrously anachronistic. It’s a fundamentally unfair criticism of older work, but I’m often unable to overlook what appear (from my vantage point, not Lem’s) to be obvious factual errors.

Needs More Demons? No.

John Mortimer: Charade

When I last visited Lorem Ipsum Books, they had a deal wherein for every x dollars one spent, one got to pick a book from the “free books” box. I told myself that I would only let myself take free books if I actually read them. Here, teacher, is my attempt to prove that I did so … and not just the Precipice Notes, neither.

I picked up John Mortimer’s Charade thanks to a case of mistaken identity: I confused Mortimer (for reasons I can’t satisfactorily explain) with Michael Innes (also a British novelist with a capital “M” in his name, I suppose). I didn’t much care for Charade. It didn’t seem to know whether it wanted to be gentle satire, a comic romp, or a serious coming-of-age novel. I’ve often find this sort of confusion charming, but Charade didn’t muster sufficient conviction in any one dimension. It struck me, though, that Charade, published in 1947, could well have been an influence on several things I liked much better, most notably Monty Python’s singular vision of the British Army, and the crazed film director Eli Cross, brought to such memorable life by Peter O’Toole in The Stunt Man.

After that rousing endorsement, I’m sure you’ll all be delighted to learn that Charade is currently in my Bookmooch inventory.

Needs More Demons? Couldn’t hurt.

Michael Shea: The Incompleat Nifft

Once upon a time (in the 1940s), Mssrs deCamp and Pratt teamed up to write a series of short novels about the magical misadventures of one Harold Shea. The tales had a proto-post-modern spin to them: Shea would get transported into myths and pre-copyright stories like Spenser’s Faerie Queene. The Shea stories have an absurdly complicated publication history which found them in omnibus editions with titles like The Compleat Enchanter and The Incompleat Enchanter.

A month or so ago I was flipping through a book rack and spotted The Incompleat Nifft by Michael Shea. My brain registered the variant spelling of “incomplete” and the presence of “Shea” and I checked the book out of the library under the presumption that it was some sort of latter-day sequel. (I’ve made similar mistakes before — once I brought home one of Robert Parker’s novels about the detective Spencer when I’d been looking for one of Donald Westlake’s novels about the detective Parker. Oops.)

In fact, Shea’s collection of loosely-linked long short stories/short novels bears no relation to deCamp and Pratt’s creation, but evinces instead a remarkable resemblance to another fantasy series which originated in the 1940s, Fritz Leiber’s Fafhrd and the Gray Mouser stories.

Fafhrd and the Gray Mouser are two of the original cornerstones on which the “sword & sorcery” subgenre of fantasy was built (for better or worse). They are best appreciated today with a liberal dose of historical perspective. Leiber’s duo encounters tropes that have long since become dreadful “Dungeons & Dragons” clichés (the fabled temple treasure that turns out to be a monster, the mysterious shop with mysterious goods which mysteriously appears and vanishes, etc), but which were fresh when the stories were first published. Fafhrd and the Gray Mouser were a bit like the French Connection car-chase scene of sword & sorcery, if you’ll let me stress a metaphor. The stories were short on character development, thematic content, and even on sophisticated plot constructs. They were long on dense prose with a rather 19th-century quality — “lush” or a bit “purple,” depending on your disposition to it. Arguably, the central “character” of the stories was really Leiber’s imagined world of Newhon itself, which was richly and vividly imagined.

Michael Shea’s Nifft the Lean and Barnar Hammer-Hand follow this template in virtually all particulars. Shea mixes in a touch of Lovecraft’s xenophobia, and is more explicitly grisly than Leiber, and there are sadomasochistic overtones, particularly when Nifft and Barnar go to Hell (which they do with some regularity). Shea is perhaps a better prose stylist than Leiber, which I think is the root of my strange reaction to the book. Like a “real” work of literature it takes more effort to read than most books which offer simple escapism. But I don’t think it offers commensurate rewards. Its characters are too shallow to offer any insights into human nature, and what thematic content it holds can be reduced to truisms like “pride goeth before a fall.” There’s nothing to learn from it but the details of a geography that is exotic, but wholly imaginary.

I found myself resenting not only the time I wasted reading (roughly half of) the book, but the time that Shea wasted writing it. He has an undeniable ability to write evocative prose and conjure memorable (if frequently nasty) images, and it seems to me that he ought to be able to write much better (and less derivative) books.

What bothers me about my response to The Incompleat Nifft is that I find myself making some of the same arguments that people make against the worthiness of fantasy/science fiction genres in their entirety. It has nothing to teach us. It’s decadent and valueless, a waste of time.

And I don’t feel the same way about the Fafhrd and Gray Mouser stories themselves. The same arguments apply, but Leiber was at least doing something that hadn’t been done before. I think what offends me about The Incompleat Nifft is the combination of its extreme derivativeness from a single template; the narrowness of its literary scope; and the amount of craft, care, and talent expended in its construction.

Which leads me to the odd conclusion that if The Incompleat Nifft had been written less well, I might have been able to enjoy it as a simple escapist lark. At any rate, I might not have felt compelled to excoriate it at (probably tedious, sorry) length.

Needs More Demons? Needs more something. Actually has a lot of demons.

Malcolm Gladwell: Blink

[editorial note: this review/essay/whatever was originally published as three separate entities over the course of a month.]

surprise benefits of pseudo-vegetarianism

I’ve been reading Malcolm Gladwell’s Blink in fits and starts over the past two months — it’s on the library’s short-term loan list, so I request it, read as much as I can before it’s due, return it, and repeat. I don’t think it’s a bad way to read such an information-dense book; it provides opportunities to digest and reflect on Gladwell’s theses.

I don’t think he delivers on the implicit promise that has made his books bestsellers among business readers. The Tipping Point provides tools for understanding why some messages — like teen anti-smoking campaigns — don’t “stick.” But it doesn’t provide tools for making messages stick. I think that’s because societies’ response to stimuli is fundamentally chaotic. Ensuring any particular meme spreads is impossible. Even Steven Spielberg directed an unequivocal flop once.*

Blink suffers from a similar problem: it identifies situations in which rapid intuitive assessments — “thin-slicing,” in Gladwall’s parlance — are invaluable, and other situations in which they’re extremely harmful. It doesn’t provide foolproof guidelines for distinguishing “good” thin-slicing from “bad.” Again, I don’t think it’s a soluble problem.

I’m not an expert on cognition; I’m a lay person with probably just enough information to be dangerous. But I think a major component of what makes for human intelligence is that our brains are abstract pattern-recognition machines. The engine that recognizes individual human faces is the same engine that sees animal shapes in clouds and inkblots. I think it’s always going to be subject to errors, particularly in high-stakes situations that require snap judgments: “He’s drawing a gun!” versus “He’s pulling out his wallet.”

Even if I don’t think Gladwell’s books quite live up to their hype, they’re informative, provocative, fascinating, and lucidly written.

For instance, his account of Sheena Iyengar’s research on consumer choice provided insight into something that’s intrigued me for the past decade. Iyengar found that customers given an opportunity to taste 6 jams in a store were far more likely to make a purchase than customers who had a chance to taste 24 different jams.

I’m a pseudo-vegetarian. This generally makes dining out straightforward: most of the menu is automatically excluded from consideration. I usually pick from the small set of available options rapidly and without much conscious deliberation. When I dine at a vegetarian or seafood specialty restaurant, I have a larger field to winnow. My selection process is radically different (and much slower). I typically try to find the entrée that maximizes features I like: the one with the ginger, tofu, and straw mushrooms. Sometimes I experience a kind of stress that’s unusual for me: no dish has the poblano pepper sauce, guacamole, and melted jack cheese; I can only get different combinations of two of those ingredients. Then I feel vaguely dissatisfied with a meal that I would unhesitatingly and happily choose if I had fewer options.

Iyengar’s research suggests that this behavior isn’t just me-being-weird. Gladwell’s synthesis provides a framework for understanding it: I “thin-slice” among a few choices, but not among a dozen.

*Of course, Gladwell has certainly “tipped” his own books, so maybe, just maybe, he knows something about hidden marketing levers that he’s not sharing.

the warren harding error error

In Blink, Gladwell devotes a chapter to exploring what he calls the “Warren Harding error.” He contends that the primary reason for Harding’s political success was that the man looked presidential.

Gladwell doesn’t apply this line of reasoning to politicians of the current era (although later he does quote Paul Ekman — who, with Wallace Friesen, assembled the “Facial Action Coding System — claiming that in 1992 he saw Clinton’s tendency for marital indiscretions literally written on his face.)

Whatever I thought of his policies or the abilities he brought to the job, I think I have to concede that Ronald Reagan looked presidential (at least some of the time). He was certainly always too much the gunslinger for my taste. But he could be dignified without entirely losing the humanizing mischievous twinkle in his eyes. If he’d been an actor cast in the role of the president, I think I could have bought it.

The real mystery is the election — twice, yet — of George Walker Bush. The presidential debates of 2004 crystallized this for me. John Kerry with his imposing height and resonant voice, looked and sounded presidential. His opponent looked like a used-car salesman by comparison: shifty-eyed, almost sneering, his voice often distinctly petulant if not actually whining.

And yet he won. Where are you now, oh Warren Harding error? Come back. We need you.

In other news, I took a few of the Implicit Association Tests Gladwell describes in the same chapter (it’s essentially the “be careful about judging books by their covers” segment of the book). Gladwell (and Greenwald, Banaji and Nosek, who developed the tool) claim that the test design is effective even when you know you’re being tested (unlike many sociological tests).

I’m not convinced. I took a test designed to identify an “implicit association” (e.g., an ingrained unconscious bias, more or less) for males/sciences and females/liberal arts. I was prompted by the survey I took beforehand to think fleetingly of famous scientists like Ada Lovelace and Marie Curie, and famous creative types like Julio Cortàazar and Pablo Picasso. My biggest problem was that every time I was shown the words “history” and “philosophy” I had to consciously think “soft science? or liberal art?” But taking the test to the best of my ability still produced outlying data.

Then I took a test to identify implicit associations between ethnic groups and positive and negative concepts. When I was told I was supposed to associate images of caucasian men with negative concepts and images of black men with positive concepts, I muttered “black, good; white, evil” under my breath. No sweat.

deli slices of security

I was initially critical of Malcolm Gladwell’s Blink for not delivering on its implied promises, but I’ve revised my opinion of it substantially. It’s had a real impact on the way I think about certain types of situations. I still don’t think it provides a foolproof method for applying its principles, but it does offer tools for identifying problematic patterns in processes. As one example, it provides a framework for examining my misgivings about approaches to security in the post-September 2001 United States.

The administration argues that the lack of major terrorist incidents within the US demonstrates the effectiveness of the Homeland Security and Transportation Safety initiatives. This argument is obviously specious. The lack of a major incident in the first half of 2001 scarcely proved that the US was well-protected from a terrorist attack in the second half of the year. And the penetration of the new system by the “shoe bomber” and razor-blade-toting blog readers (for example) makes a strong case that the new system is not necessarily more effective at threat identification than the old system.

Back when the major concerns of airport security were preventing the influx of drugs and illegal (but peaceable) aliens, I was involved with a competitive bid to develop training for the Immigration and Naturalization Service. As part of the effort, members of our team accompanied INS personnel on airport security details and took some of the courses given to the agents. (For the record: all of the material I was exposed to was unclassified.) It was obvious that the most effective agents relied heavily on the sort of intuitive assessments Gladwell describes in Blink. In particular, they were very good at identifying people who had something to hide. Other people have written about the hazards of inexperienced personnel and over-reliance on trickable technology. But I wonder: does a process that makes all passengers nervous and uncomfortable make it fundamentally easier for people with malicious intent to slip through?

As part of my ongoing research on improving MBTA usability, I’ve been listening to the chatter between MBTA dispatchers, bus drivers, train operators, station managers, and other staff. Shortly before Christmas, toward the end of evening rush hour, I heard an exchange that that went like this:

We have an incident of an unattended package that has been sighted on the east platform of [station name].

About half a minute later, I heard the following reply:

A passenger forgot her package. She’s on her way back to the platform to retrieve it now. Please just let her get her bag.

In Gladwell’s parlance, I felt that I had ample opportunity to “thin slice” the conversation. The first speaker was officious, with a pseudo-military quality that verged on pompous. He used the passive voice and awkward, redundant, and jargon-y terminology.

The second speaker was clearly fed up with the first speaker. I had the distinct impression it wasn’t the first such conversation. The tone of voice — and the word “please” — suggested that the speaker thought it was unlikely that the woman would be allowed to get her bag back without additional hassle.

The second speaker had a good opportunity to make a realistic assessment of how likely the passenger was to pose a terrorist threat. The second speaker implied face-to-face contact with the passenger — who was probably cramming in last-minute shopping on the way home from work, and carrying one package too many. The first speaker was making decisions on the basis of a blurry picture on a monitor and (I suspect) a procedural manual revised in the wake of September 2001.

I’ve spent much of my career working on training products for state and federal agencies, and I think it’s likely that the new rule book specifies that any unattended package must got through the full threat evaluation procedure, no matter what the station manager recommends. After all, there’s always a chance that the station manager has somehow been coerced into making a false statement.

The problem is, this approach just doesn’t work. Being on high-alert forever is the same as not being on alert at all — people aren’t wired to maintain peak vigilance indefinitely. Procedures that are excessively cumbersome will eventually be disregarded. And while I understand that discounting the judgment of those closest to a potential threat situation may protect the MBTA from liability, I’m far from convinced that it’s the best way to actually increase the overall safety of the system.

Needs More Demons? No. I’m not even going to make a corny joke about devils in details.

Steve Squyres: Roving Mars

You could be excused for thinking that Roving Mars: Spirit, Opportunity, and the Exploration of the Red Planet is a science book. It’s got a Martian landscape on the front cover, and the author was the “Principal Investigator” of the projects it chronicles. If you’re not careful, you might even learn a little bit about geology.

Mostly, though, Roving Mars is a book about project management. Squyres often speaks, somewhat disconcertingly, about “doing science” as if science is merely a product of having assets correctly positioned, in the same way that a movie’s revenue is the product of having copies of the film in theatres. He admits that, from his perspective, one of the critical goals of the Spirit and Opportunity missions was to justify more Mars missions, in the same way a succesful product generates more demand in the marketplace.

Much of the ground Squyres covers will be familiar to anyone who’s manged a difficult project (perhaps especially a software development effort). He covers intial brainstorming; marketing and proposal development; forming strategic alliances with competitors; the struggle for budgetary, schedule, and manpower resources; risk mitigation strategies; motivational techniques; benefits and drawbacks of delegation and outsourcing; troubleshooting and quality assurance; and aproaches to consensus-building and fostering effective decision-making. It’s a fast and engaging read. Several chapters are written in the form of Squyres’ journal entries, which gives it a “you are there,” sort of immediacy. For a book about project management, it’s often surprisingly suspenseful and moving, and Squyres’ “boldly go where no one has gone before”-style enthusiasm is palpable.

Throughout he makes a solid case for his own talents as a manager (despite his penchant for tantrums). And throughout he reinforces my growing sense that there is something fundamentally and systemically wrong with the current best-practice management of complex engineering development efforts.

The Mars rover project is repeatedly stymied by mistakes that simply shouldn’t be made: instruments designed to work sideways but not upright, confusion between English and metric units, pieces that are fabricated to the wrong size. It’s perhaps especially disheartening to compare these errors to the highly-publicized mistakes NASA has made in recent history, from grinding the Hubble’s mirror to the wrong spec to the material science failures that cost the lives of space shuttle astronauts.

Also disturbing — but eerily familiar to me — was the degree to which the developers of the Mars rover software were unable to predict its behavior. I was shocked by how frequently the rover team was faced by problems I’ve faced with notoriously buggy commercial software. Computer that crashes as soon as it boots up? Been there, fixed that. Corrupted flash memory? Ate my second cellphone alive.

I’m convinced that the issue isn’t stupidity or incompetence on the part of the team, not just because these folks have high-falutin’ degrees in their fields, but also because every smart team I’ve had a chance to observe or directly work with — including some folks who made me feel positively dim — has made similarly obvious mistakes on sufficiently complex projects. On the biggest projects I’ve been associated with, it was sometimes painfully obvious that no single person understood the whole requirements document. I once saw a data entity diagram that covered a large conference room wall from floor to ceiling. I saw team members literally start sobbing when it became evident that fundamental assumptions underlying that diagram — which represented over a year of work and several million dollars — had never been valid.

I’ve begun to think of it as a big picture/little picture problem. When teams are stovepiped, each group can do its “little-picture” work and check and resolve its internal errors. On small, well-characterized projects, group leaders can grasp the “big picture” at a level of detail that permits identification and resolution of problems that cross group lines. But on projects that are bigger and more uncertain, it becomes impossible for anyone to grasp the gestalt of the project at a sufficient level of detail. Things start to slip through the cracks.

Since Malcolm Gladwell’s books — particularly The Tipping Point — have had more influence on my thinking than any others in a decade or so, I’m inclined to wonder if large engineering projects are being constrained by the fundamental limits of human cognition. I’m even tempted to wonder if Gladwell’s “magic number” 150 might crop up somewhere in a calculation of maximum manageable size.

I don’t think the problem is insoluble, but I think it calls for new techniques for asserting correctness. There are mathematical methods for “proving” the correctness of software. They’re seldom applied in the real world, partly because they’re cumbersome and expensive, but also, I think, because they rely on not changing requirements during development. I argue that since no one ever understands the requirements for complex projects, it’s almost inevitable that the requirements will change when one or more deficencies are identified midstream. My anecdotal experience suggests strongly that many serious engineering errors arise from failure to understand the consequences of a requirements change during the development cycle.

The engineering development process of the future should attack this problem from three angles:

  • The requirements definition phase must systemically address the inability of humans to fully characterize the behavior of extremely complex systems.
  • Throughout the development cycle it must embody consistency checks that prevent errors of the English/metric variety
  • Throughout the development cycle it must explicitly maintain the constraints on its own behavior, so that flaws resulting from requirements changes are immediately evident.
    (Software often has implicit constraints, e.g., it only works if only one document is open. Currently, information about these constraints may only exist in the mind of a single developer.)

Two other takeaways from Roving Mars:

  • Good golly, rocket scientists drink more than I would have guessed.
  • Wow, a lot of Mars probes have just flat out disappeared. Some enterprising sci-fi writer ought to be able to get at least a short story out of the conceit that the Martians shoot down any probe that gets too close to their cities, and play games keeping just out of camera range of the ones they allow to land.

Needs More Demons? No, Squyres’ project is plenty bedevilled.