Contents An Essay Of Sorts: Reflections on "Programming Under The Wizard's Spell."

Jerry Pournelle

read book now





Saturday, July 25, 1998

Revisions and additions:

Saturday, April 01, 2000

read book now





My big problem with the movie PI is that I don't know how much the director wants me to take seriously: that is, since most of the movie is wrong (you don't repair computers with a quarter inch drill), am I supposed to believe that the computer shown in the movie is real, or am I to understand that everything in the picture is symbolic and probably happened only in the mind of the character?

I have the same problem with the article "Programming Under The Wizard's Spell" in the current Harper's. Fortunately the article is a reprint from Salon, so you can read it yourself if you like. Ellen Ullman, who purports to be a UNIX Wizard, and who fills the article with references to "we programmers" waxes euphoric over buying Linux instead of an upgrade to NT 3.51. She wanted software for an extra machine on which to install her clients' software. Remember that word client. We will meet it again.

She went to the store to get an upgrade to NT 3.51 Server. About time, too, given that this was May 1998. When it came down to it, though she "committed an act of technological rebellion" and bought Linux instead. Then she went home and began "by ridding my system of Microsoft." This wasn't easy, but "all that is only a challenge for a determined engineer." Her system wasn't working well, and she'd long since lost the original start-up disk, she says. It took two hours to delete NT 3.51. Two hours.

And here's my problem: how much of that am I to take seriously? That a professional programmer, one who has clients, has let her system get so badly mismanaged that much of it doesn't work, but she still uses it for client software? That it takes her two hours to uninstall NT? Gosh. If I need to do that, I can either boot in DOS and do "Format C:" or Fdisk; if it's all NTFS I'd need to do a low level disk format. Either way it doesn't take any two hours. Now if I were reinstalling NT I might not want to scrub down to bare wood like that since I'd have to reinstall all the software, but she was changing operating systems: surely, if one is changing to Linux from NT Server 3.51, one doesn't expect any applications to survive? But she had "the feel of Keir Dullea turning off the keys to HAL's memory core," but with the sign reversed: she was making her system smarter, not dumber.

As I say, how am I to take this? It's a confession of incompetence if taken literally. If taken symbolically, then what else is she saying that is symbolic but not real?

For instance, she really and truly resents the "My Computer" icon in Windows. She admits she knows you can rename it, but "even if you change the name, the damage is done: it's how you are encouraged to think of the system." Whatever that means. Once again, how am I supposed to take that? That this computer professional is so addled that having seen the name "My Computer" on an icon she's perfectly capable of renaming she is now so damaged that she can't really work on an NT system? It's what she says, but if you believe it, you surely wouldn't want to hire her.

And there's a lot more. A long diatribe against programming Wizards. (I bet she was not opposed to CASE tools back when we called them that; but then they had a more mysterious sound, didn’t they?) She is horrified that you might write programs when you don't really understand what's going on. Is that the part of the essay I am to take seriously? Because I doubt she understands the real innards of chips down to quantum level. I surely don't. The fact is we all use systems we don't entirely understand. It's part of the complexity of civilization. Specialization makes civilization possible. I don't understand my own body down to the cellular level, but I go on using it, and I doubt I will ever try to learn do it yourself brain surgery.

I think she has a point, but it's buried deeply. Now I have been tempted to write whatever it takes in order to sell an article to a major source -- I presume Harper's is a major sale -- but I always resisted; way back in early times I decided I would not write any non-fiction I didn't believe, and tell no stories I didn't want to tell, and darned few I didn't believe could happen. (I do have some problems with Larry Niven's Man Kzinn Wars because Niven's Known Space is not self consistent, but I managed to clear up some of the inconsistencies in the stories I wrote, and solving the puzzle of the home life of a species with non-sentient females was enough intellectual challenge to keep me interested.) Having crowed about my purity, I have to then admit I did the novelization of ESCAPE FROM THE PLANET OF THE APES. I needed the money.

Her point, I think, is buried under the obligatory insults to Microsoft and the poetic talk about being seduced by wizards and the like: namely that there's a lot of code out there that no one understands. This is true, and it can be a cause for concern; but I doubt that switching to Linux and C or even C++ is going to solve the problem. I long ago predicted that if we did everything in C, we would soon be in a situation in which no one would know how the systems worked because unless you work on a C program almost full time you will soon forget how you did things, and if you take up someone else's C program, even with lots of documentation, you probably won't understand it well: you have to accept it on faith and go on from there.

This was a problem Nikolas Wirth was writing about when I first got interested in computers. Edsger Dijkstra's "GOTO SEEN HARMFUL" and other essays on code were also influential at the time. However, for good or bad, Kernighan and allies won; most programs are now written in C.

Which inevitably led to the wizards and other programming aids that Ellen Ullman rails against. It had to. Since only the best programmers understand someone else's C code, and since the number of good C programmers is limited, if we're to get programs written we HAVE to use programming tools, higher level languages, and practices that generate code we don't understand. It's that or no code at all.

My wife, for instance: she wants to translate her literacy program for the Mac to Windows. She can learn Visual Basic, and she's learning it. Now granted, VB isn't going to produce an elegant program. On the other hand, she has a chance of getting a program out the door in Visual Basic. She'd never learn C.

Larry Niven has a number of favorite phrases. One I like a lot is "Not responsible for advice not taken." That's me in this instance: I long ago argued that if we wanted code that was understandable, maintainable, modular if you will, we would have to go to something like Wirth's Modula; to languages with strong type checking, and range checking, and what my friend Marvin Minsky who codes in APL and LISP calls "straight-jacket languages." I have not forgotten the scorn Marvin heaped on me when I admitted I would rather write something in Pascal and understand it a few years later, than to do it more elegantly in LISP and a week later be unable to comprehend what I had done. I still feel that way, though.

If her point is that civilization is dependent on processes we don't understand, and this is a trend that ought to alarm us, I even agree. I'm not sure what can be done about it.

But I think her point really is that the world ought to consist of professionals and clients, and if we use UNIX and Linux instead of the programming Wizards that come standard with Microsoft software, then there will be full employment of the human wizards who understand UNIX, and that will be good. For her. Whether it's so good for the clients is another matter entirely.

I think my point here is that one uses the tools one has, and while it is good to identify what we don't have that we need, it's also very often a pointless exercise. Work must be done; waiting for elegance usually means we don't get done at all. Learning to program elegantly may be best, but best is very often the enemy of good enough.

And meanwhile, I don't know that I have great faith in a professional who needs two hours to uninstall NT Server 3.51.

lav_rd57.gif (16536 bytes)

We got a lot of mail on this essay. I'll include some of it that makes points that ought to be made.



I read both Ms. Ullman’s article and your rebuttal. Both had valid points to make. I both agree and disagree with both of you, here’s why.

First of all let’s leave out any comments on how long it took her to wipe out NT 3.5.1. We both agree there so enough said.

Your rebuttal correctly points out that _end_users_ find wizard tools extremely useful. After all, how many of us design web pages with a straight text editor? For most uses, these tools are extremely welcome.

Now on the other hand, she basically says that _programmers_ need to know the underlying principles and practices of coding. I agree. Self described computer engineers who are lost outside of the pointy-clicky world of wizards are not true engineers. They are end users, power users perhaps but end users all the same.

Think of the imperial power systems engineer in Asimov’s Foundation series who could _operate_ a plant but would be lost if something went wrong. He wasn’t an engineer, he was a user.

We need to have people who still can grok technical details and who will not be crippled without their wizards and yet we still need wizards for those people who have no pressing need to fiddle with the fiddly bits.

So you’re both right. You just chose to view the argument from opposite extremes.



I tend to agree with most of that, and had the article been in a techie magazine I'd have been a great deal less harsh in my judgments. Harper's isn't techie. On the other hand, I've been strapped for a market and a sale is a sale.

We are not going to live without wizards; or if we do we are back in the hands of the experts and high priests, and that's not good at all. But it's certain that someone must understand what's underneath. Just because I don't want the high priests to run our lives doesn't meant there shouldn't be any. Thanks for reminding me.

Then we have this:

Scali Braxi []

You scored the easy point in your article about hers. If you are indeed looking for the symbolism, it's that the narrator is someone who thinks in terms of the system so much that she can't even figure out how to get rid of the system except through the system. Stupid way to think? Yes. That was the point of the article.

I like the fact she wrote her article. Made me think about things. I think there is a way for you to criticise someone publicly without tearing her down. Requires more wit. Better than savaging someone so that an intelligent mind will never get to read her thoughts.

I fear I am not smart enough to understand this one. I gave the link to the original essay, so I expect that people who never saw it before read it; so much for intelligent minds. As to savaging, come now: I reserve savaging for politicians and idiots and preferably both; Ullman is neither, and clearly I thought her essay worth commenting on and thus bringing it to people's attention when I could easily have ignored it. Of course I had not missed the point that if one stays within a system one is limited by that system. When General Graham published his strategic analysis of the Cold War he explicitly called SDI a 'strategic sidestep', a way out of the closed loop of thinking that had previously been dominant. A point Possony and I made in Strategy of Technology...



birdline.gif (1428 bytes)