I recently read (yet another) column in the journal Communications of the Association of Computing Machinery about deficiencies in programmer education, that reminded me of a recent paint job for which I’d recently paid “professionals”. The connecting point is a certain lack of meticulousness, precision, and professionalism that we now seem to accept in most professions. This lack goes back to how crafts people are trained.
In software, as in many crafts on which our daily lives now depend, the problem is that beginners are quickly taught the rudimentary skills, and then exhorted to creatively solve problems as presented. If a program appears to give the correct result for expected input, then it gets an “A” grade. Similarly, if a paint job looks to be the right color at completion, then it also is judged superlative.
However, what if the input is different than expected? What will the paint look like after a few seasons? What will happen when someone tries to add on to the current code/coat? That is where the basic training process fails. Code written to accept “2 + 2” and return “4” might take “7 + 3” and possibly return “10” or “4” or “21” (fellow geeks will see how). What happens when (not if) the input is “G + #”? Likewise, a perfect paint job over a dirty surface won’t last a season. If it does last that long, then it may peel when the next coat of paint over it dries.
What is missing is a basic principle of craftsmanship: Depth of knowledge. The best painter on my recent job had decades of experience. But all he really knew was how to lay color on a clean surface, and basic prep work. He didn’t understand enough about architecture to know what did or did not require caulk (he sealed sashes to frames, and left gaps between frames and siding). He didn’t know that many surfaces should not have been painted with latex, like hinges and sash runners. He didn’t know the implications to longevity of using a brush versus a roller.
I would have thought that he might have learned these things in his first couple of jobs, or years, or decades of practice. But one aspect is overlooked in training these crafts-folk: Temperament. Certain people have the curiosity and meticulous dedication to understand every aspect of a task, while others (the majority) just do as they are told, at best. But in America, everyone is created equal. One cannot discriminate.
Back in the bad old days of exclusionary guilds, only those who showed the necessary aptitude were accepted to apprentice. Only those who proved themselves adept moved on to journeymen, and eventually mastery. One therefore knew that any carpenter could make solid chairs, and a random tinker could permanently fix a leak. Now, anyone who can pony up the price of the tools, the schools, or union dues can call himself a programmer or a painter or what have you, and hang out a shingle.
I freely admit that I am a self-taught programmer, and painter, and carpenter, and plumber, and electrician, and so on. But I’ve got the borderline OCD tendencies to read the full manuals (Kernighan & Ritchie, NEC, whatever) and I like to play with things to find out their limits. Part of my self-education is also to find and work with or study from someone who got good results, to see how it is done.
The point is, a true craftsman has the temperament to know what he is doing, plus several levels of abstraction on either side. A painter should know what pigments actually are, beyond the color they produce. That way he can choose paint either that is safer around kids, or better at preserving wood (generally complementary characteristics). A painter should know how different binders work, to choose a paint that is better for metal, or vinyl, or wood. But to my chagrin, given the universe of things that I think every painter should know, few that I’ve hired even knew the questions for, or even that there was an issue to wonder about (2nd or 3rd orders of ignorance).
And this is part of why our civilization is coming apart.
Note: Earlier posts here on similar themes: Incompetence as the Basis of Civilization and Incompetent people don’t realize that they are incompetent.
Dan:
I too bemoan the lack of expertise in experts, the lack of insight in those deemed insightful, and the omnipresent lack of ability in those who are supposed to be able.
In my own 'field' of consulting, I continually see (supposedly expert) people who have seemingly never encountered a boundary condition they couldn't just ignore, a process variation that couldn't simply be omitted from analysis, or a requirement that couldn't be easily satisfied – usually by ignoring second- or third-order ramifications.
I'm not happy that I'm not alone in this.
I too do a reasonable job of electrical, mechanical, and other crafts-related work… unfortunately my paying job demands too much of my time, so those tasks needs must be contracted out, with results similar to yourself.
Is it education, demanding a passing grade for everyone? Is it the result of TV-land fantasies, where every man is able, and all problems are forgotten by the next episode? Is it a fixation on mass-media sports, where all that matters is winning, not how you play or perform?
I don't know. All I know is I DON'T LIKE IT!
Now GET OFF MY LAWN (!)
Concur wholeheartedly! I bought a house in Oklahoma in 1989, and coming from the East Coast, had never seen wall texture. When I asked the handyman doing some work as part of the purchase what it was and why it was used, he had me put my eye on the wall and look down. Wavy walls. He said "They don't make them like they used to back East."
I learned to program in the late 1970s. Hashing was necessary and code was optimized. Memory was a premium. No one codes efficiently today. They don't need to. NASA sent men to the moon with a computer that had 48K of RAM. Looked at the size of the executable (let alone the libraries) of Excel lately?
Obesity is a problem in program code as well as body mass, and also the laisse faire attitude. Conservation is too slow to catch on. Coincidence? Hardly. Land of plenty, and all.
Here's a message we hear all too often: "Your call might be monitored or recorded for quality assurance." Translated: We employ lots of slackers and losers. We can't trust them to provide basic information to you over the phone.
I would think that people who do the same thing over and over each day would exercise the curiosity to learn a lot about what they do so that they could answer basic questions. Not so, in my experience. Whenever I've had x-rays and MRI's over the past few years, I've asked the technicians how the equipment works. For the most part these people show little or no curiosity. How can this be?
Today in court, the clerk asked the bailiff to make two copies of a court order. Keep in mind that bailiffs do next to nothing most of the time. Therefore there is a lot of time available for learning. The fellow went down the hall, then came back with staples on the bottoms of the pages, rather than the top left corner. How hard would it be to consider that people might want to READ the document.
When I see this shallow depth of understanding, I usually don't think of intelligence issues. Rather, I think of the writings of Karl Marx, who described the phenomenon of alienation from one's labors pervasive in society.
One of my clients websites was down last night with a message implying some obscure internal server error. I called the ISP support. I have become quite adept at showing Tier 1 that my calls need Tier 2. That is, about only fifteen minutes to convince them that I am not an anykey seeker, that my problem is real, on their system, and beyond the abilities or permissions of a scripted receptionist. He posted it to Tier 2, and the site was then working in minutes.
It is an ISP that has a call center in this country. This is helpful because I can meta-communicate to a native English speaker. They know the intonations, cultural references, and vernacular that let me make them want to help, to win them to my side (as in Dale Carnegie or NLP)
Sure, I have dealt with Bangalore galore for other ISP's and services. Most of those guys are perfectly competent. But we can only communicate on the level of the exact words used. It is slower to get to the second tier that I (almost) always need.
Back in the 1980's I was working in Microsoft QuickBasic (the language in which Lotus 1-2-3 was written). I was regularly pushing the edge of what the language was capable of, and finding bugs in their compiler. So I got to where I could ask whoever answered the phone at Microsoft for the actual language developers by name. It was like having a VIP pass. Ah, the good old days.
I started programming back in the late 70s, went pro in the 80s.I think the lack of craftsmanship is the direct result of the oligopolies and near monopolies held by corporations.
Back in the 80s. computer literacy courses taught the art of programming at many levels.
Usually this meant starting with BASIC, then dabbling a little in assembly language. The focus was on programming, not on the use of proprietary software applications.
Last year, my younger son, Habib, took the computer literacy course in middle school. The course taught the kids how to use Microsoft Office suite, with a little bit of coverage on using Gmail. Corporate interests in education has twisted the computer literacy concept away fostering a creative skill and into a very long infomercial with tests for credit.
Back in college, my teachers emphasized programming for efficient use of memory and cpu time. We were told to avoid "spaghetti" code that spent a lot if time skipping around in memory, but the current paradigm pushed by the big software houses has institutionalized spaghetti code in a business model that is symbiotic profitable for the big software companies and the hardware manufacturers.
The result of this model is that most software is built out of purchased "black box" libraries strung together like Lego blocks. Instead of a finely crafted model of efficiency, instead of a custom built engine, modern end-user software is more like a castle built from plastic bricks. We now have the upgrade treadmill, where the new version promises to fix all the problems of the old version, but you need to buy new hardware to run the new software because the bloat added to the code to provide features no one uses not only requires more memory and storage, but the incredible spaghettification of the code adds boatloads of overhead procession to the application, which makes the software slower.
Part of the problem is, that after decades of denying the tools for creativity from the masses, many kids with the right stuff to be programmers are never exposed to the opportunity to learn, Good programmers are hard to find. What we have is a generation of programmers who are familiar with only one crappy tool set. The saying "If your only tool is a hammer, every problem looks like a nail."
The best new software is coming from countries where open source software dominates the market.
Similar trends have been ongoing in other industries. We have been shifted from an economy of production to an economy of consumption.
Niklaus, Dan
Regarding 'learning to program' – back when I went to college we learned basic algorithmic techniques using pascal: a language purposefully bereft of connection to the external world, the better to construct and understand clean algorithms. (we learned many other languages, to ensure we understood the difference between language/tool and algorithm – and also that selection of the right tool makes many 'problems' much easier! I still have a soft spot for Lisp, Smalltalk & prolog. I loved me my prolog-based 'prognosticator'… like the Lisp based conversational machines, except it would take in 'news' feeds and simply re-parse random sentences as a 'pundit' would. Scarily realistic!)
To further demonstrate, as a 'teaching moment' to more junior enrollees (pro & con – clarity of language, structure, etc vs applicability of tool choice!), we were challenged to write a 'real-world' application in our third year using pascal that clearly demonstrated use of algorithms and data-structures.
[exposition]
I wrote a spreadsheet program, similar to 1-2-3 or symphony, in pascal on a Vax – with consoles & line-based teletypes as the only interface. It was built primarily to examine sparse data-set management techniques (including various hashing algorithms, as well as multi-nodal 'pointer' based structures), but to make it 'work' I had to also write a parsing engine (for formulae evaluation) and a dynamic, procedural I/O protocol that allowed the use of the tool on tele-typewriters & consoles (i.e – depending on the 'device class' – line vs field addressable, 'refreshes' were either automated or only on request; on line systems a simple message that informed the user of significant changes resulting from recalc (50 cells have changed values!).
It was, I recall, a lot of fun – and because it was written in a very structured and modular way – I could easily demonstrate different data management techniques and I/O behaviors – simply using command line switches (later: via internal meta-commands during operation) to turn modules on & off.
It was also very easy to get working – with a skeleton – and to enhance with more capabilities, so that I actually had a complete working model within the first few days of the project — a good thing from a demonstration perspective.
[/exposition]
To get back to the point, however, Niklaus comment that current programming is 'lego' programming is exactly right. Why even bother to write the parser – just use a library… even if that means a much larger and much more inefficient program. Why even write anything other than the 'glue'? Using today's programming techniques and available libraries I could have completed my project code in less than a week – since the complex bits were all 'external' to the actual job of the spreadsheet (using things like xlst for storage transformation and serializing, for instance).
It would have taught me nothing about the actual problem, however. I would have learned nothing about sparse-data, about hashing, about I/O protocol building, about UI design, and many other things.
Even today, when I am as far from a programmer as possible – I still occasionally find myself 'schooling' new (and experienced) business programmers in algorithm and data-structure design!
One of my pet concepts is that only someone who has (re)invented a wheel really knows what a wheel can do.