Not Sorcery or Alchemy: Redefining How People See Software

The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. (…) One types the correct incantation on a keyboard, and a display screen comes to life, showing things that never were nor could be.
Frederick P. Brooks, Jr.
The Mythical Man-Month

In this post, I will

  • outline the idea that software has become exponentially more prominent in our lives and yet no more understood by its users.
  • liken software in the public mind to Sorcery rather than Art and Alchemy rather than Science, noting in particular the means in which it is marketed.
  • argue that this might boost egos but hinders progress, as the magic that delights can also sow the seeds of misinformation and distrust.
  • propose several ideas that could encourage users to not only better understand software, but also partner with it to make systems more powerful, more secure, and better meet the needs of a growing world.

The Ghost in the Machine

I helped my Mom buy an iPhone; she loves it! Funny thing is: she actually apologies to Siri.

So said a friend of mine one afternoon over coffee, giggling over how much fun her mother was having with her new, “magical” device. As she went on, excited about being able to FaceTime with her and send pictures from around Kansas City, I paused.

Why would she apologize to software?

I’m sure it’s not a surprise to get that kind of reaction from an Apple product… they certainly do a lot of research, particularly around user interfaces, to make them as friendly and easy-to-use as possible. So why would I be surprised that a user would actually apologize to this life-like entity in their phone for not speaking clearly?

It’s not like Apple wouldn’t prefer this reaction. A look at some of their advertising confirms it.

A screenshot of one of Apple's early advertisements for the iPad.

A screenshot of one of Apple's early advertisements for the iPad.

And it’s not just users who get this impression. A good number of reviews of Apple’s products, from professionals who follow the industry, use terms like “magic” and “wizard“.

But I can’t really call Apple to the carpet here. As a software engineer myself, I take a great deal of satisfaction from the exclamations of “WHOA! How’d you DO that?” and “I didn’t know that was even POSSIBLE!”. I think it’s true for anyone in our industry. Even my mother loves telling people I “make magic with a computer”.

But I’m beginning to believe it’s starting to make our work much, much harder.

Writing Software: Art or Science?

When I worked at Cerner, the CEO, Neal Patterson, would love to ask a roomful of software engineers and architects: “is writing software an art or a science?” The discussion would be philosophical and lively, never heated, and usually landed on one answer: both.

As I see it, writing software is as much Art as it is Science. The most common exclamation of praise between code authors usually involves how fast or well something works, but any geek can tell you code can be as beautiful in structure as it is in functionality or performance. It it constantly revised, picked over, elevated, shunned, taken as gospel. Comments within source tell a story of its creation. Some engineers print out portions simply for the aesthetic. A computer science professor of mine would give ten points for assignments: one point if it worked, two points for the exhaustive test plan, and the remaining seven for what he called “elegance“. I’ve heard code likened to poetry, “sonnets” of instruction that ebb and flow down the page, much like any spoken language.

The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. (…) Yet the program construct, unlike the poet’s words, is real in the sense that it moves and works, producing visible outputs separately from the construct itself. It prints results, draws pictures, produces sounds, moves arms. (…) One types the correct incantation on a keyboard, and a display screen comes to life, showing things that never were nor could be.
Frederick P. Brooks, Jr.
The Mythical Man-Month

Thus, painstakingly and lovingly revised, at various points in its life, the software is loaded onto devices all around the world. This is when it gets fun, as it begins to take on a “life”. Users begin to “play” with it, using it for it’s intended (and even unintended) purposes. And upon these users’ reactions lies the fate of careers, (and, really, dreams)… a lot of pressure.

So it comes as no surprise that they take advantage of this “magician” status. We’ve reached a point in this technology age where developers are constantly sought out: founders seeking “technical co-founders”, start-ups seeking “hired guns”, recruiters shamelessly flooding LinkedIn with monikers like “ninja”, “jedi”, and “rockstar”.

Thankfully, with some exceptions, developers seem to be a type not easily taken by hubris. Many engineers I meet are in love with their work, contributing to amazing communities, creating out of sheer will… basically artists in the medium of computer science.

But in this day of app stores and more devices than people, is there a cost to this perception?

Art to Sorcery, Science to Alchemy

In January, I took a User Interface job with Facebook. Where before it was simply a tool for mashups, my world was quickly enveloped with the reality of the “Social Graph”. Suddenly I could see, at the tiniest level, how powerful a click of a “Like” button could be to providing powerful services to all users. I wasn’t just telling Facebook I liked an author, I was connecting myself to a section of a system that could then connect me elsewhere, on its own… and that it wasn’t just Facebook.

The user community, not the developer community, is building out an incredible network of information based on services: the software construct has taken life.

I was floored by this sudden epiphany. And my friends wanted to know what working at Facebook was like. After a quick description, I would ask a question:

ME: So you’ve heard of the Social Graph?

FRIEND: Oh yeah, sure…

ME: Say I gave you a piece of paper and a pen and asked you to draw your Social Graph. What would it look like?

Half would try to describe a connection of entities as dots and lines… but the other half would arrive at something different. The most popular was a line graph of connections or “friends” over time.

So here we have a computer science data structure, elevated out of code and into the social sphere, one we use every day whether we know it or not… and yet people don’t fully understand how it impacts them or even how it works.

And it goes further… this is an online chat I had in March with a friend:

FRIEND: Is it possible to hack someones twitter?

ME: How do you mean?

FRIEND: like get their password, go in and tweet as them or even just take down their whole twitter site or change the password? i have a friend i am fairly sure this has happened to….and i was looking for accurate information….never know on the internet

ME: sure, it’s possible:

FRIEND: like there are programs you can buy now, right?….they dont even have to know how to do it, just pay like 50 bucks and someone will give you the password?

ME: No, that’s not how it works. It would be surprising if Twitter could be hacked that way.

FRIEND: this person I know had a specific person that was after them…the man that did it is a reality celeb…he’s got $$

ME: The most likely culprit is your friend was tricked into typing their password into a site that looked like Twitter, but wasn’t. If it IS someone your friend knows, they may have the password to his email. Just have your friend change their passwords and watch carefully how they sign in.

FRIEND: we know it’s him, we are just trying to figure out how this guy is doing it…i can’t believe twitter can’t fix it

I happened to have a transcript of this conversation, but I’ve had many like it with many different people. It’s not a big deal; question asked, question answered. But if you’re at all involved in the software industry, this conversation should be setting off alarm bells.

Why would a simple case of phishing be elevated to a hacking conspiracy, one where the blame ultimately falls at the feet of the service provider?

I believe the answer is simple: when someone uses magical functionality, they expect the magic to protect them as well.

In the world of software, companies are driven to make it simple to sign up and then as easy to use as possible. Once the user is engaged, the focus shift to “customer support”: phone banks, online documentation, bug reports and forums. This is the status quo, (and an expensive one, at that).

Yet when a consumer buys a chainsaw, they understand at a basic level how it works: gas engine, spinning bladed chain, on-off button. The manufacturer includes a manual with plenty of warnings and are thus released from any liability of damage or injury. This carries with cars, microwaves, firearms, checking accounts… these things are fundamentally understood from common experience. If the flight crew were to skip the safety speech on a plane, how many seat belts wouldn’t be fastened at take-off due to people not understanding the buckle?

Common Understanding

But it’s not that the software industry is negligent or lazy. On the contrary, software engineers are diligent, trying to solve the range of user problems in their software as they would any other problem: with software. Indeed, larger and larger proportions of programs are allocated trying to detect problems, missteps, confusion or attack from users. Indeed, if you click “Send” on an email to your mother about how to wean your son off his “attachment to his blue blanket”, Google Mail will warn you your message lacks any file attached.

At some point, software understanding users has trumped users understanding software.

The “software is sorcery” concept also pervades our popular media. How many movies or television shows have used nebulous software concepts as either hero or villan in their plot? In some like-our world, an evil geek in glasses unleashes some 3D hydra virus upon a banking network, stealing billions. Elsewhere, like the Deus ex Machina of ancient Greece, fast hands over a keyboard in the FBI office by some geek in glasses overcomes evil.

The fact is it doesn’t take much to suspend disbelief. Take this clip from a popular television show, depicting a world where two people can type on one keyboard, and unplugging a single terminal stops a network “hack”?

Or this one, where a criminologist decides to create a graphical user interface in an ancient language to track an IP address?

Educating users on the fly, only as necessary, has led to distrust, more expensive support, and diminished demand. Further, it has caused a reluctance to be open, from users to try new systems and from services who don’t want to harm the trust they’ve built thus far.

Knowledge is Power

The cover of the Chilton Repair Manual for a Grand Am

Chilton Repair Manual

At 16, I bought a 1988 Grand Am for $2000. As with any car that age and price, there was an expectation that it would “need a bit of work”. In the end, it turned out to be one of those high school cars to which you simply added oil, rather than change it.

Among other things, my father and I had to replace a portion of the engine block. Rather than spend a lot of money on a service shop, we bought a Chilton Repair Manual, a comprehensive book of instruction for the car from bumper to bumper. In the end, I provided its pages with plenty of grease marks and it provided me with plenty of saved cash and freedom.

Rarely, if ever, does software come with a Chilton manual.

Imagine if people had an opportunity to understood software better at such a fundamental level? That Siri isn’t some magic assistant in a phone but is really a Internet-powered vocal-recognition engine with a well-seeded database of responses? That a modem gives a unique “phone number address” that is public, so it’s safer to have an internal set of numbers with Caller ID? That “the blue E” is not “the Internet”, and there are other windows through which to view the actual Internet?

Wouldn’t this lead to the contracting of this “infinite possibility and evil” of the software world? That granting a video service access to your graph means not having to watch an advertisement for a steakhouse when you’re a vegetarian? That the word “privacy” is no longer the black and white difference of what’s written in your diary vs on a billboard?

And wouldn’t understanding software lead to people being better equipped to recognize problems? That a site promising free music wants to install something you don’t recognize first? That a Nigerian prince sitting on vast wealth plucked an email address out of thin air? That the person trolling your blog didn’t drop fifty bucks on some program to hack your Twitter account?

This is a topic I explored in a Pecha Kucha talk: why aren’t people interested in writing software? I’ve come to believe it’s less about the code and more about our culture.

I believe it’s time we bring the software systems we use, including their development, support, capabilities and limitations, beyond their meager support systems and into greater awareness.

Four Possible Steps

“Never oppose if you can’t propose.” It’s a great mantra for software development, and it’s one I’ll hold here. I’m supporting four solutions that could lead us to a better ecosystem.

More Open-source Initiatives

Software manufacturers don’t want the “wow, how’d they do that?” exclamation only from their users: they also want it from their competitors.

At the core of any modern business strategy is the desire to hide the implementation of a product in deep secrecy, (just look at Coca-Cola). You must recoup and then profit from the expense of creating a top-tier product; dealing with clones cuts at that profit.

Yet to any developer, under the appropriate circumstances, sharing code is a no-brainer. More eyes equals more passion, more use and a better product. And while the ultimate construction can remain proprietary, the individual pieces benefit from an open-source license.

For end-users, more open-source software means greater transparency and more understanding. Knowing that your favorite iPhone app isn’t hundreds of thousands of lines of code that doesn’t transfer your contact list or bank information to the developer should be a fact easily believed. Public, readable code can help.

Even though consumers don’t care to know how their favorite sausage is made, they have a very good idea already. This is not the case with software.

Pivot Marketing

Product marketers are the first contact for a product relationship. Not only should they and their materials be well-versed in what the product is (and isn’t), both should resist the temptation to cast their products as “mystical”. Leaving the details to the user manual, (which, let’s face it, most people ignore until they have a problem), will only lead to frustration.

Users should be more delighted and amazed by how well it works and less by that it works.

Creating a Resource for Popular Culture

Perhaps none but geeks have noticed a fundamental shift in the origin stories of popular super heroes like Spider-Man and The Incredible Hulk. Specifically, the dramatic means by which they gained their “super powers” has changed significantly over the decades… and always in lock-step with the public’s understanding (and fear) of technology.

In the 50’s and 60’s, radiation was the catalyst. Bombardment of gamma rays and the bite of a radioactive spider were, in the eyes of the public, credible means of these transformations. In today’s world, this would be laughable, for obvious reasons: we understand the technology, and its limitations.

But notice the shift. In the 2000s, origin tech for Spider-Man and The Hulk shifted to genetic modification and nanotechnology, respectively. And, even now, the idea that the bite of a genetically-altered spider could cause a teenager to shoot webs from his wrists is a stretch, (the latest reboot appears to have quickly done away with it).

Sources of entertainment can be more effective in education than any textbook or manual. Case in point: some tourists ask to see more info on Jack when they visit the Titanic museum.

Therefore, I believe providing writers with a credible source of information and “tech fact checking” could be a useful tool. It could help to keep plots involving software and technology more “in the clouds” of the possible, rather than in the stratosphere of the incredible.

I’m currently working to launch a new .org website to curate and collect “tech facts and stories” for the entertainment industry.

Creating Popular Culture

My last suggestion is perhaps the most ambitious of all… but as a software engineer with a background in theatre, I had to list it. 🙂

I’d like to pitch a television series in which the first 15 minutes is used to describe a common technology and how it works. This could include little-known facts, detailed descriptions and how to use it properly and safely. The remaining 45 minutes would be a guest-written and directed comedy or drama in which the tech is prominently featured.

By making technology the focus instead of the foil in a popular culture context, people might gain better understanding of how it works and fits in their lives.


These are just a few of my thoughts on this topic, and my list of solutions is certainly not exhaustive. I intend for this post, like software, to have a life after this first writing. I’d love your comments, suggestions and critiques. Let me know what you think below!

Follow Me

Subscribe to our e-mail newsletter to receive updates.

7 Awesome Comments So Far

Don't be a stranger, join the discussion by leaving your own comment
  1. Matthew Lero
    April 19, 2012 at 4:06 pm #

    I’ve always assumed that you guys (programming ninjas?) didn’t really want the average person to understand how this stuff works. That to reveal how it works and what makes the sleek new interface or website work so well is akin to the magician giving away his tricks. Sounds like, and I hope, that isn’t the case. Peeking behind the curtain is exciting, understanding even a little about how things do what they do is fantastic. Look at how many “how its made” shows are out there now. Maybe this could be the start of the software equivalent…

  2. Andy Kallenbach
    April 19, 2012 at 4:27 pm #

    I love this topic because much of my time is spent creating and managing software projects for people who often DON’T understand what they are asking for. I find that my customers and users often want nothing to do with understanding anything technical. They have an objective and want me to solve it.

    The last few years I have been actively trying to educate along side the development process much to their displeasure in many cases. I feel this has been productive and creates a better product… but it is like pulling teeth. The attitude of many users seems to make this pretty difficult problem to solve. I don’t have the answers, I like some of yours..

    This is just an off-hand observation, but I wager that there have been two moments that users have been mostly competent with their computer.

    1) The early 80s because users had to. Maybe the command line forced people to know a little more… maybe there were not so many moving pieces… maybe it was more hobby oriented… but people actually typed commands to do stuff and had a reasonable assumption of how/why it worked.

    2) iPhone – Enough of the user action was simplified with limited and strict behavior to create an approachable product that people used with a fair level of competency.

    #2 puts much of the problem in our arena to solve. Much of what we learned in school or as a interest was how to operate and interface with a computer “well”…. not how to help others understand or use it well.

    It is my personal goal to not only help others understand their technology better but also to create technology that is easier to understand.

  3. Sam Bao
    April 19, 2012 at 8:46 pm #

    Too much to try and communicate all of this on twitter, but one suggestion is to say something about what kinds of incentives there are for people to want to understand software, to actually drive change in behavior. A lot of the people you and I know and work with have an intrinsic incentive to want to know, understand, and learn how some piece of software works. Those of us with this drive naturally migrate to doing software development at some level.

    For others that don’t have that motivation, from their perspective (not mine), why should they want to perceive software differently and understand it more? Is it because they want to write software? Is it to save them money somehow? Is to be able to hang out with the hip kidz? I really would like to hear your thoughts on this.

    Let me rephrase it this way using your car example. You bought a manual to help you fix your Grand Am because you had some strong incentive to learn how your car works; it saved you cash when you’re a poor high school kid. Or maybe you do have a intrinsic urge to want to know how to rebuild an engine block and being an auto mechanic was a career option for you 🙂

    For me, however, I’ve not been in a situation where I wanted to know how to fix a car because I’d rather pay someone to fix it, and I have the means to do so. Thus the incentive for me to understand how to fix my car isn’t strong enough. Whenever I go to a mechanic, I nod and agree with their assessment, and then I ask my father-in-law whether I’m getting ripped off or not, but for the most part, I trust what they tell me. I also believe that all the car-mods they do in Fast And Furious is based in reality (am I mistaken?).

    As a software engineer, yes I agree with you. I too want more people to better understand software, and to realize that it’s really not all that magical. I want people to be able to tell when Hollywood is exaggerating. I want people to realize that technology is going to take over more and more of our lives, and they should want to understand the world around them. However, is this desire selfishly for us? So that I won’t be called up when Netflix won’t stream, or when Facebook isn’t liking something, or BSOD pops up? From my relatives, or neighbors, or even my wife’s standpoint, what’s in it for them to gain this understanding? And whatever those incentives are, is it enough to actually change their behavior?

    • Clint
      April 19, 2012 at 9:03 pm #

      Very well put…! Behavior change wasn’t really a term I was thinking, but it’s a good one. I was taking it more from the perspective of simply not providing anything realistic. Honestly, even without an incentive, I’d live the opportunity to influence that understanding positively… and once the mysticism of APIs and services is dispelled, imagine encouraging a non-programmer to write something custom that displays their Twitter feed or updates their Tumblr.

      That’s kind of where I’m going with the “influencing pop culture” idea. To push the analogy even further, yes, not everyone wants to work on their car. But everyone who drives has a fundamental understanding of how it is working, how the engine runs. And no matter what noise it makes, everyone opens the hood of the car and looks in when there is a problem. Sure, you take it to a mechanic, but when the battery is dead or the tire is flat, you at least know how to repair those items, (even if you choose to call roadside assist).

      I think the best chance we have of both inspiring interest in and fundamental understand of our work from the general public is not to create a Learning Channel documentary, but to inject realism into mainstream culture.

      Until CSI hit the airwaves, you would rarely if ever see an advertisement for criminal investigation trade school; now you see commercials every half hour when you’re home sick watching Judge Judy. I’d love to get the same enthusiasm behind our field.

      I think if we can at least convince people that software is accessible to them, I think behavior change will happen because people will incentivize themselves with their ideas.

      • Kris Nessa
        April 20, 2012 at 2:43 am #

        Sent you an email Clint…but wanted to reply here because of the great convo Sam started.

        To your statements Clint that “But everyone who drives has a fundamental understanding of how it is working…”, this is not true. Even your statements about dead batteries and flat tires, people don’t know how to repair those things. Honestly.

        Now, as a girl, I have witnessed many a crazy things from other women of the world. And not to call out only women, there are plenty of men who “never pop open the hood” as you mention, when something is not working. People call up the mechanic, tow service, or their fathers and friends immediately.

        I get asked routinely to speak at “Women in Technology” and STEM events. Even at plenty of Cerner events because yes, there is a shortage of women in STEM initiatives. I get asked by parents all the time, “how do I get my girl to be interested in (STEM) like you were?”. Great question, always hard to answer. Developing the intrinsic enthusiasm, or maybe it’s frustration, that something is broken and I CAN FIX IT MYSELF. You want a child to pick up tools on the father’s workbench and try to fix their own broken toys. THAT is the ultimate curiosity that you want a person to have because then they naturally have the base curiosity to try software programming. Not being scared of things in the world, wanting to know how they work, taking them apart, and putting them back together (sometimes better than before).

        This leads me to your final point Clint, about “injecting realism into culture”. Realism may have the adverse effect of what you are looking for. The increase in criminal science and investigation degrees in our culture recently is directly correlated to the non-realism of pop culture. Real CSI people don’t have cool cases like the TV show everyday. Probably only 0.5% of the workers have probably ever appeared on a real-life Dateline or Investigation TV show about a real case they were involved with.

        You know what helped some of our age group? Star Wars. We all loved the moves, wanted light sabers, and wanted to talk like Darth Vader. I had this awesome little space helmet that would transform my voice from this internal microphone into Vader-like voices. I loved it. It broke one day, and I was devastated. You know what I did? I walked downstairs to my dad’s workbench and got to work trying to get that helmet working again. When I got it working early the next day, it was the best day of my life. And now I have an electrical engineering degree and believe I can do whatever I put my mind to.

  4. Skyler Brungardt
    June 7, 2012 at 3:40 am #

    This article really struck a chord with me. Having read it, I couldn’t help but feel a feeling of dread arise in my belly, as the fear I’ve long had was a fear that was real enough for other people to share: that the work I do to make technology easier to use also makes it easier to abuse. And the less the layfolk understand about it, the less likely the problems are to get fixed, and the more room for predators and laziness amongst developers there is.

    Thinking on the propositions you list, (and as a fellow software engineer with a theater background – I really like #4, btw) I’d like to offer that there’s another one worth including: Widespread Gamification.

    What we really need is for the software to teach users about itself -on the fly-, as a user is using it, similar to how games have been doing for a long time. The point of entry should be obvious (like the analog stick or d-pad on a controller), and through positive reinforcement and deliberate tasking/challenges the software should reveal its Ins and Outs, teaching about risks and pitfalls, just as games teach about resource management through life pools, magic reserves, or ammo clips pitted against increasingly challenging odds.

    Klout does a fairly good job of this, so far, incentivizing a user to participate in complicated and arbitrary tasks. Same for Coderwall.

    Love to hear your thoughts. Good luck on the proposition.


  1. Rounded Corners 371 – Enabling comments /by @assaf - May 27, 2012

    […] Siri Not Sorcery or Alchemy: when you build technology that’s indistinguishable from magic, it’s important to […]

Leave a Comment

Remember to play nicely folks, nobody likes a troll.