What is the future of art? What media will it use? Computers, obviously. Information technology is very good at imitating old media: drawing programs, music programs, word processors designed for playwrights or authors. But none of these tap into the intrinsic strengths of the computer, able to do something no other medium can: simulate. Bret Victor, the man so demanding of user interfaces he left Apple, is dissatisfied with the tools available to artists that allow them to simulate. So he made his own, and gave a one-hour talk on it.

Those interested should definitely take the time to watch it, but to summarize, he demonstrates the power of simulation in creating art that is part animation and part performance, with the human and computer reacting to one another. He then lifts the curtain and show us the tools he used to simulate the characters in the scene, and it’s not code. Instead, it’s a drawing program, with lines and shapes, that he uses to define behavior. Code, he points out, is based on algebra, but his system is based on geometry. Finally, he concludes with a short performance that he built with these tools. Higher is the story of earth, from the stars to cells to civilization to space travel back to the stars.

What blew my mind about Higher is that a few years ago, I had independently created a short film on exactly that topic, with exactly the same background music (Kyle Gabler’s Best Of Times from World of Goo). Victor’s piece was far more polished, but we had both been inspired by the same music to express the same idea, the journey of life to the stars. Remember when I complained about not finding people who shared my narrative? So this is what that feels like.

What drove Victor to create his tools was the belief that art is an attempt to communicate that which cannot be put into words. By binding simulation to lingual code, we make it inaccessible and unsuitable for art and artists. Direct manipulation of the art, which is how art has been created going back to cave paintings, allows the artist to interact with and lend emotion to the art in ways not possible through code’s layer of indirection, of abstraction.

The reason artists’ needs have been neglected by developers is that, for the rest of the world, code works just fine. As I’ve previously blogged, language is one of humankind’s most powerful inventions. The direct manipulation that is liberating to the artist is confining to the engineer. Language is how we manage many layers of abstraction at once; without it we are reduced to pointing and grunting. It’s harder to communicate with a computer in code than a well-designed direct manipulation interface, but code is more powerful. In the sciences, a good result is consistent with what is already known; in art, a good piece is unexpected and shakes our established worldview. More fundamentally, the sciences observe and record some objective outside truth; art looks inward to offer one of many interpretations of the subjective human experience.

This tension that we see between science and art also shows up in schools. In a recent TED talk, Sir Ken Robinson extols diversity as a fundamental human trait, which schools attempt to erase and replace with standardization. We agree that standardization has its place, but I personally think he downplays its importance. Standardization is writing, is language; those things can’t happen without common ways of thinking. At first, children need to explore concepts and use their own terms, without a top-down lesson plan imposed by school administrators. Nevertheless, the capstone is always learning what the rest of the world calls it. That isn’t smashing creativity, but rather empowering the child to learn more about the topic from others and from reference sources. It’s creating a minimum level of knowledge common every adult member of society, which is assumed by all media. Being able to communicate  facts with others isn’t just the result of education, it’s what makes education possible in the first place. With language, groups of people can unambiguously refer to things not present, a shared imagination. Verbalization is a form of abstraction.

Let’s get back to the role of diversity in school. Students should be able to explore what interests them, but the converse is not true: some topics must be taught to everyone, even if some people do not find them interesting. This is especially true before high school. I know you’re not passionate about fractions, Little Johnny, but you need to learn them. Society expects everyone to have a minimum level of competence in every subject. Additionally, passion for a field isn’t always “love at first sight”. The future mathematician isn’t always the first in the class to get basic arithmetic.

Although the curriculum needs to be largely standardized, the pedagogy does not. The neglect of diversity in schools is most heavily felt not in what kids are or are not learning, but how they are learning it. The inflexibility imposed on lesson plans is degrading to teachers and failing our kids. Teachers should be trusted to adapt lessons to their class, and empowered with testing results they find useful, early enough to use it. Standardized testing as it exists today does not fit the bill. Every student needs to achieve the same core competencies, but the paths to doing so will be as diverse as the children themselves. A broad exposure to both methods and topics promotes the development not just of knowledge, but of personality and identity. The reason to have art in school isn’t to improve test scores but because it’s part of being human.

To be more precise, we should distinguish between “the arts” and “art”. The arts are how to create with the media classically used for art: paint, music, poetry, drama, dance, and so on. Like any other discipline, the arts require a standardized language to record and transfer this knowledge. Sometimes it’s plain English, sometimes it’s jargon, sometimes it’s symbols, but it’s still an agreed-upon abstraction. Diversity of ideas expressed in the language is inventive and healthy; diversity of the language itself is nonstandard and chaotic. With this in mind, the arts take their place at one end of a spectrum of knowledge: mathematics, natural science, social science, and history. And the arts.

But art is something entirely different. It is the personal and emotional perception of an experience that communicates without words. Art is direct and concrete; it is subjective and sublime. Much of the arts attempt to create art. Victor’s tools advance the arts; what he creates with them is art.

It’s a defensible position to say that art, because it does not rely on language as all the other fields of knowledge do, is not knowledge at all. But I’ll indulge Victor and say that not all knowledge can be verbalized. That doesn’t mean that art is beyond classification; Victor and I saw the same artistic ideas in the same piece of lyricless music. Conversely, just because something is written down doesn’t mean it’s standardized or useful knowledge. Recently, the mathematics community has been bewildered by an inscrutable set of papers which claim to prove a fundamental piece of number theory. No one can decipher them to tell if the proof is valid, and their author has not been forthcoming with an oral explanation. So in extreme cases, the analogy between language and standardization breaks down. The wordless expression is more coherent than words.

For all the knowledge that abstract language has brought us, ineffable art remains part of the human experience. It is important for our children to learn about art to become mature and thoughtful adults. It is equally important for us to provide tools that support the nonverbal side of thought, to engage the visual and auditory parts of our brains in ways words never can. These are the same failure: the refuge in abstraction, the desire to have everything neat and orderly and predictable. Art exists to explore ambiguity and paradox; it does not demand simple answers but asks complex questions.

A lot of futurists imagine a time when technology makes everything easy. There is a faith in technological convergence, where everything speaks the same language and interacts intelligently and flawlessly. But historically we see technologies become incompatible. If there’s an open standard underneath, such as email, you still get dozens of providers and clients; and if there’s not, you get the walled gardens of social media, loosely tied together by third-party “integration”. What’s important to realize is that the path of technology is not fixed. Our gadgets don’t have to make us more productive and connected; they can make us more artistic and provide privacy, if we design them so. We should stop aspiring to a monoculture of technology because, not only will it not happen for technical and economic reasons, it shouldn’t happen. Standardized technology leads to standardized thinking, especially when coupled with standardized social institutions. Creativity is  not only what drives technology further, but art and humanity as well.

The Khan Academy dashboard is meant to provide students and teachers with information that can help them target where a student is struggling, and improve. Unfortunately, the data given isn’t what’s useful to teachers, just what’s easy for computers to measure. These metrics include time spent watching and rewinding videos, time spent on different topics (broken up by videos and quizzes), and proficiency levels in exercises. But as one of the teachers I follow on Twitter points out, these programs don’t let him “SEE my students’ work so I can know HOW/WHY they got questions wrong.”

Mathmistakes.org attempts to counteract that. Math teachers send in anonymized samples of student errors that they find telling, common, or inscrutable. Teachers comment as to what they think the student is not understanding and how to fix it. In the process, newer teachers get to see the thought process of their more experienced colleagues. There are patterns to these mistakes, so can a computer be programmed to recognize them?

The most obvious example, which I doubt I’m the first to come up with, is to anticipate patterns of wrong answers. Let’s say you’re testing a physics problem where students need to plug values into an equation. (Yes, this is a naive view of physics but go with it.) Have experienced teachers compile patterns of mistakes students are likely to make: forget to square something, leave off the constant, divide instead of multiply, use a different formula, and so forth. Then the grading software picks new values to plug into the formula, and calculates all the wrong answers for these values (picking numbers so that none of the wrong answer patterns lead to the same numeric answer). Then, if the student gives any of the anticipated wrong answers, the program knows exactly what mistake the student made and can correct them. Hopefully finding a mechanistic error will provide the human teacher with a window into finding and fixing a qualitative misconception.

Let’s take a more complex, real-world example. In some computer science classes at Tufts, the programs written for homework are subjected to a battery of tests, written by both the professor and the students (and their predecessors). In one case, the assignment is to create a programming language interpreter that determines the “type” of pieces of code. For example, it needs to know that true is a boolean, 7 is an integer, and asking if true equals 7 is an error. To clarify, there’s the code the students write (called an interpreter) and then the code that it tries to type, as a test. An interpreter can fail in a number of ways: it can find the wrong type, find a type when it should raise an error, raise and error when it should find a type, stop unexpectedly with an exception, or never stop at all.

I know that’s a bit much to wrap your head around, but (1) that’s the sort of complexity we’re up against and (2) it’s not just an example, it’s a case study. I have a visualization for this data already made, as a class project. My group wanted to take this data and provide actionable reccomendations for the professor, to be able to say, “you’re not handling this properly” or “you don’t understand this very specific detail”. So we hand-built an automated classifier using what we knew about the errors. Here’s part of the visualization we came up with.

The size of each circle represents the number of students who failed at least one test with that error. The vertical position of the circle corresponds to the average number of tests passed by the students who got that error. Colors encode categories, and the horizontal spread means nothing (just a way to prevent overlap). Click on a circle and you’ll get:

Each bar is a student, identified by an anonymized hash. Their errors are grouped together, with the taller bars being the error we have selected. On the real thing (not these images), you can click on any other bar to jump to that error. Hover over the bar and move around to show each of the tests failed with that error below the circles. This highly-specific information allows the user to look at the individual tests and hypothesize the underlying cause of the error.

You can play with the interactive visualization online here.

Education isn’t a no-computers-allowed clubhouse, but software developers are must be smarter about how we approach these tools. Programmers need to work with educators and fill their needs, not just offer up whatever statistics are easy for them to collect. We have powerful tools like machine learning and visualizations, and teachers with decades of experience. We can make useful automated systems, if we stop acting as if it’s a trivial job.

And yet… all of this takes the views education as a series of questions with right and wrong answers. This is largely true in the STEM fields and even in the humanities, but not in the arts. There really is no good way to automate grading of the arts (or to grade the arts, for that matter). We need to nurture our future artists, but more importantly, we need to teach the skills necessary to appreciate the arts, and to dabble in them. As part of the human condition, we all find ourselves with emotions and ideas that we need to express, through music, painting, or writing (writes the blogger). The fact that very few people will appreciate these works is fine; what matters is the catharsis they give to their creator. That’s something no machine could ever understand.

Tufts is an activist’s college. You can hardly walk across campus or read The Daily without encountering arguments about the Middle East, sexual assault, racial or class privilege, the LGBT movement, or any other social movement. Don’t get me wrong, these are worthwhile causes and injustices worth correcting. But nevertheless it becomes grating. Everyone at Tufts wants to save the world, and it’s tiresome. My housemate summed it up when she said, “there are eight sides to every coin”. My campus doth protest too much.

What if I’m tired of ambiguity? What if I just want to curl up into my own little corner of knowledge marked “Engineering” and make things that work? Unfortunately, I don’t get the choice.

Today was the final project presentations for my Visualizations class. The field straddles art and science. One one hand, we learn about the human visual system, the mathematics of multidimensional data, and the coding techniques behind certain graphics. On the other, there are many subjective design and aesthetic choices to be made. What questions do you ask of the data, and then how do you present them to the user? Success is cruelly defined by whether the target user can navigate and learn from what you produce, which is as definitive as any physical phenomenon but much harder to predict or explain.

Later in the afternoon I attended a colloquium by Mike Eisenberg (computer science and cognitive science, U Colorado). He spoke about education technology that does not involve traditional screens. He asked how we can design physical spaces and materials to be conducive to science and math education. My mind wandered upstairs to the electric engineering lab, where I had been the beneficiary of an obsessively-organized collection of resistors and capacitors in constructing a complex circuit for another final project. I then thought back to the machine shop where I built robots in heigh school, which was called “Chaos Central” for a reason. Tools did have their homes, mostly, but old robot parts hung from the ceiling and it was never truly clean. Much as I would like to fantasize about a large, organized workspace to host my own robot team years down the line, I realized that chaos is part of the equation, even (especially?) for engineering. Eisenberg read my mind, rather literally. He said that although math and science are rational disciplines, the paths into them are anything but. The apex of his talk used language quite similar to what I have previously used on this blog: if only we could find the perfect way to present material, the universal narrative, education would be a solved problem. But we can’t, because it’s idiosyncratic, subjective,  and personal. My head reeling, I wondered: how can we eliminate the stigma and misconception that the sciences are dull and austere? They are anything but.

Then, later that night, I attended a talk given by Daniel May, Director of J Street U. Try this one on for size: a Jew, speaking mostly to Jews, about the right of Palestinians to a state, because it was the right thing to do. (Liberalism defined). Some of the questions he fielded were from even further left. The conflict has been notoriously divisive, even by Tufts standards, so it was pretty jarring to hear reasoned and well-articulated criticism of Israel. The larger point was that this was not a utopian undertaking but rather meant to be merely a more perfect union, an improvement over the status quo.

I have nothing against those who take up advocacy of Cause X, but I realized tonight that it’s not my narrative. Zoom out enough and you get “righting wrongs”, a banner most college students will happy walk under. But zoom in just a bit on Tufts and you get “righting social wrongs,” and that’s not what I’m in to. I want to right intellectual wrongs. I want to change education, to end the stigma over science and math, and stop those who deny the ways in which we have made such tremendous progress. Homeopathy upsets me. The antivaccination movement infuriates me as much as any other activist trying to end the senseless deaths of children. And the fourth-grade “science” quiz that’s been circulating the internet makes me livid.

The secret to life is education. It’s really that simple. Yet I feel that my chosen institution of education, for all that I love about it, has a subtle yet core value not quite aligned with mine. Change doesn’t begin in a foreign country with one of the most intractable conflicts of human history. Luckily, Tufts realizes that engineering and education don’t take place in a vacuum. I can’t treat people as robots or build robots without thinking of people (although many do). On the other hand, I do not want to fight against every injustice every committed. There just isn’t time or energy. The political causes I support have objective grounding: if you can’t get someone to accept a fact, how can you ask them to accept a person? Everyone at Tufts wants to save the world, but maybe the engineers will actually make some headway.

The Jewish holiday of Passover requires adherents to not consume or possess any leaven during its eight days. Leaven symbolically represents excess, to be puffed up and arrogant. As I became increasingly aware of the time I was wasting on social media, I hit upon the mechanism that sites like these use to identify you: cookies. Which are leaven, right? So I decided to spend Passover without Facebook or Twitter.

It started small: I would stare at my phone where the apps had been idly looking to kill a few moments. Instead of needing to be on top of my newsfeeds, I found myself putting my phone away more readily and engaging in what I was doing. I began to notice details in the objects and buildings around me, but human contact proved elusive. Without sifting through other people’s half-baked ideas on an hourly basis, I felt like I had a tremendous amount of free time on my hands. The time I would have spent clicking “refresh” became time I spent walking around campus lost in thought. Unplugging led to a good deal of, not necessarily loneliness, but solitude.

Sometimes details matter, and a focus on trends leads to dangerous generalizations. The problem with social media is that it’s all details, with no underlying story or pattern to cling to. Our brains are taxed by trying to glue these shards together, but no two come from the same whole. Occasionally we manage to create a passable mosaic.

After all this introspection, I came to be more at peace with my religion. There are things that we know to be true, like evolution, and we should promote them. And there are things we know to be patently false, like homeopathy, and we should decry them. But a lot of things fall in the middle, not least of which is the human condition. The most visible parts of religion espouse certainty, but the Passover seder is really about uncertainty. It invites questioning and varying interpretations. Judaism embraces ambiguity and even contradiction. (Especially contradiction.) This is a welcome break from a world of pithy opinions crammed into 140 characters and relationships pigeonholed into a dropdown menu. It’s complicated, indeed.

Is slavery a presence or an absence? On one hand, I still feel that social media distracts us as a society from what really matters. It fragments our thought processes, and our relationships, by prizing efficiency over effort. On the other hand, technology connects and empowers us; without it we regress. Is social media a way to break down the interpersonal walls on campus, or a cause of their existence? Does the spread of information strengthen the spread of knowledge, or short-circuit it?

I have always felt that long-form writing helps me put together my thoughts on factual issues. In writing this piece, I realized that the same is true in the personal realm. True connection means giving people the freedom to think independently and then express themselves to others. Slavery is to uncritically compress and post whatever comes to mind.

“With a few simple lines of computer code,” boasts Moriarty in BBC Sherlock, “I can crack any bank, open any door”. (Paraphrased from memory, shh.) Without any spoilers, I can tell you that Sherlock’s nemesis is portrayed as controlling every detail, forseeing every possibility, and manipulating a web of individuals through blackmail, bribery, snipers, and sowing distrust. And, he makes vague claims of having the ultimate computer hack, stronger than any security system.

What kind of software would this be? Most of computer security relies on mathematics that is computationally hard. Consider a traditional padlock. If you know the combination, it takes almost no time to open the lock. If you don’t know the combination, you have to try every possible code. The combination is easy to check, but difficult to discover.

A completely general computer hack of the sorts Moriarty claims to have would be like being able to open a padlock without the combination as fast as you could with the combination. Sherlock operates in much the same way. Anyone can verify his string of deductions after he’s made them; his genius is to devise them in the first place.

So that’s what separates fact from fiction. These portrayals of genius are unrealistic because they take the same amount of time to produce a solution as it takes to verify one. Right?

Not quite. “Can any answer than can be checked quickly also be created quickly?” is one of the great unsolved problems of computer science. We don’t know.

I recently reread Neal Stephenson’s The Diamond Age. It’s a work of science fiction that depicts a future infused with nanotechnology, set in Shanghai and the surrounding areas. It offers some great material for a discussion on the role of technology in education and the limits of computers. Its themes are also relevant to edtech, which is pretty impressive for something published in 1995.

As a quick summary, Lord Finkle-McGraw asks engineer John Hackworth to create a computerized book (the Primer) to supplement his granddaughter Elizabeth’s schooling. Hackworth attempts to create a second copy for his own daughter illicitly, but is mugged and the book falls into the hands of the young street urchin Nell. The Primer guides Nell through leaving her abusive domestic situation and educates her using a customized fantasy story. Though the Primer is capable of reacting to voice commands and displaying a wealth of information, its narration is performed by a human actress Miranda whom Nell does not know. Hackworth, charged with intellectual property theft, makes a plea bargain to provide the source code of the Primer so copies may be distributed to tens of thousands of young abandoned Chinese girls. In the process of modifying the Primer to use a computerized voice, Hackworth is finally able to secure a copy for his daughter Fiona, before disappearing to serve his ten-year sentence.

That’s the first act. To do a proper analysis, I’m going to have to drop a few more spoilers from the most memorable parts of the book, so be warned.

Although we only see details about Nell’s Primer, it’s implied that the fundamental story is the same for all girls, just adapted superficially to its users. Nevertheless, Elizabeth, Fiona, and Nell develop quite differently. Lord Finkle-McGraw summarizes,

Elizabeth is rebellious and high-spirited and lost interest in the Primer several years ago. Fiona is bright but depressed, a classic manic-depressive artist. Nell, on the other hand, is a most promising young lady.

To elaborate, all three girls had previously served detention together at a strict Victorian-style school. Elizabeth eventually snapped and began tearing pages from the books they were being forced to copy, shouting, “I don’t care about any of the goddamn books, and I don’t care about the Primer either!”. From Nell’s point of view,

The reason she’d been furious was that copying out of those books was such an unforgivably stupid waste of time. There was no end to what she could have learned reading the Primer for those eight hours. For that matter, the normal curriculum at Miss Matheson’s Academy would have been perfectly fine as well. She was tormented by the irrationality of this place.

How often do children see schools as irrational places? Where the rules are arbitrary, the drills are pointless, and the work unengaging? And that’s before the current generation of babies, who have grown up stabbing tiny fingers at iPads, reach the classroom. Kids today expect interactivity and instant gratification. One approach is to yield, leading to 1-1 iPad programs, educational apps, and interactive whiteboards. The jury is still out on whether these are effective. The other is to hold firm, and insist on discipline, focus, and maturity. Students should be able to predict, anticipate, and imagine rather than see everything at once.

The novel provides no clear answer either way. It’s rough on harsh discipline, but more forgiving of other curricula. However, formal schooling makes a relatively brief appearance. What’s emphasized instead are the experiences the girls have. The well-to-do upbringings of Fiona Hackworth and especially Elizabeth Finkle-McGraw are held against them, while Nell’s impoverished and treacherous early years lends her a “feral alertness” and assertiveness. This gives the girls very different approaches to play and expectations of life, much to their parents’ bemusement.

The most energy is spent on the Primer itself. Its story’s finale begins with Castle Turing and its mechanical inhabitants, where Nell learns about Turing machines while pondering the Turing test. After ascertaining that her correspondent is in fact a machine, she crowns herself Duchess of Turing and learns to program her robotic subjects. She progresses through other castles, each featuring a more complex Turing machine (and making winking references to other computer science concepts). She climbs to the fantastical and heavily-detailed castle of King Coyote, where a machine called the Wizard (as in Oz, it’s implied) interacts with the other Turing machines in the Kingdom. On Nell’s code, Wizard overheats and forces out the man behind the curtain, King Coyote, aka Hackworth, who explains, “You snuck a zero divide past all my defenses.” (That’s unrealistic – it’s easy to detect division by zero. The weakness of Turing machines is that you can’t tell if one will ever stop running without actually running it.) Coyote shows that “when a message comes here from the Ciphers’ Market, I read it myself, and I answer it myself.” So, if Hackworth’s Primer avatar can be taken to be human, the system is actually not a Turing machine after all.

These human relationships are a key theme. Elizabeth’s Primer was narrated by hundreds of separate individuals; Fiona’s is implied to be narrated by machine (but with influence from her absent father), and Nell’s is read by Miranda alone. Taking her thoughts on Turing to their logical conclusion, Nell ponders,

In Castle Turing she had learned that a Turing machine could never really understand a human being. But the Primer was, itself, a Turing machine, or so she suspected; so how could it understand Nell? Could it be that the Primer was just a conduit, a technological system that mediated between Nell and some human being who really loved her?

Yes, the human connection, in this case with Miranda, who Nell finds at the end. The novel implies that for all the wonder of gadgetry, machines are fundamentally limited, and that a person is always superior. Mass quality education is therefore impossible. Referring to the tens of thousands of little girls and their Primers, another character explains

It was mistaken to believe that they could be raised properly. We lacked the resources to raise them individually, and so we raised them with books. But the only proper way to raise a child is within a family.

Sure enough, when we encounter these girls they are not given any characterization beyond their sheer multitude and efficient hierarchy, which is based, computer-like, on powers of 2. (Well, 4.) They are not seen as individuals.

Ultimately though, very little detail is given as to the design work that went in to creating the Primer. It’s implied that Hackworth did all of the AI himself, when he is never stated to have an educator’s background. One theme that the novel supplies, but does not develop, is the control of engineers over education. I personally don’t mind placing computer science at the pinnacle of knowledge, as Hackworth does, but this is a subjective choice that does not fit everyone. Instead, knowledge is personal, and education is as much an art as a science.

The holy grail of edtech, as some see it, is to automate teaching (as the Primers do). But both Stephenson and teachers agree that this is an impossible goal. Rather, careful thought needs to be given as to the proper balance between human and machine contact for our children.

The novel contains a clue to the answer. When Coyote shows Nell his personal involvement with the messages, Nell’s first reaction is to declare that Wizard “does nothing”. Hackworth corrects her, saying “Wizard does a great deal. It helps me keep track of things, does calculations.” Today we call this human-in-the-loop, and used properly you get the benefits of both worlds. That was how Hackworth constructed his most powerful invention (and I don’t mean Wizard). As the Primer notes, “King Coyote did not preserve his power by armed might but by cleverness, and sentinels were the only army he needed, information his only weapon.”

But information alone is useless. I’d like to leave you with a advice offered to Nell by a benevolent gaurdian:

The difference between ignorant and educated people is that the latter know more facts. But that has nothing to do with whether they are stupid or intelligent. The difference between stupid and intelligent people—and this is true whether or not they are well-educated—is that intelligent people can handle subtlety. They are not baffled by ambiguous or even contradictory situations—in fact, they expect them and are apt to become suspicious when things seem overly straightforward. In your Primer you have a resource that will make you highly educated, but it will never make you intelligent. That comes from life.

Here’s to a future of edtech that promotes not just education, but intelligence.

 

A generation ago, computers only understood text. You would program the computer in English text. You would ask your questions on punchcards encoding text. Your answer would be provided as monospaced, unadorned text. Since the early 1980s we have refined the graphical user interface, or GUI, to allow humans to communicate with computers on more familiar terms. Although a boon for the layperson, GUIs have been troublesome for computer scientists. They are hard to build because they are so open-ended. They are hard to test, because rather than printing a single correct answer there are many paths the user may take to accomplish the same goal.

Computer science still starts with a text editor and a compiler, because programming is better served by text. Text affords programmers absolute control over their programs. Written language is far more expressive than pointing and clicking, allowing for a explicit and precise descriptions. Clean code is a clear explanation of an algorithm directed to a mindless worker. The struggle of a programmer is to achieve sufficient clarity for both the computer and him- or herself. It can be a very enlightening experience, to debug an algorithm and then discover it doesn’t quite do what you wanted it to do, and so adapt it further. That said, the sheer austerity of the task can make it daunting without the right training and motivation on the part of the programmer.

GUIs are quite the opposite. They show many available options, reward experimentation, and make complex actions easy by hiding detail.  GUIs make computing accessible to a wide audience. A user interacts with a GUI as a peer, clicking and dragging and seeing how the interface reacts. Ultimately, convinced the GUI is logical and predictable, they embrace it as a new way of thinking. But GUIs are limited. They make it very difficult to perform analogous actions repeatedly or store a sequence of actions for later use.

There is an analogy to be made with education. Programming is like direct instruction, where knowledge is relayed linguistically and authoritatively. (No wonder Bill Gates and Salman Khan like it.) GUIs are like constructionism, where feedback loops reveal non-arbitrary behavior of a system that the user/student slowly begins to internalize. (So I constructed my own definition. How meta.)

Both methods of interacting with a computer are valid and potentially productive, so it seems both educational philosophies are valid as well. But there is a critical flaw in the analogy. For GUIs, students are analogous to the user and the computer is akin to some representation of the material itself: manipulatives, an experiment, a video, a graph or plot. But for text-based programming, the student is not the programmer; they’re the computer! The teacher is the programmer, the direct instructor, who crafts clear explanations of algorithms for the students to mechanically follow.

Direct instruction is degrading. It robs them of their ability, desire, right to explore and create. Knowledge transfer is not like copying a file, where we wait as it is methodically duplicated. Knowledge is personal, with idiosyncrasies and unique contexts. To insist on teaching children the same way we program a computer is simply wrong. It cuts to the core of what Dethorning STEM is about: our society treats people like computers and computers like people.

On a positive note, this analysis suggests that we should introduce computational thinking as another way for students to interact with the material in a constructionist setting. Having students write their own psuedocode for long division may be a viable way to teach it, if  it needs to be taught at all. Computational literacy will play an increasing role in the next century as computers become more ingrained in out lives. In the future, following an algorithm won’t be good enough — you’ll have to be able to write one.

Unfortunately, the state of computer science education is in shambles. Basic computer classes often teach how to use Microsoft Office by following rote algorithms — truly the blind leading the blind. Computer science itself takes a back seat to all other subjects, and is only sometimes offered as an elective. But I think that computational literacy does not require a computer scientist, a computer lab, or even a computer. It’s not content; it’s a technique. By cleverly inserting the right activities into the existing curriculum, teachers can cover computational thinking alongside any subject. Training teachers how to do that, and getting the administrators to sign on, will prove difficult.

A new, innovate approach is needed. One that breaks from the ossified red tape and small scale of the classroom and equally from the poor pedagogy underlying of most edtech products. The next generation of children deserve no less.

In a previous post, This is You: Agency in Education, I argued that we need to create educational software that, through extremely limited tools, forced its users to think in new and powerful ways. I used the term agency to refer to having an avatar (or agent) in a larger system with which one can identify. I’m sure educators have another word for this concept, which I will happily adopt just as soon as I find out what it is. I presented agency as one deliberately limited tool that can be used to focus thought. I also held up the video game Osmos as an example of how to use agency in compelling ways.

This time, I will present another video game as an example of taking agency too far: Spore. The highly-anticipated and much delayed 2008 title lets players evolve from a single cell into a space-faring civilization. The cell stage is the epitome of agency: the player’s cell is all they can control as they battle for survival in the primordial soup. Later, on land, the lines are blurred as players recruit other creatures to their posse and eventually control a planet-scale civilization with fleets and cities. So far, it’s set up to start with agency and gradually wean the player off of it into abstract, non-localized thinking.

But then the space stage starts. Instead of setting dials on dozens of colonies, establishing trade routes without worrying about the individual freighters, or sending fleets of identical ships into battle, the player controls a single spaceship flying around the galaxy. It’s impossible to wage a war or build colonies when existing colonies, and those of the player’s alien allies, need to be babysat and they’re half a galaxy away. The player should be Emperor Palpatine but instead they’re Captain Kirk.

What makes Spore so tragically flawed is not only does it fail to give players the controls they need, but it spends the bulk of the game teaching the player the cognitive skills to use the absent controls. Spore‘s galaxy of half a million planets is created using advanced procedural generation algorithms, but the game doesn’t offer its players so much as an if-statement. In this way, Spore is the opposite of most bad education: it provides a guided learning curve but no expectation or way to apply what was learned. Much of the learn-to-code movement provides plenty of tools, even innovative tools, but instructing their use is an afterthought. Neither of these symmetrical mistakes makes for a good user experience.

What Spore does right, up until liftoff, is to start with agency and move away from it. Agency is actually not a powerful idea at all; it’s too weak to support the space stage. Agency is a natural idea, one that is easy to adopt based on our physical bodies and evolutionary wiring. It’s a way to get comfortable with advanced concepts but it’s also really crippling when you want to apply those concepts. We need to find ways to provide learners of all ages with tools that are easy to learn and powerful — and rarely will the same tool be both.

I came across this 1996 paper that, as a thought experiment, took the principles of Macintosh user interface design and inverts them, just to see what would happen. For example, by accepting the Mac’s point-and-click interface, “it’s as if we have thrown away a million years of evolution, lost our facility with expressive language, and been reduced to pointing at objects in the immediate environment. Mouse buttons and modifier keys give us a vocabulary equivalent to a few different grunts.” They go on the show that metaphors can be crippling, direct manipulation can be tedious, consistency can be boring, and stability can be unhelpful. We expect our computers to stand still and not touch anything for fear of confusing us or breaking something, a relationship that paints the computer as an incompetent servant and the human as a weak-minded control freak.

What if your computer was actively helpful? What it it opened your mail every morning, and your webcomics only on MWF when they update? What if it cleaned off the 30 items on your desktop and put them into the right folders, and then changed the desktop picture to something it knows you like? This sort of user experience is extremely difficult to do well; it may even be AI-complete.

Nevertheless I feel that as computers become more prevalent and more capable our relationship with them needs to change. In 40 years nearly everyone will be a “digital native”, and this can be either a blessing or a curse. If we are locked into the interaction paradigms of our immigrant parents, we will be crippled by them forever. But we will reap benefits if we can raise a generation capable of enjoying lingual, contextualized, and diverse computing experiences.

What’s most interesting though is that the authors identify many of the user experiences seen today, more than a decade and a half later, on Linux (primarily) and Mac (ironically). Users encounter the desktop metaphor still, although with third-party apps creating some different experiences and branding. In the browser, the idea of contextualized locations is manifest, and it reflects reality. Different web pages look different because they are made by different people, much in the same way people have houses of different sizes, cleanliness, and decor. Far from having a single word processor interface, we deal with several on a daily basis, both in and out of the cloud. WordPress looks different from Gmail, which looks different from MS Word, which behaves very differently compared to vim. That’s okay, because they each have their own flavor and slightly different use case. Finally, the command line is resurgent, with new interest in programming, its tight integration with Linux, and the ubiquity of search bars. The command line is lingual, promoting automation and semantic meaning. Today’s user is presented with an arrangement of user interfaces, each tailored to a specific need, which in turn places requirements on the user. Or is it the other way around: our computers are “optimized for the category of users and data that we believe will be dominant in the future: people with extensive computer experience who want to manipulate huge numbers of complex information objects while being connected to a network shared by immense numbers of other users and computers.” Sounds like 2012 to me.

Operating in this diverse computing environment requires a large amount of cognitive work. We need to bring children through the evolution of language and thought quickly, teaching them to understand hierarchies, modularities, and contexts, the differences that characterize computing today. This will take time. The authors write that “it would not be unreasonable to expect a person to spend several years learning to communicate with computers, just as we now expect children to spend 20 years mastering their native language.” We should teach computers using computers. It’s time to use technology to strengthen and organize human thought. In order to do this, we need a new generation of educational technologies that not only do a better job teaching about math or history but also teach about thinking itself. Working a computer is a puzzle, and it can take years of practice. We need a way to simulate these nested challenges, and to promote the structured ways of thinking that solve them.

This is the opening of the ambient puzzle game Osmos, by Hemisphere Games. “This is you,” is all it says, as if you’ve always been a glowing blue orb. Most games start by introducing the player to their avatar, but it’s usually a human character with a backstory. Puzzle games are an exception: they rarely give the player an avatar whatsoever. Normally you play an unacknowledged manipulator of abstract blocks according to fixed rules and patterns. Osmos is an exception to the exception.

Osmos also has masterful use of player training and learning curve. It begins in the simplest possible setting with the simplest possible task: “Move to the blue circle and come to a stop.” You accelerate by ejecting mass, which propels the rest of you in the opposite direction. The game tells you these things in the same order I relayed them to you: first objective, then the means. Osmos could have said, “Hey, look how you can move by ejecting mass! Now use this ability to move to this outlined circle.” But it didn’t. The progression is guided, focused, and objective-based, especially at first. The levels build on each other, reinforcing knowledge from previous levels as the player gains experience.

Osmos presents players with the Odyssey, an sequence of levels that introduce gameplay concepts in a logical order. The Odyssey runs from the tutorial described above up through medium difficulty levels. After that, players gain access to a Sandbox mode where they can explore different game types at different difficulties. That is, a level of Osmos is distinguished not only by quantitative difficulty by qualitatively, by the kinds of game mechanics and obstacles found. More fundamentally, Osmos is played in discrete levels that can be won, lost, restarted, and randomized, rather than an endless arcade of increasing difficulty like Bejeweled. Players can skip to and play any level they have unlocked at will; a session of Osmos can last three minutes or three hours.

Players are incentivized to complete the Odyssey and get as far as they can in the sandbox, but there’s no climactic end of the game. No explanation is given why some levels seem to take place in a petri dish while others in orbit around a star; it’s wonderfully abstract in that regard. It’s impossible to “win” and there’s no victory cutscene. It’s neither so boring and open-ended you don’t want to play nor so scripted you only want to play once. There are achievements (badges) awarded but they seem extraneous to me.

And now to the point of all this: what can we learn from Osmos when designing software for education?

By the structure of its gameplay and incentives, Osmos lends itself to the sporadic and time-limited technology access found in many schools. Instead of leaving students behind who didn’t win the game, or trying to pry a child who’s “gotten far” away from the computer or tablet, it’s easier to take a break from Osmos. Meanwhile the nature of gameplay means that it’s very much a solitary experience, a personal journey of discovery. For all the hype given to social gaming over the last few years, it’s not conducive to deep thinking.

And yes: agency, in the sense of being a specific agent. In Osmos, the player is someone, or at least something: this is you. As I’ve said, most puzzle games don’t give the player anything to latch on to. Neither does formal arithmetic, nor algebra. Symbol manipulation provides no agency. It forces mathematics into the unnatural third person perspective (unnatural from a human’s point of view). When I played with blocks as a child I would often imagine an ant climbing and exploring my structures. Pen and paper mathematics allows the mathematician to move blocks around but not to be an ant inside his or her own creation.

Seymour Papert developed LOGO to provide children with agency. LOGO is a cross between a game and a programming language. Players manipulate the “Turtle” by telling it to turn left or right or move forward, where forward is relative to how it is turned. When children first encounter difficulties, they are told to ”play turtle”. By moving their own bodies through space, they are able to debug and refine their program. And by thinking how they move their own bodies through space, they are given a tangible grip on both computation and thinking.

Scratch was developed at the MIT Media Lab, which was co-founded by Papert. Scratch, though very much a descendent of LOGO, adds more commands to increase the user’s power and control. Many of the commands were discussed by Papert in his book Mindstorms or seem to be reasonable extensions of it. Others (thought balloons, sounds, arithmetic, color effects) are superfluous. Still others, like if-else statements, while loops, and Boolean operations, are taken from the nuts and bolts of programming. This comes at the cost of downplaying the two high-level skills which Papert thought were so vital to learning any subject: subprocedures and debugging. With LOGO, children learned to compartmentalize knowledge into reusable pieces, and to make incremental improvements to see the results they wanted.

One of LOGO’s defining characteristics was its limited set of commands, which are relative to the current position and heading of the Turtle. Osmos players can eject mass in any direction, but nothing more. In both cases, artificial scarcity of control forces users to think in a particular way. On the other hand, Scratch freely mixes LOGO-style “move forward” with Cartesian commands, both relative (“move up by”) and absolute (“move to”). It’s impossible to have agency with something that can be teleported across the map. Rather than force the user out of lazy and weak ways of thinking, Scratch offers multiple paths and users take the one of least resistance. Often this will be a hodgepodge of many different styles and systems of commands, reflecting incomplete and imprecise thinking.

The large number of commands create a cluttered and unintuitive interface. 78% of Scratch’s default interface is control while only 22% of it is the canvas. The results, the thing the user cares about, are squished in a corner. Osmos has minimal controls that disappear when not in use, leaving the entire screen as the portal into the game world. Moreover, Osmos has just enough visual detail to be eye candy and not clutter. Games, in general, have excellent usability because bad usability is by definition not fun.

The differences in the command set and user interfaces belie the different purposes of the software. Scratch is meant to provide a canvas for a play or a animation, and so gives the user plenty of options for control. Osmos and LOGO are both puzzles in the sense that the controls are extremely few, yet powerful. a A tool is designed to give a competent user maximum power to create; a puzzle is designed to teach new ways of thinking under constraints. By this metric, Scratch has more in common with CAD software used by engineers to design mechanical parts than it has with Osmos and LOGO.

But there is another feature that groups the three differently. Both LOGO and Scratch are sandboxes; they enforce no requirements or value judgements on the player’s actions. Papert envisioned a teacher guiding the student and keeping her on task. Osmos takes a different route. As a game, it has clear objectives to complete and obstacles to avoid. There are good moves and bad moves. There are levels, with definite beginnings and ends. The Odyssey is just a long tutorial: it presents each feature and some advanced ideas before handing the player full control. Scratch and LOGO do just that as soon as they’re opened. In particular, Scratch provides no guidance on its cockpit’s worth of controls.

There is a misconception, common among edtech types but not among traditional teachers, that the answer to all problems is better distribution. People are ignorant because they don’t have access to knowledge. People can’t code because they don’t have access to the software and documentation. But this is simply not true. Give people tools and they won’t know what with them to do or how to use them. Instead, we need to give students of all ages training, knowledge, and understanding. We need to force students to think about wrong ideas and make them right, and to see why they are right. We need to to show students the metacognitive tools to solve problems. An educational game isn’t about what to think, but how to think.

Pop. Sizzle. Spark. The unmistakable smell of burning electrical parts. “Shoot shoot damnit!” I say, doing the best I can to keep my voice down and not swear while the DJ is on the mic. I unplug the ethernet jack but it’s too late. The cable tester, a handheld device that tells you what conductors in an ethernet cable are connected, was fried. LED number 4 had exploded from being connected to line voltage, mocking my attempts to deny what just happened.

In hindsight, it was obvious the line from the air studio through the ceiling to the control room hadn’t failed, but it was part of a system that wasn’t working. We try to be classier than playing YouTube on-air, but we give DJs the option of playing the audio output of the studio computer’s web browser. According to the computer’s meters, it was outputting audio, but according to the equipment it plugged in to in the control room, it was not. So the problem had to be between the computer’s digital audio jack and the equipment in the control room, connected mostly by ethernet cables. Right?

“It was a perfectly reasonable thing to do,” my friend consoled me. He was a fellow computer science major but with less experience with how the station was wired, and I was hoping to show him how the cable tester worked. Those ethernet lines were a mystery, and I was hoping to get a better look inside. (Which I suppose I got – that line was definitely connected!) But I later realized what had gone wrong was that, on a fundamental level, I was a computer scientist trying to solve an electrical engineer’s problem.

Computer scientists like the break big problems down into little problems. We like to understand those little problems as a series of things that happen in sequence: first this line then that one. When something goes wrong, the crudest form of “debugging” is to pepper the code with output commands. We got here and the value is this. Do this with enough granularity and you’ll pinpoint exactly where something isn’t as you expect it to be.

This is very much the methodology I brought to the nonworking web browser output. I found the earliest and latest places something could go wrong and then honed in. The problem is that while output statements in code are free, electrified wires tend to break things. This sort of danger is completely alien to a computer scientist, or at least those of my generation. Yes, it used to be you had to wait hours for your code to compile and if you weren’t careful you could overwrite the OS, but those days are over. The focus now is on trying anything, seeing if it works, and improving it in a tight feedback loop if it does not. Drill down and get data, instead of careful planning and reasoning. That’s not necessarily the best way to work, but it’s viable when you’re dealing with unfamiliar code that isn’t too complex.

In this light, two computer scientists connecting a tester meant for very low voltages to a live wire is a perfectly reasonable thing to do.

I guess this is the part where I lament that even closely-related disciplines have widely varying problem solving techniques, and how a field’s techniques arise from its content, and how you should find a field that thinks the way you do. But no. I think more subjects should work like computer science. Starting with math.

Math, as currently taught, shares key similarities with electrical wiring: there are rules you shouldn’t break, but no feedback until you break them, at which point you can’t recover. Yes, those rules are in place for very good reasons, but they’re not always obvious. The feedback loop takes even longer: instead of immediately frying something, an elementary student may do a whole page of problems wrong with the same error, and only get the grade back until after the class has moved on to new material. What can be done to make math more like programming?

Quite a lot, as it turns out. Seymour Papert’s Mindstorms (after which the Lego robotics kit is named) describes a system of geometry that is fundamentally computational. Students create programs that guide a “turtle” moving on the plane that leaves lines behind it. Forward 50. Right 90. Children can kinesthetically identify with the turtle; they can relate to moving in 2D space far better than abstract symbol manipulation. But more importantly it turns math into executable code: the turtle’s path can be seen immediately. Errors are no longer obscure and arbitrary but obvious and meaningful. With the fluidity of digital text and the real-time speed of computation, students quickly debug both their program and their understanding in a tight feedback loop.

As much as this would benefit math education, the meta aspects are even more intriguing. I remember educational games of the 1990s, which involved following chains of item exchanges. You’d find a magnet, which you’d use to retrieve a steel crankshaft, which would operate a piece of machinery, which would contribute to the plot. These sequences were often discovered by trial and error, and shared among classmates; the objective was to beat the game. As Papert observed 15 years prior, “the computer [is] programming the child.” But with the turtle, “the relationship is reversed: the child, even at preschool ages, is in control: the child programs the computer.” Instead of following a trail of breadcrumbs laid out by someone else, with the turtle the child is free to set  and follow his or her own agenda. No program (that is well-formed) is ever “wrong” but merely does something undesired.

Papert makes the argument, one I’m only too happy to agree with, that the programmer’s mindset of debugging and incremental improvement taught by the turtle are a superior way of learning and thinking in general. This mindset rewards sustained effort over initial brilliance, and provides children with confidence to overcome their failures and set things straight. Equally importantly, “in teaching the computer how to think, children embark on an exploration of how they themselves think … [which] turns the child into an epistemologist.” A more positive relationship with math leads to an embrace of knowledge in general.

But we can’t stay in Papert’s Mathland, “a place that is to math what France is to French.” We have to work in the real world, with unforgiving electrified cables and math too complex to be explored with turtles. Our best option is to come into this world as prepared as possible, and to recognize that when mistakes do happen, they may not be as terrible as we thought.

For the $40 cost of a new cable tester, I think this lesson is well worth it.