I recently watched The Humane Representation of Thought, a conference talk by Bret Victor, where he discusses why and how we should design new representations of ideas that fit better with our human capabilities.

Bret highlights how, throughout history, intellectual progress has been connected to the development of new ways of representing and communicating knowledge. The Hindu-Arabic numerals and place-value notation expanded the space of possibilities in the world of mathematics. Better representations of the atom—from Dalton’s to Rutherford’s to Bohr’s—facilitated the advancement of physics and engineering. And the development of print and electronic media shaped how we learn, work, and share ideas. These forms of media changed not just what we thought and how we thought, but what was thinkable.

When we carefully study the development of these media, however, we see that they have led to an unbalanced way of living and working. Due to the widespread use of print and electronic media, what we call knowledge work today typically involves “sitting at a desk, interpreting and manipulating symbols on tiny rectangles”. We have rich, natural ways of interacting with and experiencing the world, but we have developed a culture constrained by paper and screen. To Bret, this mode of creating is inhumane: wasteful of the vast human potential.

Take music as an example. The most common way to interact with music is by listening to it. But the experience of music isn’t limited to the auditory alone. Through sheet music, it’s possible to also read music. If you’ve ever been in a loud concert, you know it’s also possible to feel the vibrations of sound waves. And if you play a musical instrument, you can feel music in a different way: you know what a B chord feels like, for example, from the shape your fingers make when you play it.

These modes of interaction produce different kinds of experiences and each one is meaningful in its own way. We wouldn’t choose to limit our expression of music to any single form—like, by only reading symbols on a sheet of paper. But that’s not very different from how we program today. By far, the most common way of writing, reading, and interacting with software is by manipulating symbols.

What would a more humane representation of ideas look like?

We could use pictures and videos to communicate how programs work. But while these formats improve on text in some ways, they are still largely static and rudimentary. Instead, Bret proposes three different dynamic representations of media for teaching and learning.

First, we can move abstract concepts into physical space with dynamic spatial media. A good example of this would be creating a physical representation of a book as a room or a museum. A “reader” of this "book" would interact with its concepts by moving around the room and manipulating physical objects. In the case of software, this could mean using physical, instead of abstract, objects to write programs, taking computation out of the machine and into the physical world.

We can also create context-sensitive learning models. Learning materials, like books and blog posts, typically expect the reader to have some background knowledge relevant to the material. Alternatively, a more humane version would be sensitive to each reader’s unique backgrounds and interests. It would be aware of what each reader knows and adapt to their needs.

Bret also proposes what he calls “explore-the-model” reading material. In this media, the author isn’t just lecturing the reader, but guiding them through exploring dynamic models. Compared with unresponsive formats like text and pictures, the reader will be able to engage directly with the learning content itself.

It was here that the lightbulbs lit up for me. Without being able to put it in such clear terms, I've attempted creating explorable models here on this blog. Some articles I've written, like Text Search with Tries and Quadtrees in the Wild, have inlined interactions where the reader can play around with the material being taught.

By clicking and dragging on controls and watching the visualizations, the reader augments syntax with the visual and the tactile and can gain a richer understanding of the subject. I fully agree with Bret that models like these can help improve the way we teach and communicate programming. If we build better tools that capture the richness of our human capabilities and make better use of our means of understanding and experience, what new possibilities will that open up?