Spatial Computing and AI: Competing Inflection Points

February 15, 2024 Leave a comment

I clearly remember when the original Macintosh was unveiled in 1984. It was more than just a piece of technology, it was a bold declaration by Steve Jobs that computing was now for literally everyone. With its graphical user interface (GUI), the Macintosh turned what was once the exclusive domain of the technically adept—taking orders solely through arcane text commands—into a immediately accessible and intuitive visual experience. At the time, I recall that this leap was not without a good number of skeptics, many of them with vested skills who liked the inherent barriers of entry to computing. Others at the time believed the GUI to be a mere gimmick, a dead-end in the face of “serious” computing models.

I was a budding software designer myself back then, and I had plenty of experience on the command-lines of the day. But when I started to use my first Macintosh, I realized almost immediately that we had now entered a completely new era. An era where computing was going to be forever visual. I was struck at the time how the visual interface was really a high bandwidth conduit right into the human mind. There was so much more you could experience than with just a screen of characters, and it was so much faster. And it went both ways, input was infinitely more exploratory and revealing. Syntax errors were replaced with easily-clicked functionality.

What’s Old Is New Again: Symbolic Interaction

Yet, as a software architect and thought leader who has witnessed and contributed to the myriad transformations in technology over the last forty years—including the eventual rise of mobile and social platforms to the current era of AI-as-UI—I can see the original Macintosh was more than just a significant footnote in history but the harbinger of the many fundamental shifts that would follow. The 40-year journey from the playful but profound simplicity of the initial Macintosh to the sophisticated and similarly very nascent capabilities of today’s equally visionary Apple Vision Pro neatly bookends a period of unprecedented innovation in visual interfaces and reimagining of human-computer interaction.

In short, the GUI genuinely did what it set out to do: It truly democratized computing. It made technology accessible and personal, laying the groundwork for the subsequent mobile revolution that would bring computing into our pockets with new, even easier user interfaces made of touch and voice. Social media platforms capitalized very well on these intuitive interfaces, connecting billions and embedding computing even deeper into the fabric of daily life, for the better and then sometimes, to my regret, for the worse.

Throughout these shifts, my role has steadily evolved from designing systems within predefined paradigms to questioning and redefining exactly what those paradigms should be. Is there an end-state to this journey? Where are spatial computing and AI actually taking us? Just asking this question will reveal vital insights to aid our passage, a topic of my current research.

Today, we are now fully in the resurgence of the command line, powered and made far more workable again by AI. It marks a fascinating turn in this ongoing evolution. Far from the hard-to-master command lines of the past, today’s AI-driven interfaces are as approachable as the GUI, but often more profound: Conversational, capable of understanding and generating sophisticated natural language.

AI-as-UI represents a full digital realization of the most ancient and powerful mode of human interaction: Symbolic dialogue. Unlike the static command lines of old, these AI interfaces are dynamic, learning and adapting to the user’s needs in real time. They democratize computing in a way the original GUI only hinted at, tapping into radically advanced computational capabilities now accessible to anyone with the ability to ask a question.

Maximization Is Not the Path: Simplicity Is

However, as we marvel at the capabilities of AI, I believe it’s crucial to consider the places where the GUI—and its three-dimensional evolutions in virtual and augmented reality—may have overreached. The push towards ever more immersive 3D environments has, in some instances, prioritized spectacle over substance, complexity over clarity. This is not to dismiss the potential of spatial computing in any way. I believe it’s a major part of our future. But I wish to caution against losing sight of good user experience in the rush towards technological maximalism. As many designers would tell you, the most profound innovations often come not from adding complexity but from removing barriers, a lesson the Macintosh taught us four decades ago. It’s one of the reasons that I’ve added simplicity maximization as one of my most important trends for the Future of Work in 2030.

Our User Experiences Risk Getting As Complex As Our Minds. Image: DALL*E 3

Amidst these reflections, I find that in my research that the role of AI stands out not as a competitor to spatial computing but as a highly complementary force. AI has the potential to make spatial computing more intuitive, more nuanced, deeply personalized responsiveness to our needs, and ultimately more, well, human. By integrating AI’s understanding of language and context with spatial computing’s immersive capabilities, we will create effective environments that are not only visually and interactively rich but also deeply personalized, far more potent to use, and highly accessible.

From where we are today, the intertwining paths of AI and spatial computing is what actually represents the cutting edge of digital experience. They are not at odds, but are instead two sides of the same coin. Each will push the other towards greater heights and major new breakthroughs. Importantly, one (AI) can return simplicity to the other (spatial computing.)

No, the challenge and opportunity for us as architects of these digital realms are to ensure that these technologies properly reinforce and complement each other in ways that enhance human capabilities and enrich our lives. Our immersive experiences will have AI interfaces. In the end, it’s as I’ve started to remind people lately, if all this tech is not here to serve humans well, it does not have much of a purpose.

So, as we chart this course, I’d like us to remember the lessons of the original Macintosh and now the Apple Vision Pro and ChatGPT as well as crossovers like Mobeus: That the true power of technology lies not in its complexity but in its ability to connect with us, to understand us, and to make the once unimaginable a part of our everyday lives. To the extent spatial computing and AI manage to keep us on this road towards progress, they will flourish and open up the future.