Why Microservices Will Become a Core Business Strategy for Most Organizations

September 7, 2018 1 Comment

As an industry, we have collectively returned to that eternal debate about what constitutes a largely technical evolution versus when an important digital idea becomes a full-blown business trend. This has happened before with Web sites, e-commerce, mobile applications, social media, and other well-known advances. It can be hard to remember that at first these were looked at as mostly technology sideshows. Yet they all went on to become serious must-have capabilities on the business side.

Microservices is now a current topic of this debate, as the overall approach is perhaps the most strategic technology trend that’s come along in quite some time. First, a brief definition: Microservices provide a well organized digital structuring of our business capabilities that are exposed to stakeholders who need what our organizations can do, and are usually accessed via open APIs. The concept is now poised to — sooner or later — become the primary digital collaboration fabric with all our enterprise data, IT systems, 3rd party developers, business parters, suppliers, and other stakeholders.

So, you read it here first: Microservices are how most organizations will eventually conduct the majority of their business, internally and externally.

Yet there is still considerable debate and confusion about whether microservices are merely just slightly more elegant network plumbing of our digital systems, of if they actually represent the primary conduit for operating our organizations. I fall in the latter camp, as this platform way of thinking in general has steadily emerged as the leading model for composing and integrating networks of systems and organizations. Don’t get me wrong: We had SOA, Web services, and APIs before — where I once posited that this would turn into a global service phenomenon, which it has — but these each had key details missing or not quite right. At this time, microservices does appear to be the best model we have, honed and culled from over a decade of thousands of organizations experimenting with various approaches.

I am now also clearly seeing from many of my CIO and IT contacts that developing a microservices strategy is rapidly becoming a key priority this year. Not sure that this is broadly the case? Just take a look at the recent JAX Enterprise IT priorities survey, which shows that microservices are currently the 3rd leading IT priority, nearly eclipsing the big trend on the block, cloud computing, one of the other hottest IT topics of recent years.

Yet microservices are often conflated with concepts like APIs, for which there is indeed a considerably close relationship, and so can often be relegated to the ‘we’ve been here already’ bin.

Why the sudden popularity and interest in what appears to simply be a more refined technique to easily integrate and communicate between digital systems? For almost all the same reasons that the Business of APIs and the API Economy had their days in the sun: Microservices take so many of the lessons learned in creating more composable, reusable, and platform-centric version of our digital organizations, strips them down to their very basics in terms of design and consumption, and then places them at the very center of how our organizations operate. (Note: Not everyone would agree at the strategic level that microservices should be designed and offered at the business domain or architecture level but many, including myself, do.)

Naturally, the question is why would we do this, and why would it be just about the most important thing we could do to enable a host of vital business activities and outcomes? Put simply, microservices hold the promise of truly unleashing the greatly underutilized assets of our organizations, both strategic and tactical. These assets include everything from data to talent to innovation, and up until now, we’ve been doing it piecemeal and without a real enterprise-wide design (though I’m cautious about overly top-down efforts here as well.)

Microservices: Building Blocks of the Modern Digital Value Chain

Microservices, by virtue of offering a well-structured way to engage and integrate with the world at large in scalable, digital terms, now appear to hold the answer to enabling faster digital transformation, lowering our levels of of tight coupling and technical debt, and substantially increasing much needed levels of IT integration. More centrally to business impact and growth, they also make it possible for us to build and cultivate bigger and more robust digital ecosystems with our stakeholders. This includes 3rd party developers and business partners to our very own workers and customers.

For me, I first saw the writing on the wall several years ago when I was helping develop the API strategy for the CIO of one of the largest organizations in the world. We had just completed an all-day workshop studying the benefits of opening up systems and data more simply and easily to make them as consumable as possible. I stressed these key points: 1) Open APIs make it far easier to create and innovate on top of existing IT and data, 2) they make it easy to create additional value many times over through nearly effortless integration between systems, 3) they achieve this asynchronously and highly cost effectively by systematically designing a high leverage and productized point of global interaction upfront, instead of hundreds of expensive point-to-point integrations over time. Upon reviewing this, the CIO suddenly sat back, the light clearly having come on, and said, “I get it now. The logical conclusion of all of this is that we need to provide a URI for every piece of data in our organization.” He was exactly right.

Put simply, this means that every element of enterprise data would have a unique link to it through a well-defined interface, which anyone can easily find and use to (yes, securely) access it and update it if appropriate. As I’m fond of saying, civilization advances when formerly difficult things become easier. This is exactly the vision behind microservices: Build and provide an incredibly simple and straightforward way of exposing our businesses in a highly useful and constructive manner so that the effort to connect systems into value chains becomes essentially near zero in practical terms.

Mindset: What Would Happen If Anyone Could Build Anything On Your Business?

The question I then put to those still trying to understand all this is the following: If we could access all our enterprise data simply and easily and could then integrate systems together with just a few lines of code, what could we do this with power? Virtually anything we can dream of, with almost no economic, technical, organization, or political barriers to achieving whatever we — or, and this is the big key, others — could dream of doing with our systems and data.

Because once strategic microservices that enable this are operational, then anything is possible. That’s because virtually all of our enterprise data can be reached, it can be harnessed, analyzed, and it can flow through to wherever it needs to be to extend and empower the stakeholder/customer experience. In fact, it’s the most potent way we know of yet to create and capture shared value and to do this so efficiently that literally orders of magnitude more high value integrations, connections, and innovations will take place (see: How Amazon Web Services makes most of Amazon’s profit.)

So why hasn’t this happened except in organizations at the very leading edge of the digital maturity curve? Because it takes 1) an understanding of the vital — even existential — importance of doing so in order to rapidly gather around a vibrant ecosystems of app creators, integrators, partners, suppliers, customers, and stakeholders and 2) the pre-emptive removal of the aforementioned economical, technical, organizational, and political barriers to doing so. In short, creating microservices, though they themselves are profoundly elemental network-accessible business capabilities to our organizations, takes real work, much of it consisting of softer, non-technical obstacles in the realm of culture, mindset, inclination, and leadership.

We already see examples of this happening at the enterprise vendor level. A particularly compelling example of a global set of microservices that expose much of what an organization does is Microsoft Graph, along with their microservices-friendly Service Fabric. While some will quibble with whether MS Graph is a set of microservices in the pure sense, the point is this: Much of what Microsoft offers its customers via its products is accessible within a well-organized enterprise-class set of data services. This is strategic to the point that Sayta Nadella has even called Microsoft Graph their “most important bet”, for all the previously cited reasons.

Microservices are also well established at some of the leading organizations in the world, including Amazon, Netflix, Uber, and a good many others. Less clear is traditional enterprise adoption at the strategic level, though my personal anecdotal evidence is that this is now very much underway in a growing number of organizations. Another proof point of expected growth is that business consulting firms like Deloitte are seriously talking about microservices as enablers for open banking and other industry transformations.

Microservices and Business: The Future

However, in today’s extremely fast-moving world, coming to the conclusion through a largely accidental and piecemeal route that microservices are the future will simply take too long from a competitive standpoint. This will result in a very much less than optimal set of services for your stakeholders. Thus, my advice on microservices in the enterprise is currently this:

  1. Most organizations should now begin a concerted effort to create an enterprise-wide set of microservices. And do it as a part of an overarching business strategy.
  2. This effort should be decentralized but a centrally coordinated effort. To be used to identify and design needed microservices.
  3. A commitment must be made to be in the business of integration and dynamic digital value chain building. Half measures have long-doomed efforts at SOA, APIs, developer networks, etc.
  4. Use design thinking to understand the needs of microservices consumers, then meet them. Understanding what the needs are, and being deeply empathetic to key issues like ease-of-use, performance, and the right to build a 3rd party business on them is key.
  5. Operate your microservices like your core business. Because they soon will be. Invest in them, advertise them, evangelize them, encourage usage, support them, and generate revenue with them.

A growing number of organizations I work with, including most recently one of the largest federal government agencies in the U.S., are now fully cognizant that most of their business will soon be conducted through digital channels. That aforementioned agency is already doing over a quarter of its business through APIs, and expects it will be over half in the next few years. They believe moving from data-based APIs to business-oriented microservices is their next task to go to the next level. So should it be for most organizations.

For the enterprise, achieving success with microservices is certainly possible through a patchwork of department APIs that are designed and operated without an overall business strategy, design, or structure. Or we can adopt a holistic microservices approach to create a more uniform, rational, consistent, and contextual set of open digital capabilities that also forms the basis of business strategy and architecture for the organization. The story is unfolding rapidly, and as I mentioned, I’m seeing an all-time high interest in microservices at the most strategic IT levels. Now that story must be told, understood, and realized on the business leadership side as well.

Update on September 20th: A few commenters have noted that they don’t think that most organizations believe microservices and APIs are actually viewed as business strategy, much less core to it. However, supporting many of the assertions I make above, I recently encountered a recent study from Cloud Elements. Their 2018 integration survey (which included 400+ companies, 27 industries with 26 outside of tech including finance, communications, engineering, and transportation on 6 continents) reported that 61% found APIs to be critical to their business strategy, and 85% fundamental:

APIs (open access to microservices) is Essential to Business Strategy

Additional Reading

My current Astrochart for the New C-Suite: Microservices figures prominently as a key C-level technology and business strategy

A Discussion of the Past and Future of Web APIs with Dion Hinchcliffe | InfoQ

How can businesses keep up with tech change today? | ZDNet

Filed under Analysis, Digital Transformation, Enterprise Architecture, Strategy, The Enterprise, Trends and Statistics

Why the Underlying Laws of Cloud, Social, and Digital Business Matter

May 25, 2015 1 Comment

I’ve spent much of my Memorial Day Holiday here in the United States pondering the Red Queen effect vs. network effects, seminal laws of technology and business both, that are often held as gospel by their adherents who believe they are the natural and intrinsic properties of their operating environments (digital ecosystems in this case.)

Both hypotheses apply to any connected, relatively closed environment of some kind, which very much includes everything from traditional marketplaces, online communities, app stores, cloud services, SaaS, enterprise social networks, social media, and cable/video networks to e-mail, telephones, and package delivery. I cite these, as many practitioners don’t even consider system concepts when trying to figure out why they aren’t succeeding as hoped, despite those very ideas defining the rules of the playing field they’re working upon.

Both Red Queen and network effects have years of rigorous research and thinking to back them up, and as far as they go, they make real sense of the modern world as explanations for why things are the way they are in various places. This includes how organisms compete, survive, and thrive in the natural world or how networked products and services grow and maintain dominance on the Internet, respectively. These combined environments will ultimately lead to what I’m currently calling the 4th Platform.

Digital Business: Network Effects and Red Queen

The issue here is looking at why there are so many so-called ‘winner-take-all’ players at the top of the technology industry, especially on the Internet. In the shadows, nowhere near as successful, there are usually hundreds — sometimes thousands — of also-rans, depending on the sector, who struggle mightily, but usually fail. Rarely are the winners dethroned, though it does occasionally happen (see: AOL, MySpace, Friendster, RIM/Blackberry, all top digital players in their day.)

What triggered this line of thinking was Gartner’s fascinating new report on the Magic Quadrant for Cloud Infrastructure, which makes the stunning claim that Amazon now has 10 times more cloud computing capacity in use than the entire total of all the other 14 providers combined. The company is deeply wired into over a million customers’ ecosystems, and thousands of online products. By any estimation it is uncatchable.

Sidebar: Interestingly, this hasn’t stopped technology leaders such as Staples CIO Tom Conophy from stating last week that they’re going after Amazon “in a big way.” This despite the fact that both organizations aren’t even playing on the same digital meta-level.

Or is Amazon uncatchable? Don’t we just need to go ‘up the stack.’ As it turns out, not if it just leads to commoditization.

If traditional enterprises — and yes, more likely, startups — can only understand the fundamental rules by which the Amazon, Apple, and Facebook et al became leaders, the reasoning goes, we should be able to use those very rules against them. We should then be collectively able to out-innovate, out-invest, out-maneuver our competition and get to the top of an increasingly shallow but very, very steeply inclined pile of winners.

Deep thinkers in this space like my industry colleague Simon Wardley note that network effects create enormous inertia for change that make it very hard for a new player to get started, even if they’re much, much better. Worse, this is true even if you realize this and try to organize and optimize for it. You are simply spitting in the wind. Market-leading network effects, in fact, nearly always easily trumps the seemingly-powerful Red Queen Effect.

None of this is a new discussion of course, and the power laws of digital networks, such as Reed’s Law, have been relatively well-understood for at least decade. But their effects are now inexorably felt in the traditional business world like never before.

Using Networked Relationships Built on Enduring Digital Values

Ultimately, as organizations increasingly perceive the imperative to put a digital layer over their entire organization, even completely re-imagine it for the new models of digital business, there are some key fundamentals that we have to remember to close the gaps:

  1. Count on nearly everything in the ecosystem to eventually drop to effectively zero cost. Do not build your digital business along the commoditization axis. Only the underlying properties, like network effects, are exempted from this rule and this is a potent realization. We will eventually get the cost of nearly all technology to nothing, making replication and competition both easy and rampant. This isn’t where most businesses would like to be. However, network relationships like communities, total aggregate API integrations of partners, total daily users contributing value, these cannot never be copied whole cloth on an increasingly low cost scale. When people are involved, commoditization is not possible, and real, meaningful human relationships then become the strategic coin of the digital realm.
  2. Use differentiated models for commodity elements of digital business, versus the methods you employ for the innovative (and most important) elements. Social business, bi-modal IT (actually, tri-modal), devops, and even agile, are still the mantra here. These methods have been discovered to operate on the scale, speed, and collaborative dimensions required of today’s rapid digital cadence. Even better, they connect people the best way — that we know of so far — with how modern technology profoundly alters and — mostly — improves the way we work.
  3. If you are a large organization, you’ll have to operate in sometimes truly unexpected new ways to deliver well on digital. In particular, this means powerful new network models of organization structure — in IT and everywhere else — as well as new widespread digital workplace skills to be able to change at a sufficiently high and sustained rate.

    4. In other words, most organizations must rapidly evolve their operating models to adapt to the rules of the digital networks. A few have, but most still have not. There’s still time in a few industries however, but not a tremendous amount. For most, however, it’s now later than they think.

Filed under Analysis, Enterprise Architecture, Network Effects, Social Business, Strategy, The Enterprise

Going Beyond ‘Bolt-On’ Digital Transformation

May 28, 2014 3 Comments

Much has been made recently of the imperative to fully transition our businesses into the modern digital world. It now hardly needs to be said at this point. There is even some encouraging news for traditional enterprises: The latest data from Forrester shows that companies are indeed at long last making digital transformation a top priority, with 74% of executives saying that they currently have a strategy to get there.

Yet “having a digital strategy” can also mean just about anything, depending upon who you ask. At this point however, there are basically two main forks in the road to digital for most organizations:

There is the ‘bolt-on’ strategy, which typically means adding a few new digital channels to existing touchpoints — typically social and mobile — and maybe creating an associated but minor sideline business with some digital revenue.

Then there is the ‘digital transformation’ approach to digital. It’s a full-on, meaningful reconception of the business, often using a startup or incubator model, with the intent to re-imagine a digital native organization with all that it entails, from new business models, culture shifts, remodeling of the structure and processes of the business, and rethinking of the very foundations of the enterprise across the full spectrum of digital possibility.

Enterprise Digital Business Transformation

Unfortunately, the latter approach also has many of the characteristics that corporate leadership tends to avoid: a) The big bang initiative which has a high likelihood of failure, b) cross-silo involvement, meaning it will encounter numerous bureaucratic and political obstacles, and b) the likelihood of of success being dependent on securing rarefied talent with scarce expertise that crosses the domain of the business, the world of strategic emerging technology, and next-generation IT.

The reality is that both forks have real risks: The bolt-on approach is too little and too incremental to have the requisite strategic impact, though it’s certainly a valid interim approach (as long as it’s not the only one.) On the other hand, the full digital transformation model entails a major investment and commitment across the organization with a seemingly all-to-uncertain outcome.

Yet, the latest data tells us unequivocally that the act of doing nothing — or just too little — is also sure to fail. The march of technology is wiping out traditional companies faster than ever before, and the pace is only accelerating.

Another way of putting it is that the CEOs, CIOs, COOs, and CMOs — the four roles most directly responsible for guiding this transformation — will secure rewards for their organizations that are directly commensurate with their commitment to drive broad digital adaptation and change. For the data is unambiguous: Those that don’t fully align with the state of the marketplace will be absorbed by those that do.

Forrester Digital Business Strategy Not Yet Business Strategy
Many industries even today are resistant to digital. Source: Forrester

Thinking Like a Digital Business

What can organizations do if they are serious about their responsibilities to lead the business into the future? Several clear options are emerging:

  • Seek out digital change. Avoid having it imposed. Successful next-generation enterprises — see the start of my 2014 NGE target here — won’t wait until adopting new digital channels, tech, and business models are unavoidable. They will pro-actively seek them out, learn them early on, experiment, and be ready to grow when they mature. Even fast-followers will be at risk if they don’t avidly seek out new opportunities. Dave Gray has previously pointed out research from Shell showing that the longest-lived companies are pro-active seekers and explorers of new markets. What’s more, digital change is now nearly continuous, and the organizations must establish long-term processes that tap into and pull these changes into a new “digital metabolism” that makes incorporating strategic innovation both routine and sustainable. Organizations that only respond to change will always be several steps behind those that are change-seekers. Finally, be bold it seeking out these changes. As the latest McKinsey report on digital transition notes, the winners will “be unreasonably aspirational.”
  • Cultivate capabilities to support multiple operating models. As John Kotter pointed out this week, there is now “an inseparable partnership between hierarchy and network.” We will have two and probably more major operating models in our organizations going forward, at least the legacy and the digital. We must operate and exploit each of these systems to their fullest — and together — to produce competitive and effective results today. To get there, successful leaders will strategically enable the shift of hierarchy into much more network-centric models, while cultivating the strengths of both simultaneously. Since most organizations currently have significantly underdeveloped networked operating models, this will require special investment and integration into the digital transformation process.
  • Understand and absorb the new competencies of digital across the organization. If one thing stands out clearly when I look at digital transformation efforts, is that they are often led by those who are experts in the existing business, who often don’t have the competencies in the digital space. It’s not that it can’t be learned, but it is a fast-growing and already enormous field. The profound difficulties that many transformation efforts have encountered, despite the vast on-hand resources including thousands of workers and millions of customers, has been to the distinct boundary of and very different rules for success between legacy business and digital business. I recently summarized what many business leaders don’t quite get right in their mindset and assumptions when it comes to digital transformation, but it boils down to deeply understanding and emulating what those successful in digital have done to get there. Understand the power laws of digital business, deeply absorb the concept of engaging with and co-creating new products and services with digital ecosystems, and wielding powerful new ways to scale innovation.

This is not to say that businesses have not already extensively digitized. They have, but as Sameer Patel recently pointed out, they generally have not transformed. The single biggest obstacle to successful digital transformation is a broad shift to a ‘native digital’ mindset that will consistently inform broad action. I’ve come to believe that traditional companies can make this transition, but only if they decentralize tech innovation that is coupled with a supportive new network operating model, while carefully controlling downside (typically security, data control, etc.).

So, while bolting-on a digital mindset may lead to some short-term successes, it will certainly stunt the future of your organization. Instead, employ internal and external networks to create a naturally-supportive environment where digital change is far more scalable, emergent, adaptive, and continuous.

Additional Reading:

The New Top Level Operating Models of Business

Digital diaspora in the enterprise: Arrival of the Chief Digital Officer

Filed under Analysis, Crowdsourcing, Enterprise Architecture, Peer Production, Strategy, The Enterprise, Trends and Statistics

Defining the Next Generation Enterprise for 2014

May 12, 2014 Leave a comment

Many of you know that over the last several years I’ve tried to make the case that most organizations are currently falling behind the advancing pace of technological change. That business is so centered around technology today is the reason why addressing this has become a top competitive issue. Becoming better adapted to tech change is even tied to the medium-term survival of many organization as I recently explored in my look at digital transformation.

But to say that technology alone is what is disrupting traditional businesses would be inaccurate. We ourselves have changed — have co-evolved — along with technology. Our mindsets have become expanded by the new possibilities of super-connectedness, new models of working, and pervasive data-based insight that today’s networked revolution has wrought.

That’s not to say there aren’t important pros and cons to these advances as well. Along this journey of global, open, and social digital networks, we’ve also encountered enormous challenges in grappling with issues such as individual privacy and equal access, as well as the inherently large inequalities that emerge from the gaps between the digital haves and have-nots. This is precisely because technology is a profound force multiplier of just about everything it touches. There are other potential worries as well.

As The Economist fretted over recently, most technology revolutions have created more employment, not less. We hope that this is true for the next generation, but we’ll see, given how current models show that producer power is generally moving outside of traditional organizations to external networks that have less well-defined employment models:

Everyone should be able to benefit from productivity gains—in that, Keynes was united with his successors. His worry about technological unemployment was mainly a worry about a “temporary phase of maladjustment” as society and the economy adjusted to ever greater levels of productivity. So it could well prove.

Yet to most of us, it’s quite clear that digital channels combined with engagement at scale within them amongst all our stakeholders is at the core of the future of business. But what does this actually mean? What does it look like to most organizations? How can we articulate the changes to structure, process, and management of our organizations in a deeply digital age? It’s my belief we need a comprehensive yet eminently understandable model of how all this reshapes our organizations.

Ecosystem View of the Next Generation Enterprise for 2014: Workforce Community, Customer Community, Partner Community, Market, Social Business

I’ve come to realize we’re trying to hit a fast-moving target with poorly aging models for service delivery and IT governance when it comes to digital transformation. The reality is that it usually takes several years for a large organization to achieve large scale change. By this I mean three to five years, and often more, and that’s just for an individual enterprise-wide initiative.

In today’s operating environment of yearly — sometimes quarterly — waves of highly disruptive enterprise technologies and products, that’s just too long. We need a clearer and more updated sense of where we need to take our organizations, and it must also show us how to increase our technology metabolism as well. This model should include the broad strategic outlines as well as specific adaptations to the latest powerful new digital capabilities such as big data analytics, omnichannel customer engagement, the Internet of Things, social business, and so on. These subjects are all highly strategic to the future of our organizations at the moment, yet they are also interrelated and must fit together relatively well in this model somehow.

Related: Digital Business Ecologies: How Social Networks and Communities Are Upending Our Organizations

Motivation for an Infinitely Renewing Model of Tech & Business

Over the last few years, I’ve adopted a term known as the next generation enterprise or NGE for short. It’s the idea that we can maintain an up-to-date strategic model when it comes to digital transformation. The vision for the next generation enterprise is different from one year to the next and has specific technological phases as well as overall strategic themes at any given time. This vision has its own management theories as well, such as shifting from organizational hierarchy to networked community or reorganizing how we operate to the three new top-level modes, to name just two examples.

In other words, the idea of the next generation enterprise is a relatively complete view — including both business and technology — of the target that typical organizations should be aiming for in their objectives for digital adaptation and growth. For the moment, let’s put aside whether there even is a typical organization, since many of the most important technology innovations are usually agnostic to your particular industry or unique company attributes. In other words, most major technology advances will derail your boat if you ignore them long enough, no matter what business you’re in.

To give us a shared roadmap and a point of reference, I’d like to start putting a clearer definition behind what we think is meant by a next-generation enterprise. Early this year, I mapped out the most important strategic new enterprise technologies, but it was a purely technology view and included a good many tactical elements that aren’t that important when it comes the big picture.

Instead, I’d like to have a more enterprise-centric view that includes the most important advances in business that technology has directly enabled. Some would say that the advent of being digital connected to every human being on the planet at all times (at least in the developed world) is one of those advances, and I agree. This realization is that communities are moving increasingly to the center of our businesses. But it’s more than that. The enclosing strategic conception is really one of ecosystem, whether that’s inside a segment of the enterprise with a single networked team, an external customer community, or a full-on developer network of thousands of application development partners who have welded your digital supply chain to their apps. All of these are ecosystems that must be created (or identified), grown, cultivated, managed, secured, and governed.

In fact, one of the largest issues we have in digital transformation today is that we look at business in a far too simplistic traditional model. In this legacy view, there are functional silos with workers combined with management hierarchies that together actually make decisions and operate our organizations. Then there are suppliers, business partners, and customers, and that’s about it for the big moving parts.

Baseline for Next Generation Enterprise 2014: Networks, Communities, and Support Programs (Social Media Center of Excellence)Today’s next generation enterprise plays on a much larger and more complex chessboard. There are thousands of relevant ecosystems that now exist for most businesses, most informal, and across thousands more channels, all with a long tail structure.

This means that while the head of the distribution consists of big channels you’ve heard of — from major social networks and call centers to traditional media and Amazon’s cloud — there are thousands you haven’t heard of and will never be able to deliberately consider and plan for. Business architecture has thus moved from simple planned models to complex and highly dynamic emergent networks across every business function we have. We’ve gone from a few dozen groups of stakeholders to ultimately tens of thousands that we must still manage to somehow. Ultimately, our org structures must adapt to reflect this.

I’ve previously proposed a set of enterprise strategies which have a good chance at addressing many of this issues, which were originally brought forth by the channel fragmentation, scale, and decentralization that we saw greatly exacerbated by IT consumerization a couple of years ago. But I now see that bring-your-own-device was just the forefront of a wave of grassroots led network-enabled change, including bring-your-own-application, bring-your-own-community, and soon, even bring-your-own-workforce.

Related: Designing the New Enterprise: Issues and Strategies

The Element of The Next Generation Enterprise for 2014

So I’d like to put a stake in the ground and define what I think the next generation enterprise for 2014 should look like. There are several views here, but I’ll start with the more business-centric view of ecosystem and expand to other views as I’m able. In this ecosystem view we have the following components:

  • A more network-centric enterprise. Less hierarchical and consisting much more of online communities for achieving cost-effective outcomes at scale. This will happen within and amongst the workforce (network/social collaboration), business partners, customers, and the marketplace. Management and leadership through networks will become an essential skills and will require knowledge of the concepts and operation of digital and social businesses.
  • Workforce communities. While we’ve had a primitive model of team in the legacy workplace, it becomes much more fluid, dynamic, and high scale in the networked world, often directly supported by powerful new collaboration capabilities. Teams-based, project-centric, and — still evolving — process-based work conducted by communities will increasingly become the norm. Why? Because the data has consistently supported that the network/community model provides better business results.
  • Business partner communities. One of the least developed models of networked communities, there are however good examples that can be pointed to. Strategic partners, affiliates, and suppliers can be engaged together in operations, in particular — as John Hagel famously pointed out — with exception handling scenarios.
  • Customer communities. This is one of the strongest and most easily started models for strategic community. The evidence for business value is strong enough that I’ve wondered if the window is already closing on customer communities in certain industries. Certainly in my research I’ve found that customer care communities can reduce costs by 30% in the first year alone over traditional approaches. Social support also at the very top of Ray Wang’s social business use cases.
  • Marketplace. The single most scalable asset that businesses have is networked access to their customers and the broader marketplace. While this constituency also includes regulators and influencers, two groups that can be hard to manage, it also includes online advocates, crowdsourcing participants, software developers, and other interested parties. If you’re surprised to see developers in this list, then don’t be: Developers have become one of the single most important new constituencies as their innovations can drive primary growth and network effects. This is a very different view of business than before, where companies directly engaged their stakeholders.
  • New channels. The next generation enterprise will still have some legacy aspects including physical offices/stores — just smaller and more virtual — it will be the Web and especially on mobile devices that value is primarily created and captured, both. Social business environments (communities of all functional types and audiences) and the application as the new CRM will be key channels here as well. Ultimately, however, APIs — which I define as open digital supply chains — will be the most strategic channel for many industries because it scales faster and creates far more robust outcomes for very little investment.

Using this model, we can also baseline the various states of maturity of each part of the modern enterprise ecosystem for comparison, as in how far along are we? The essential point here with this view of the next-generation enterprise is that it’s the current target model, not what you should look like today. It’s what you should be aiming for, although you should certainly have some elements of it in place today (see figure 2.)

What do you see as other essential views of the next generation enterprise? What else needs to be added?

Related: The Second Wave of the Contemporary Workforce

Filed under Analysis, Crowdsourcing, Enterprise Architecture, Network Effects, Social Business, Social Media, Strategy, The Enterprise Tagged with

Designing the New Enterprise: Issues and Strategies

April 5, 2014 5 Comments

I’m looking forward to traveling to Paris, France the week after next to provide the opening keynote to the Intersection Conference. Intersection is an intriguing new multi-disciplinary event organized by Milan Guenther that’s intended to explore how we should design our organizations for the future. Milan also wrote a terrific book by the same name, which I urge you to read as well. The write-up for the conference itself says it best:

The role of design in economy and society is shifting. We see disciplines such as Service and Interaction Design moving beyond individual services and their digital components, to tackle experiences between enterprises and their audiences.

Enterprises and entrepreneurship are everywhere, playing a vital role in our lives. They are ubiquitous in the mass of organisations of all sizes we are in touch with as consumers, employees, investors, or in other roles.

This event is about designing the new enterprise, making it less awkward and more humane. We will explore how to design enterprise-wide brand experiences, social organisations, and digital businesses. To do this, design practitioners, consultants and architects combine methods and models from Service and Experience Design, Information and Enterprise Architecture, Systems and Design Thinking, to drive innovation and transform complex enterprise ecosystems.

Those who’ve followed my most recent musings on how our organizations are trying to change to adapt to a rapidly transforming world, and being changed even more by external forces outside their control know this has been a prime focus of mine. Technology in particular is the author of so much of what is reshaping markets, communities, corporations, and even our cultures. The growing question is whether our organizations will break under the loads of so much change, or are there paths we can navigate and steps we can take to transition more gracefully?

Additional Reading: What is the Future of Work?

Business Architecture: Change and Designing for the Future Enterprise

It’s very clear that most companies feel an imperative to update how they operate to match the current state of the marketplace. But as I pointed out in ZDNet recently, the data shows that the average lifespan of the enterprise continues to drop steadily, due to poor adaptation to the latest marketplace conditions.

In addition, to make matters worse, due to misalignment between our constituents’ respective goals, we also see that most workers in the typical enterprise are generally are poorly engaged. Only 40% of workers are ‘well-engaged’ according to recent analyses. This is a unacceptable state of affairs, but we’ve really only ourselves to blame. I believe we can do better.

How do we adapt sustainably to constant change?

The big question is there an intersection between change, the role and function of businesses, and the future of work that will allow us to adapt more readily? Can we still do this while creating an environment that enables far better and more satisfying work and outcomes for everyone, including employees, customers, and yes, even shareholders? What skills must be brought to bear to realize a fundamental restructuring — in the face of the many major new modes of work — for how we absorb, adopt, and manage the external march of change facing us — and therefore frequently imposing serious business challenges — that we are encountering at an ever accelerating pace?

Personally, I believe getting to equilibrium between our organizations and external change will require a specific set of modifications to the core of most of our organizations. First, achieving this will require a much-more pragmatic and decentralized view of business and enterprise architecture. It will also require that we honestly look at technology and how we metabolize and assimilate it into the way we work and then seek ways to reconcile with it. We’ll also need to cultivate workers that have effective skills in design thinking, which they can use within and across the organization to locally redesign it for current realities. And finally, we need to update how we collaborate in a fundamental and more focused way, and by that, I mean all modes of collaboration between all stakeholders.

These then are the issues I’d like to explore in Paris in just over a week. I hope you will join me. Milan has assembled an all-star cast for this discussion, including corporate strategist, author, and futurist Chris Potts, branding, innovation, and design expert Erik Roscam Abbing, and the lead designer for Dassault, Anne Asensio, to name just a few of the leading thinkers who will be speaking and collaborating there. Either way, I encourage you to join this vital conversation.

You can view the deck for my keynote talk at Intersection 2014 here.

For more background, you can also view my Slideshare profile with sampling of my latest keynote decks on this topic.

For More Information:

Intersection Conference - Paris, France - April 16-17, 2014

Filed under Analysis, Enterprise Architecture, Strategy, The Enterprise, Trends and Statistics

Enterprises and Ecosystems: Why Digital Natives Are Dethroning The Old Guard

February 26, 2012 2 Comments

Why is it that so many traditional companies with an enormous wealth of assets largely fail to transform them for the digital era? By assets here, I mean established customer base, closely held relationships with trading partners, mountains of data and IP, as well as their bread and butter, the actual products and services they offer. For large organizations, these assets typically represent many billions in long-term investment and accumulated value that is being stranded beneath a digital ceiling they cannot seemingly break through. The lesson has been a hard one: It’s been surprisingly difficult for many companies to make a genuine transformation to digital.

For those just joining this conversation, this transformation is about opening up and digitally enabling the strategic assets of our organizations for better consumption and participation, with as low a barrier as possible. It’s also means doing so in a way that continually maximizes their value over time in today’s deeply networked marketplace. Achieving this triggers the primary engine of growth for digital ecosystems, namely network effects. This is how Apple, Facebook, Google, Amazon (new guard), and Microsoft, IBM, SAP, Oracle, and many others (old guard) eventually built hundreds of billions in combined value. They tapped into the relevant power laws of networks by carefully and deliberating cultivating and then closely managing them by harnessing peer production over the network.

Digital Business: Cultivating and Managing Digital Ecosystems (Open APIs, Social Supply Chain, Web Services, SOA, Online Communities, Peer Production, OEMs)

How exactly was this accomplished? They did it by digitally platforming their businesses in specific ways: Enabling self-service on-boarding, viral adoption, open participation, best-of-breed data capture, ownership and control, and took advantage of the fact that relationships — and therefore, ultimately transactions — must take place on the network with as little friction and cost as possible. They realized that we are now all connected together continuously in a single global network and then designed their organizations around this central fact of the digital age. They are now reaping the results of this mindset:

Networked ecosystems must be a core focus and competency of modern business.

This begs the increasingly urgent question: Why then are a large number of older organizations neglecting their digital ecosystems, often failing to meaningfully cultivate them at all for many of their most valuable assets?

This is a key question that fellow Enterprise Irregular Vinnie Mirchandani recently asked in an internal EI mail thread and later posed on his blog:

But for every Apple which has gone one way, I see so many others piss away this huge asset that is their ecosystem. I hear about musicians and filmmakers auditing, even suing studios for accounting disagreements. I hear SAP mentors complain about legal issues getting licenses and other access to new technology. It’s easy to dismiss Joe Konrath’s litany of complaints against book publishers as one from an unhappy author, but the 230+ comments it has drawn shows a deeper angst about how poorly publishers are managing their author ecosystems.

Ecosystems, communities – call them what you want. They are a vibrant organism which deserve far more ink from all of us. And they need professional managers at the companies at the middle who nourish them and not just treat them as railcars to be hustled away whenever inconvenient.

As companies remain inadequately connected to their customers, partners, and workers via digital ecosystems, many of which they do not control, they are missing a rapidly narrowing opportunity. That’s because it’s very, very hard to disrupt a well-established network effect, which is much more powerful than the equivalent notion in the pre-digital era: traditional market share. Network effects are primary focused on pull distribution, while marketshare is heavily based on push, which is much harder and much more expensive to sustain. As digital natives sew up more and more industries and lay down network effects years ahead of their traditional brethren, any chance to reclaim the throne will be very unlikely.

For leading examples of potent digital ecosystems, see open APIs, social customer care, and app stores.

Why is this? There are a number of reasons but a few are particularly significant. What I wrote in the EI thread in response to Vinnie’s original question was this:

I find that in general, the farther you are from the tech business, the less native skill or familiarity there is with system thinking, which is perhaps the critical capacity to have in order to regard your business in ecosystem terms. This is something that in tech is standard fare with constant discussion and focus on platforms, network effects, SDKs, open APIs, app stores, etc.

Traditional publishers are typical of the technically challenged industries that are being blind-sided by newer, much savvier, techno-centric, network-oriented new digital businesses.

Business leaders that can’t deeply see the way forward for their organizations as flexible, highly dynamic, and organic digital networks of customers, partners, workers, and other (likely and unlikely) participants will ultimately fail. But this isn’t the set of skills or mindset that made them successful in the first place, so they don’t value it and don’t think in these terms. They literally throw off digital rethinking like a sort of corporate immune system. Surprisingly, from my talks in the C-suite the last few years, everyone individually seems realizes they have to change, but collectively they are resistant, it’s fascinating to watch.

A big part of the problem boils down to this: Companies are inherently designed to perpetuate the problem they were invented to solve. It’s a particularly thorny instance of the Innovator’s Dilemma, which ensures that a company is unlikely to aggressively re-invent itself until it’s in the process of being disrupted. Unfortunately, this often means it’s already too late.

In fact, it may be too late for a growing number of industries to fully make the transition to being ecosystem-centric. This includes media, publishing, telecom, retail, and many software companies. Under looming threat is real estate, higher education, financial services, professional services, accounting, and even venture capital. In each of these categories, ecosystem-centric firms are building network effects with open network-based products increasingly built by worker/open communities and delivered to customer communities. Those products are in turn built upon by hundreds or thousands of loyal 3rd party partners to bring their own customers and ecosystems to the table. This is an embarrassment of riches that only a few companies, again mostly digital natives, seem interested or able to tap into.

As Fred Wilson once said, the Web (and therefore digital business) is all about “building networks on top of networks“, which leads to even more powerful outcomes, like 2nd order network effects.

Fortunately, the force multiplier of the ecosystem model can be stated in a simple, fundamental way: It allows one to tap into the vast size and strength of the external network to drive growth, innovation, and revenue for your own ecosystem. As Peter Kim and I wrote in our new book, the fundamental principle of business in the ecosystem era must be to let anyone participate in every aspect of the business, primarily by inverting the facilitation process of driving shared value (i.e. network effects by default.) Being able to elicit the network (Internet, community, shared data, whatever) to maximum effect to fuel and growth your ecosystem is thus the core competency of the digital era. Unfortunately, this lesson is being lost to most organizations that were built well before this next-generation business model was understood. It will be a great loss that doesn’t necessarily have to happen in my opinion, but will ultimately result in the needless disruption of a large number of companies that just aren’t able to become digital natives.

For additional reading see:

4 Ways to Create Sustainable Business Ecosystems

Why Information Power Is The Future of Business

What Will Power Next-Generation Businesses?

A View of Digital Strategy in the Ecosystem Era

Are We Building Businesses Or Are We Building Platforms? Yes.

How Digital Business Will Evolve in 2012: 6 Big Ideas

Filed under Analysis, Enterprise Architecture, Network Effects, Peer Production, Strategy, The Enterprise

Consumerization: Why the Workplace of Tomorrow Looks Like The Internet

November 18, 2011 1 Comment

The title of this post is almost right. The workplace of tomorrow will look like a lot of things actually, including the Internet; just not a whole lot like the way our organizations look today. For one, the workplace itself has steadily begun to disappear as teleworking becomes more and more prevalent, though the latest data shows this will take longer than other more imminent changes. These other disruptive forces, such as next-gen mobility, social networking, cloud computing, and big data, are so close at hand that most organizations are already extensively affected by them. It’s not a stretch to say they are eclipsing how IT is applied to business in many ways, even as IT shops are significantly underestimating their current impact, according to brand new research from Unisys.

Over the last couple of years, I’ve been following this set of closely interrelated trends, each one that began “out there” on the Internet or in the consumer world, and have little or no roots in the enterprise world. It’s this singular fact that induces in so many IT executives and business leaders a profound feeling of disquiet. Yet the ones I’ve spoken to this year realize that they have to respond to these changes. Why? Because technology innovation today is driven mostly by the Internet or the consumer world, yet technology is one of the leading ways we use to automate and drive productivity improvements in business. High technology — and particularly the fundamental architecture of the Internet — also has an innate tendency to dislocate the old ways of working. It tends to tear down the traditional — yet less effective — means of operation, along with their associated cultures, norms, and expectations. However, it’s fair to say that no one being held to a quarterly earnings cycle or holding a market leading position vulnerable to technology change (media, software, travel, education, etc.) likes to experience dislocation. So it’s up to organizations to get (much) better at realizing an effective digital strategy, just as innovation and change is happening much faster than any other time in human history.

Recently, the phenomenon of “CoIT” has been growing. It’s a new concept that says that the adoption of IT is now proceeding rapidly outside of the CIO budget, often in entirely unsanctioned initiatives by lines of business. In its more mature form, CoIT also stands for a much closer yet decentralized notion of IT where innovation and technology leadership is driven on the ground by the business, yet supported by IT. The business — as well as IT — brings in the latest new cloud services, mobile apps, APIs, data sources, and mobile devices. IT then makes it safe, secure, and manageable, or provides guidelines for doing so. It’s a smart, efficient, scalable new partnership. The former is the “Consumerization of IT” while the latter model is the “Cooperation of IT”. Both are represented by the moniker, CoIT, which was originally coined by Computerworld Editor-in-Chief Scot Finnie a little while back.

Clearly there’s widespread interest in the topic, as one of my most popular writings this year was the exploration of the “Big Five” IT trends of the next half decade, one of which is consumerization, for which it could be argued it’s actually an encompassing supertrend. All of this ultimately culminated in a gracious invitation by Eric Norlin to come and present my research at Defrag 2011, which I did last week.

Below is the deck itself, which I gave as a keynote last Thursday morning:

If you don’t have time to review the deck, the key points to take away are the following:

5 Strategic Points about CoIT

  • Evidence is growing that current productivity gains aren’t coming from traditional IT investments. They are coming from somewhere else, or the cost of IT is collapsing radically. Almost certainly both are true by comparing slides 3 and 4.
  • There is far too much new tech for any centralized process (like IT) to absorb. New types of processes must be created that can unleash and scale the application of powerful new technologies (next-gen mobile, social business, cloud computing, big data, etc.) to the business..
  • If the only real constant is change, change must be in our DNA. But these ‘genes’ are usually not present in large enough quantities in the enterprise. This is the concept of moving from fixed processes to dynamic relationships embodied by the Big Shift in order to transform the enterprise as we know it.
  • Some changes will be more transformative than others. While mobility is the hot topic right now, social business and big data will have the largest long-term impact and especially the former will have truly game-changing and transformative consequences.
  • Ten to hundreds of times more apps and data are coming soon, get ready for it. Cultivate the skills, create enterprise app stores, build social layers into the organization, define decentralized enterprise architectures (really, business architectures), and create a new CoIT playbook. Or this will all route around you. 30% of IT is already outside the purview of the CIO and growing fast.

I’ll be exploring this more soon with new data and examples. In the meantime, I’d love your thoughts on where you are seeing IT going in a rampantly mobile, social, big data world. In addition, here are 10 strategies for coping in the CoIT era.

Filed under Analysis, CoIT, Enterprise Architecture, Next-Gen Mobility, Social Business, Strategy, The Enterprise, The Web

Exceeding the Benefits of Complexity? A Fractal Model for the Social Business Era

September 12, 2011 10 Comments

Over the weekend my friend and industry colleague JP Rangaswami wrote an insightful post that pondered how we have gone about delivering on customer experiences as connected to our back-end capabilities. Specifically, he explored an issue that is increasingly challenging many of the large-company CIOs I speak with these days: That the present rates of change demanded of the accumulation of 20-30 years of legacy business systems is greatly exceeding the ability of our enterprises and associated software “stacks” to deliver on them, particularly as cloud, social, and mobile dramatically transform computing today.

The problem lies in our classical views of enterprise architecture and business architecture both. But JP puts it more poetically:

Development backlogs are endemic, as the sheer complexity of the grown-like-Topsy stack slows the process of change and makes it considerably more expensive to change. The stack has begun to fossilise, just at the time when businesses are hungrier for growth, when the need to deliver customer-facing, often customer-touching, applications is an imperative.

Which makes me wonder. What Tainter wrote about societies, what Shirky wrote about companies, are we about to witness something analogous in the systems world? A collapse of a monolith, consumed by its own growth and complexity? As against the simpler, fractal approach of ecosystems?

This fractal aspect of user systems, Web 2.0, and SOA is one that I deeply explored in the 2005-2007 timeframe, and my ideas on this even made the cover story of the SOA/Web Services Journal at one point. This is when we began to see successful, composite systems sprouting up “in the wild” that were eminently natural and highly successful, such as Amazon’s, Flickr’s, and many others. Innovative new open API-based businesses had definitively emerged and shown us — in fact, virtually proved to anyone willing to observe — that business/technology ecosystems could be routinely produced that were far more successful than the ones virtually any traditional business had managed to produce for itself internally with methods like SOA. It was clear something important and new was happening and we tried to learn from it.

Traditional Closed Business Stack vs. Open Networked Supply Chain API

The signature lesson in this time period was that being “in the business” of ecosystems, at multiple levels, was the key to resilience, growth, and sustainability. It’s still a far cry from how most organizations are structured today, but I do see the first hints that this is starting to change. Companies like Best Buy and Sears have been getting the message among the old guard firms and looking at integration, partnership, and engagement as an open network activity.

The Web has also shown us that complexity, as important as it is to address and resolve the many inherent vagaries of dealing with the real world, is largely the enemy, whenever it exists needlessly. Our assumptions of where the complexity should be, in the transports for example, was wrong. It was in the ecosystem itself and we paid dearly for half a decade (and running) based on that misapprehension. But we’ve learned. SOAP usage has almost completely shifted to REST, complex WS-* stacks have largely moved on to simple standards and methods like Web-Oriented Architecture.

But technical discussions obscure the very important truths about what we’ve learned in the interim between the “aha” moment when we realized that what was happening so successfully “out there” was something that would make us just as successful inside our organizations, for internal and customer-facing needs both. After all, this holistic ‘integratedness’ — usually triggered via our connections to the Internet — was a part of our businesses increasingly and we simply had to realize this and engage as Internet natives. But back then, Andrew McAfee worried that focusing on the plumbing was the wrong emphasis to have on the exciting changes taking place, and he was largely right from a communication perspective.

However, building fractal architectures that addresses the old and the new isn’t as easy as just deciding to cast off the old, less effecive ways, as JP notes:

What I’d established in my own mind was a growing belief that the issue was to do with rates of change and costs of change. Vertical integration paid off when the rate of change was low. Networked small-pieces approaches paid off when the rate of change was high.

We are clearly living in times of much greater rates of change. Unfortunately, the old systems, the old architectures, the old business “stack” is often running the core of the business. It’s usually deeply vertically integrated, and not made of small networked pieces, loosely joined that are truly agile, deeply harness innovation on the edge, and turn IT into a profit center instead of a cost center.

So, while I have a lot more to say about this, because I believe that social business context will be at the root of the future of our how we design our products, services, processes, and workplaces, below are a few basic rules you can take away that if you stay true to, can indeed help you make the transition to the future. Note that customer and worker experiences are outside the scope of this, this has more to do with the new business stack below the social layer:

4 Ways to Create Sustainable Business Ecosystems

If resilient, networked, recomposable, open business capabilities are the future of business and IT architecture, getting there then requires a substantial change in stance:

  • Open Your Business Stack. To everyone, internally and externally, generally in the order of the systems and data that are most often requested for integration. Make it easy for anyone to onboard themselves and use the simplest and most egalitarian technology possible to deeply connect the world to your business. Self-onboarding is crucial because it enables the killer asymmetric business advantage of systems that respond to external engagement best: Export most of the change effort to outside your organization and impose it on those that wish to participate and integrate with you. Enterprises that close the “clue gap” with Big Data will have an enormous advantage.
  • Go Into the Business of Ecosystems. Stand behind your open supply chain: Invest in it for the long term, market it, evangelize it, support it, and stand behind it like your customers — again, internally or externally — will be building world changing products and services from them (if you do this, the track record is that they will.) Be fair and generous with your IP and enterprise data; you are under-utilizing it by several orders of magnitude until you do this, leaving much or even most of the potential of your business on the floor. Worse, until you treat it like a business, you won’t get external contributions to enrich your ecosystem that will drive down the cost of change while greatly increasing the velocity of new solutions and local adaptations.
  • Begin Fractal Reconciliation of Your Classical Systems. While hanging a trendy Web API off an ancient COBOL accounting system might be a step in the right direction, long-term success will mean going well behind the API. The existing business/IT landscape must be converged and rationalized so they themselves are made of “networked small pieces” that are faster to change as well as more resilient and responsive to being so much more connected (and therefore useful and valuable) to the world.
  • Spin Up New 21st Century Business Models. All new possibilities, many of which will be foreign to non-digital native firms, are reachable once you have created a fractal enterprise stack. Dominating classes of data, metered business knowledge, all new monetization methods, and more become possible once the three transformations above have started.

Unfortunately, after having advised and directly assisted many large companies in doing much of what I’m describing here over the years, it’s clear that most organizations are at a major disadvantage in making this transformation. The organizational will required is actually quite low, since most of this is fairly easy to do now with cloud technologies, increasingly capable technology partners, and the latest development stacks. Instead, it’s the understanding of how important it is to kill excessive complexity and focus on ways of thinking about business and technical architecture that will propel companies into bright futures.

We must leave behind vertical integration and radically collapse the size of our seams (APIs) in our businesses, while radically increasing the number of useful endpoints such that we look at them as the very fundament of our enterprises. The benchmark for success is how many others are depending on your ecosystem for their own success. Starting now, the activity in your organization must be focused on optimizing for growth of this number. That is the only way we’ll ensure we’re building businesses that are the most relevant, meaningful, and valuable to us going forward. The enterprises that don’t simply won’t survive in their present form. I believe that much is at stake here. Unfortunately, the “cultural gap” John Hagel wrote of back when this conversation started between the enterprise crowd and the Internet crowd is still much too large and is still hindering the move forward for many organizations.

I look forward to continuing this discussion and exploring just how much we’ve recently learned about the future of business and enterprise architecture.

Filed under Analysis, CoIT, Enterprise Architecture, Network Effects, Strategy, The Enterprise, The Web, Web 2.0

A Timeless Way of Building Software

January 10, 2006 Leave a comment

Most of my readers know that I’m a software architect by trade.  I’ve been creating software large and small for over twenty years.  And I’ve experienced movement after movement in software design from object-orientation in the 1980s and early 90s to component-based design, distributed objects, Web-based software, service-oriented architecture and too many others to even mention.  I’m pretty jaded at this point because I’ve learned, in general, the problems that you couldn’t solve in the previous generation of technique are often only marginally more solvable in the next generation (which is invariably invented to "fix" the previous problems.)

Alas, a genuinely better mousetrap is really hard to find.

So in the end, if you couldn’t do whatever it is you wanted to do with the previous generation of technique, it’s actually not that likely you’ll succeed in the next.  Certain software problems remain hard, and in general, it mysteriously happens to involve the juncture between technology and people in some way.  To paraphrase this, I could say that the software and techniques get better fairly constantly, but people remain the same.

And please, bear with me because I will attempt to connect a great many dots in a moment. 

Because every once in a long while, something new and big actually does come along.  Or at least something that looks new and big.  One of the new and big things that came along about ten years ago was the concept of design patterns.  It was pretty neat stuff.  It said that despite the current technology we have, the processes that continue to evolve, there are certain timeless solutions to certain software design problems.  It was a revelation at the time.  And the writers of the book that explained this got both famous and very successful.  Why? Because these design patterns really worked is why.  And anyone who has read the books and has ever really built software recognizes these patterns.  And what was strange was that no one really expected it. One day, we just had them.  And the kicker was, they were always there, but now they were in our conscious thought and we had real names for them.  My point: They were in our face all the time but most of us couldn’t see them.

We are in a similar place with the Web right now.  We’ve done this Web stuff enough now that we are just beginning to see the design patterns.  What works, and why, in a specific situations, bounded by forces.  Some folks have had the hubris to give this next generation a name and to tease out these patterns.  Some people are just now going aha, and some people haven’t got it yet, and most of the rest of us either aren’t ready for it or just haven’t heard of it.  But, I will tell you this.  It’s quite real.  The best practices and design patterns of Web software are just starting to become understood.  The strange part is, we’re discovering the same things over again.  What’s old is new again.

Now, before you get all worked up or worse, I bore you and you stop reading, I will give you a nice list of the the forces behind these patterns.  If you recall, design patterns are a solution to a problem in context.  We are starting to get the context and even the outlines of the patterns of this "new" generation of software.  But we have a long way to go still.  The Web is a monstrously big space with big problems, and it’s not getting better.  There are one billion of us out here now.  Clearly understanding what it takes to create great software on the Web that is successful, useful, and vibrant will be an ongoing challenge for a long time.  But it will get easier because we are codifying our knowledge of this exciting and huge place where we now find ourselves.

Comparing SOA, Web 2.0, and a Timeless Way of Building Software
Figure 1:  The driving forces in modern software.
With a rough comparison between SOA
and The Timeless Way (Web 2.0 by any other name).


So, here is where we take a quantum leap to the next level of thinking.  I’m going to use Christopher Alexander’s opening chapter of a Timeless Way of Building and tailor it to describe this old-but-new way of building the Web and software for it.  We are lacking for a little inspiration and this book in particular continues to sell upwards of 10,000 copies a year, 25 years after it was frst published.  And Christopher Alexander, for those of you who may not know, was the person that originally discovered the design pattern.  But it wasn’t for software.  It was for creating great, timeless buildings.  He was one of the first that realized that his field of endeavor has certain elemental, timeless cores, no matter the technique, building material, or the people.  It was an amazing discovery that poured over into the software world with considerable success. 

My assertion is that nothing has really changed in software, we might understand the forces better but they are almost always the same.  People want software that does what they want, is available when they need it.  They want software that grows with them, helps them, teaches them, and lets them do the same with others.  They want software that gets out of their way, disappears, and is more convenient by far than inconvenient.  And they want to pay as little as possible for it, but enough so that it’s worth it.  They are willing to have software get right into the middle of their lives.  If it’s the right software.  And as long as we’ve had software, they’ve always wanted this. But now they might actually start getting it.

In any case, I don’t literally believe every phrase in this take-off, but I do believe the overall concept deeply and profoundly as a software professional.  And I will continue to update the diagram above (clearly marked beta 1) until we have more of the forces in it. And some are definitely missing.  Please, as always, leave any comments and suggestions for improvement below.

And now, without further ado, here is the The Timeless Way of Building Software, with sincere apologies to Christopher Alexander:

The Timeless Way of Building Software
Inspiration For The Next Generation of Web Software


There is one timeless way of building software.  It is decades old and is the same today as it’s always been.  And because it is timeless, it will always remain this way.

The great software of our time has always been created by people who were close to this way.  It isn’t possible to create great software – software that is satisfying, and useful, and makes itself a natural extension of life – except by following this way.  And as you will see, this way will lead anyone who looks for it to elegant, vibrant software which is itself timeless in its form.

It is the process by which the function of a piece of software grows directly from the inner nature of people and naturally out of the raw bits, the otherwise meaningless digital medium, of which it is made.

It is a process which allows the life inside a person, or a group of people, or a community to flourish, openly, in freedom, so vividly that it gives rise, of its own accord, to the natural order which is needed to be contained within it.

It is so powerful and fundamental that with its help you can create software that is as beautiful and enriching as anything else you have ever seen.

Once you understand this way, you yourself will be able to create software that is alive, that is intertwined comfortably with your life and the lives of others. You will design worlds where you and others will want to work, play, and co-exist together; beautiful places where you can sit and dream comfortably.

This way is so powerful, that with its help hundreds or thousands, or even hundreds of thousands of people, can come together together to create software and community which is as alive and vibrant, peaceful and relaxed, as any living experience has ever been.

Without the central control of authorities and experts, if you are working in this timeless way, a genuine place will grow right from underneath your fingertips, as steady as the grass in the fields or the trees in your backyard.

And there is no other way in which a software which is fundamentally good can possibly be made.

That doesn’t mean that all ways of making software are identical.  Quite the contrary. It means that at the core of all successful software and at the core of all successful processes of creation and evolution, there is one fundamental invariant feature which is responsible for their success.  Although this way has taken on a thousand different forms at different times, in different places, still, there is an unavoidable, invariant core to all of them.

Take a look at the some of the great Web software like Google’s search page, Flickr or del.icio.us.  They all have that unique, yet unhurried, grace which comes from perfect ease and natural balance.  But what is it they have in common exactly?  They are beautiful, ordered, harmonious – yes, all of these things.  But especially, and what strikes to the heart, they live.

Each one of us yearns to be able to bring something to life like this. Or just be a part of it somehow.

It is a fundamental human instinct, as much a part of our desire as the desire to be part of something greater than ourselves.  It is, quite simply, the desire to make a part of nature, to complete a world which is already made of mountains, streams, stones, buildings, ourselves, our living systems, and our increasing connectedness together.

Each one of us has, somewhere in our heart, the dream to make a living world, a universe, and place of our own for us to share with others.

Those of us who have trained as software designers have this desire perhaps at the very center of our lives; that one day, somewhere, somehow, we shall build a software experience which is wonderful, beautiful, and breathtaking; a place where people can go and live their dreams.

In some form, every person has some version of this dream; whoever you are, you may have the dream of one day creating a most beautiful place, virtual or otherwise, where you can come together with others and freely share your knowledge, learn, participate in your community or government, and otherwise conduct your daily interaction with the rest of the world.

In some less clear fashion, anyone who is concerned with communities and other large group efforts has this same dream, perhaps for the entire world.

And there is a way that software can actually be brought to life like this.

There is a definable sequence of activities which are the heart of all acts of software design, and it is possible to specify, precisely, under way conditions these activities will generate software which is alive.  All this can be made so explicit that anyone can do it.

And just so, the process by which a group of independent people can make software become alive and create a place as real as any other can equally be made precise.  Again, there is a definable sequence of activities, more complex in this case, which are the heart of all collective processes of software creation.  And it is also possible to specify exactly when these processes will bring things to life.  And once again, these processes can be made so explicit, and so clear, that any group of people can make use of them.

This process is behind the design of community built software like Linux, Apache, Wikipedia, and many others.  It was behind the design of the great virtual places for people to live and work: the Internet, Usenet, and the World Wide Web.  It was behind the creation of simple, satisfying software of the kind that powers the iPod, the Blackberry, and Firefox; of SourceForge, Wikipedia, and BitTorrent.  In an unconscious form, this way has been behind almost all ways of creating software since the beginning.

But it has become possible to identify it, only now, by going to a level of analysis which is deep enough to show what is invariant in all of the different versions of this way.

This hinges on a form of representation which reveals all possible design processes, as versions of one most fundamental set of patterns.

First, we have a way of looking at the ultimate constituents of the environment: the ultimate "things" which a piece of software is made of.  As we shall see, every piece of software is made of certain fundamental entities known as design patterns; and once we understand software in terms of its patterns, we have a way of looking at them, which makes all software, all of their parts and function, all members of the same class of thing.

Second, we have a way of understanding the generative processes which give rise to these patterns: in short, the source from which the ultimate constituents of software come.  These patterns tend to come from certain combinatory processes, which are different in the specific patterns that they generate, but always similar in their overall structure, and in the way they work.  They are essentially like languages.  And again, in terms of these pattern languages, all the different way of building software, although different in detail, become similar in general outline.

At this level of analysis, we can compare many different software creation processes.

Then, once we see their differences clearly, it becomes possible to define the difference between those processes which make software vibrant, alive, and useful, and those which make them the opposite.

And it turns out that, invariant, behind all processes which allow us to make great software, there is a single common process.

This single idea is operational and precise.  It is not merely a vague idea, or a class of processes which we can understand: it is concrete enough and specific enough, so that it functions practically.  It gives us the power to make software and virtual communities live, as concrete as a match gives us the power to make flame.  It is a method of a discipline, which teaches us precisely what we have to do make our software what we want it to be.

But though this method is precise, it cannot be used mechanically.

The fact is, that even when we have seen deep into the processes by which it is possible to make software alive, in the end, it turns out this knowledge only brings us back to that part of ourselves which is forgotten.  Although the process is precise, and can be defined in exact scientific terms, finally it becomes valuable, not so much because it shows us things which we don’t know (though it may do that as well), but instead, because it shows us what we know already.

Of course, this way of building software has never be named.  It’s not service-oriented architecture, or the personal software process, or agile methodology, or the unified process, or CMM, or any of the others.  It’s the actual things that are conceived and done and worried about when software is created and used.  For now, because all software is quickly becoming connected to all other software, and because the Web is becoming the place where more and more of the relevant software is, and finally because it is a more complete reconception of what we thought we knew, we’ll give it a name temporarily.  An unsatisfying name, but one that we can remember for now.

We will call it Web 2.0.

What do you think?  Are we at a place where we can really identify the design patterns in Web 2.0?

Filed under Enterprise Architecture, Social Technology & Standards, Strategy Tagged with , , , , , , , , , , , , , , , , , , ,

Is Web 2.0 The Global SOA?

October 28, 2005 2 Comments

Are we heading towards an architectural singularity in the software industry? Sometimes it looks that way. If you do a superficial comparison at least, Web 2.0 is all about autonomous, distributed services, remixability, and is fraught with ownership and boundary/control issues. And yet, Service-Oriented Architecture (SOA) is all about, you guessed it, autonomous, distributed services, composite functionality, and is fraught with ownership and boundary/control issues. Sound similar, no?

It does seem that we have a classic case of fractal architecture on our hands. Is Web 2.0 actually the most massive instance possible of service-oriented architecture, realized on a worldwide scale and sprawling across the Web? The answer folks is, apparently so.

I’ve been thinking about this carefully for several weeks now as the similarities seemed to inexorably call to each other as I worked with each of them in turn (disclaimer: I’m a SOA architect by trade). Both Web 2.0 and SOA are already slippery, nebulous concepts yet there are unmistakable patterns within each that actually are very tightly related, though wrapped in slightly different cloth. Each encourages the liberation of the underlying functionality of software systems by providing open access to everyone that needs it. Both warmly embrace Web services and the aggregation of existing functionality into new solutions. And Web 2.0, according to O’Reilly, looks at Data as the Next Intel Inside, making large, back-end database driven functionality a core competency. SOA totally gets this as well. And both Web 2.0 and SOA provide the building blocks for creating people-centric processes starting at the scale of an organization and going up.


Granted, most SOAs are conceptually trapped inside an organization’s firewall or VPN. And Web 2.0 envisions the global Web as the stage writ large upon which to act out your grand visions of building collective intelligence and mashed up functionality. But scale is only one of the minor differences really, and not a genuine discriminator at all.

Do the linkages go deeper, to a more fundamental level? Are Web 2.0 and SOA different at their core and if not, how exactly do they relate?

I believe that there are at least two significant connection points. One is that Web 2.0 can be conceptualized as a global SOA. Two, that many traditional brick-and-mortar business that are currently using SOA as their architectural model will want to connect their Web/Web 2.0 faces up to their SOA. This makes Web 2.0 not just being the Global SOA but makes meeting smaller SOAs everywhere not just likely, but inevitable. Note that the respected industry analysis firm, Gartner, recently said that by 2008 that 80% of all software development will be based on SOA. And interestingly by 2006, Gartner believes that 60% of the $527 billion IT professional services industry will be based on exploiting Web services and technology. We’re talking serious convergence of focus here folks. If true, this means that more than half of all software development, SOA and otherwise, will revolve around the Web, inside or outside organization boundaries. All of this means Web 2.0 and SOA will meet each other both coming and going, and begin to become each other as well.

Web 2.0 = Global SOA? Why Should We Care?

But the real question really is does the relation of the two give us an advantage as we design and build Web 2.0 applications, services, and SOAs? One problem could be that many folks outside the IT industry just haven’t heard of SOA. And even then, there are vociferous arguments about what an SOA really is, just like there are endless debates about what Web 2.0 is. But in the end, there are best practices that need to cross pollinate here and SOA’s IT-bound sphere of influence isn’t really a factor. In fact, really only Web 2.0 designers (yes, you) will have to understand these techniques and their connections. Web 2.0 users themselves will generally be blissfully unaware of Web 2.0 as the global SOA.

Now, don’t get me wrong. Web 2.0 and SOA also have significantly divergent elements too. Web 2.0 emphasizes a social aspect that SOA is completely missing. And probably to its lasting detriment. SOA has much more central control, management, and governance while Web 2.0 is free wheeling, decentralized, grassroots, and with absolutely no command and control structure. Web 2.0 also talks about presentation, the front-end displayed to the user. SOA is largely silent on the issue of presentation though it certainly admits it exists. So SOA tends to be generic and faceless where Web 2.0 makes much of human/service interaction. They seem to need each other to be whole. Finally, Web 2.0 is almost too informal and practically calls out for discipline while SOA is mute and autistic in comparison, a technical virtuosity that wants to be social but doesn’t know how.

All of this makes me want to view one through the other to check basic principles. For example, SOA has best practices for building business processes into vast supply chains (so too does Web 2.0). SOA is also a mature view of software that eschews a technical view of information and data. And it identifies a motive force for these processes via something called orchestration. This is a concept that Web 2.0 does not have explicitly and could certainly use (an orchestration mash-up anyone?) though it is provided in some degree by its users. SOA principles also encourage creation of a common vocabulary across systems that is in the language of the domain (common definitions of customers, order, channel, product, for example). So it gets very close to addressing a big issue in software development today: That too many IT systems today tend to have technology myopia and ignore their most important elements… the people that use them and the way that they work. Web 2.0 gets this part even more right in all the significant ways. Web 2.0 embraces people, collaboration, architectures of participation, social mechanisms, folksonomies, real-time feedback, etc. All things that SOA, in its grey, dull, corporate clothes, does not, at least not explicitly. The complementary nature between the two seems clear.

So, I believe there are complementary synergies between these two powerful software approaches. One can very much be used to check and finish the other. SOA is both the “Mini-Me” of Web 2.0 (identical in almost every way but 1/8 its size) and a key archetype for it as well. Though admittedly one that lacks a few important ingredients. What is compelling, and I’ll talk about this in detail in future articles, is that Web 2.0 actually has powerful mechanisms that “complete” SOA (if you’ll allow one last Austin Powers reference.) Web 2.0 offers a face to SOA with numerous best practices for presentation, has emerging technical innovation like radical decentralization that is necessary for stability and scalability which SOA too often doesn’t address, and Web 2.0 identifies important techniques to immerse users into social processes that can make SOA data and services vastly more valuable.

Yes, so Web 2.0 is a global SOA, done right for the whole world. It’s big, it’s everywhere, and it’s here today.

Do you agree that Web 2.0 is the Global SOA? Post your thoughts below…

Update: Early stage VC investor Peter Rip had some interesting things to say about this article, including “Web 2.0 is a lighter weight version of SOA.”
Update 2: Both Richard Monson-Haefel and ZDNet’s Joe McKendrick weighed in on this topic with good observations and commentary.
Update 3: This eventually turned into the SOA Web Services Journal cover story in Dec. 2005: Web 2.0: The Global SOA.  This in turn led to March’s SPARK event with Microsoft on the convergence of Web 2.0, SOA, and SaaS.
Update 4: This topic (and related issues for Web 2.0 in the enterprise) has turned into a regular blog on ZDNet.
Update 5 in May, 2006: Om Malik writes a detailed piece on the future of Web 2.0 and is most sanguine about it for the enterprise, interestingly enough, and links to this post.

Technorati: web2.0, soa

Filed under Enterprise Architecture, Social Technology & Standards, The Enterprise, The Web