How We Gave Up Control Over The Social Web

A short but pithy piece over the weekend by Dave Winer titled “Why the Web 2.0 model is obsolete” got me thinking about where we’ve ended up with social media after nearly ten years. Blogs, wikis, and other tools of easily shared self-expression from the early days have given away in recent years to a much less diverse social media monoculture. A few large social networks now control our social identity, content, and behavior, and through their terms of service, often literally own our online existence legally and de facto.

This evolution was perhaps inevitable given the rules through which networks operate and certainly the result has its strengths. It’s far easier for consumers and businesses to adopt a hosted service than set up their own social presence, with all the complicated bits it requires to set up a fully functional social identity on your own these days. It’s also probably more secure, safe, and reliable long term. It’s certainly the shortest route to connecting with the vast captive audiences that the leading social networks now wield.

Yet in the process of making many short term decisions in the name of reach and convenience, many of us have given away our social capital, and along with it much of our online autonomy and freedom. I’ve long since stopped advising companies to drive their traffic to Facebook (disclaimer: I am a shareholder) and build their own online communities and digital ecosystems if they are intending to be strategic about things like social business and open APIs.

Network Effects, Social Media, and Centralized vs. Federated

The impressive thing is that we’ve largely achieved the original vision of Web 2.0 and it’s just how we do things now. We share by default. We use social media more than any other digital activity. Social media is now woven into so much of what we do today. Yet the majority of all of this user generated content and online community is now centralized in a few large social silos that can no longer talk to each other.

Even worse, if we go the opposite direction that might seem better long term, we’ve discovered issues with that model too. For example, we’ve learned that when we create many smaller, self-controlled, and more autonomous social environments we then create fragmentation. We can’t easily communicate or collaborate with each other across these social islands. Thus, for as many downsides as the monolithic social networks have, they do achieve one important thing: They create a very large single social universe that we can all communicate across.

So what should we collectively do? Should we cave in and trust that the corporate owners of the social world will be benevolent, even when they clearly have business models that are very often at cross purposes to our needs and desires? Or should we find a way to solve the problem of creating our own social corners on the Web and then connecting them together, all while making it very easy to do so? Personally, I’m hoping it’s the latter. Certainly I’ve explored previously how open social standards have a genuine shot at helping with this, even if it might be a bit of a long shot.

The reality is that social media silos are now holding us back, both as individuals and as businesses. We can do much better if we want. But getting there requires a little long-term discipline and plenty of widespread demand. That makes it pretty unlikely in the face of the enormously strong network effects of the largest social networks today. But perhaps there’s a third option to regaining control over our social lives. In fact, I predict the next big breakthrough in social media is likely to come from the need to resolve this tension between the unfortunate long term consequences of centralized social media and the benefits of a much more federated and user controlled model. Unfortunately, recent history has been a steady march towards the former.

So until then, we all need to mull over where our collective decisions are taking us, for as social media is perhaps the greatest communication revolution in history, its intrinsic power cuts both ways.

Enterprises and Ecosystems: Why Digital Natives Are Dethroning The Old Guard

Why is it that so many traditional companies with an enormous wealth of assets largely fail to transform them for the digital era? By assets here, I mean established customer base, closely held relationships with trading partners, mountains of data and IP, as well as their bread and butter, the actual products and services they offer. For large organizations, these assets typically represent many billions in long-term investment and accumulated value that is being stranded beneath a digital ceiling they cannot seemingly break through. The lesson has been a hard one: It’s been surprisingly difficult for many companies to make a genuine transformation to digital.

For those just joining this conversation, this transformation is about opening up and digitally enabling the strategic assets of our organizations for better consumption and participation, with as low a barrier as possible. It’s also means doing so in a way that continually maximizes their value over time in today’s deeply networked marketplace. Achieving this triggers the primary engine of growth for digital ecosystems, namely network effects. This is how Apple, Facebook, Google, Amazon (new guard), and Microsoft, IBM, SAP, Oracle, and many others (old guard) eventually built hundreds of billions in combined value. They tapped into the relevant power laws of networks by carefully and deliberating cultivating and then closely managing them by harnessing peer production over the network.

Digital Business: Cultivating and Managing Digital Ecosystems (Open APIs, Social Supply Chain, Web Services, SOA, Online Communities, Peer Production, OEMs)

How exactly was this accomplished? They did it by digitally platforming their businesses in specific ways: Enabling self-service on-boarding, viral adoption, open participation, best-of-breed data capture, ownership and control, and took advantage of the fact that relationships — and therefore, ultimately transactions — must take place on the network with as little friction and cost as possible. They realized that we are now all connected together continuously in a single global network and then designed their organizations around this central fact of the digital age. They are now reaping the results of this mindset:

Networked ecosystems must be a core focus and competency of modern business.

This begs the increasingly urgent question: Why then are a large number of older organizations neglecting their digital ecosystems, often failing to meaningfully cultivate them at all for many of their most valuable assets?

This is a key question that fellow Enterprise Irregular Vinnie Mirchandani recently asked in an internal EI mail thread and later posed on his blog:

But for every Apple which has gone one way, I see so many others piss away this huge asset that is their ecosystem. I hear about musicians and filmmakers auditing, even suing studios for accounting disagreements. I hear SAP mentors complain about legal issues getting licenses and other access to new technology. It’s easy to dismiss Joe Konrath’s litany of complaints against book publishers as one from an unhappy author, but the 230+ comments it has drawn shows a deeper angst about how poorly publishers are managing their author ecosystems.

Ecosystems, communities – call them what you want. They are a vibrant organism which deserve far more ink from all of us. And they need professional managers at the companies at the middle who nourish them and not just treat them as railcars to be hustled away whenever inconvenient.

As companies remain inadequately connected to their customers, partners, and workers via digital ecosystems, many of which they do not control, they are missing a rapidly narrowing opportunity. That’s because it’s very, very hard to disrupt a well-established network effect, which is much more powerful than the equivalent notion in the pre-digital era: traditional market share. Network effects are primary focused on pull distribution, while marketshare is heavily based on push, which is much harder and much more expensive to sustain. As digital natives sew up more and more industries and lay down network effects years ahead of their traditional brethren, any chance to reclaim the throne will be very unlikely.

For leading examples of potent digital ecosystems, see open APIs, social customer care, and app stores.

Why is this? There are a number of reasons but a few are particularly significant. What I wrote in the EI thread in response to Vinnie’s original question was this:

I find that in general, the farther you are from the tech business, the less native skill or familiarity there is with system thinking, which is perhaps the critical capacity to have in order to regard your business in ecosystem terms. This is something that in tech is standard fare with constant discussion and focus on platforms, network effects, SDKs, open APIs, app stores, etc.

Traditional publishers are typical of the technically challenged industries that are being blind-sided by newer, much savvier, techno-centric, network-oriented new digital businesses.

Business leaders that can’t deeply see the way forward for their organizations as flexible, highly dynamic, and organic digital networks of customers, partners, workers, and other (likely and unlikely) participants will ultimately fail. But this isn’t the set of skills or mindset that made them successful in the first place, so they don’t value it and don’t think in these terms. They literally throw off digital rethinking like a sort of corporate immune system. Surprisingly, from my talks in the C-suite the last few years, everyone individually seems realizes they have to change, but collectively they are resistant, it’s fascinating to watch.

A big part of the problem boils down to this: Companies are inherently designed to perpetuate the problem they were invented to solve. It’s a particularly thorny instance of the Innovator’s Dilemma, which ensures that a company is unlikely to aggressively re-invent itself until it’s in the process of being disrupted. Unfortunately, this often means it’s already too late.

In fact, it may be too late for a growing number of industries to fully make the transition to being ecosystem-centric. This includes media, publishing, telecom, retail, and many software companies. Under looming threat is real estate, higher education, financial services, professional services, accounting, and even venture capital. In each of these categories, ecosystem-centric firms are building network effects with open network-based products increasingly built by worker/open communities and delivered to customer communities. Those products are in turn built upon by hundreds or thousands of loyal 3rd party partners to bring their own customers and ecosystems to the table. This is an embarrassment of riches that only a few companies, again mostly digital natives, seem interested or able to tap into.

As Fred Wilson once said, the Web (and therefore digital business) is all about “building networks on top of networks“, which leads to even more powerful outcomes, like 2nd order network effects.

Fortunately, the force multiplier of the ecosystem model can be stated in a simple, fundamental way: It allows one to tap into the vast size and strength of the external network to drive growth, innovation, and revenue for your own ecosystem. As Peter Kim and I wrote in our new book, the fundamental principle of business in the ecosystem era must be to let anyone participate in every aspect of the business, primarily by inverting the facilitation process of driving shared value (i.e. network effects by default.) Being able to elicit the network (Internet, community, shared data, whatever) to maximum effect to fuel and growth your ecosystem is thus the core competency of the digital era. Unfortunately, this lesson is being lost to most organizations that were built well before this next-generation business model was understood. It will be a great loss that doesn’t necessarily have to happen in my opinion, but will ultimately result in the needless disruption of a large number of companies that just aren’t able to become digital natives.

For additional reading see:

4 Ways to Create Sustainable Business Ecosystems

Why Information Power Is The Future of Business

What Will Power Next-Generation Businesses?

A View of Digital Strategy in the Ecosystem Era

Are We Building Businesses Or Are We Building Platforms? Yes.

How Digital Business Will Evolve in 2012: 6 Big Ideas

Transforming the Enterprise As We Know It

As I was reading David F. Carr’s latest piece on The Brainyard today, it drove home again for me some of the practically insurmountable challenges that many organizations have in avoiding the growing forces of digital disruption. David’s piece talked about Don Tapscott‘s proposition that we have to fundamentally remake the way our organizations engage with the world and produce useful work. The very-near future of business consists of new methods that are effective in today’s world, not for the era they were created in:

“When most people think of Enterprise 2.0, they think of the use of collaborative tools,” Tapscott said. “I’m arguing that something much bigger is happening than the application collaborative tools within the enterprise–it’s a profound transformation of the enterprise as we know it.” Basic principles of organization that have been established over the last 100 years are being upended, leading to “huge changes in how we orchestrate capabilities to create goods and services,” Tapscott said.

Like Tapscott, I’ve long been a proponent, along with thought leaders like John Hagel, that there’s a deep and profound Big Shift taking place as we get deeper into the 21st century. To survive, we must think in deeply networked, decentralized terms now, not in the rapidly receding business concepts of an age bygone. This means platforms instead of products, ecosystems instead of businesses, peer production instead of central production, and networks instead of hierarchies, to name just a few of the more significant aspects of the shift.

Emergent Business Processes and Enterprise Transformation: CoIT and Social Business Implications of the Big Shift

But how can traditional organizations get there? Web companies have a hard enough time getting there themselves, as digital natives. Most of them certainly don’t become the next Amazon or Facebook, two companies that virtually embody much of the changes taking place. Instead, I see many traditional firms engaging in the cargo cult mentality, hoping that by adding window dressing like social media, a few APIs, and perhaps some user-generated content, that they too will suddenly have a healthy, sustainable future.

Well, it’s not going to happen that way. The changes required are deep and sometimes painful. In fact, the more I examine the issue, successful transformation to a new mode of existence that naturally avoids the disruption inherent in these shifts boils down to a surprisingly few number of key changes. But those changes, though not generally that complex in and of themselves, are almost impossible to drive deeply into many organizations by virtue of their existing structures and processes. As they say, culture eats strategy for lunch.

Many of you know that I’ve been exploring how to foster social business approaches in large organizations for a number of years. When I see successes, they seem to have much in common with what made things like social media so successful. Yes, that’s network effects but also, and more to the point, about enabling an environment where emergent change can actually take root and thrive. A network effect can’t take hold if everything about the traditional way a business operates is to lock everything down into fixed transactional processes. That just doesn’t work in a fast changing new era where the value is in sustaining dynamic relationships and not fixed transactions.

As JP Rangaswami recenty wrote, it’s now all about “The capacity to change. Designed as an integral function. Native.

How then can businesses “fundamentally remake” themselves? What critical changes are at the heart of moving from regular business to things like social business? I’ve been exploring the answers to that question recently in quite some detail, but I’d start with these three things:

  • Local autonomy. Effective, resilient response to business change can’t only be driven by top-down, hierachies. It’s far too slow, low in innovation, and far from problems on the ground. Restructure the organization so that change along the edge is not only possible, but well-resourced, common, and effective.
  • Freeform collaboration. Going beyond Enterprise 2.0 to reinventing the way business models scale and provide value. I’ve previously written about the orders of magnitude cost reductions that are possible and the things they enable, plus how to get there.
  • A culture of experimentation. Of the three, this is the hardest. The first two are different; it’s always possible to create a startup culture at the edge of organizations and it’s also possible to drive mass collaboration. We increasingly see it done all the time in large companies, though it takes time to really establish itself in a transformational way. But to get an organization to be fundamentally more accepting of innovation is very difficult to instill when it does not already exist. Some of it is a skill problem, but a lot of it is more systemic. Solving this is going to be one of the great generational challenges of the social business era.

There’s a lot to consider when undergoing the large-scale transformations that businesses must undertake today, but a focus on these core issue will go way towards getting started.

Exceeding the Benefits of Complexity? A Fractal Model for the Social Business Era

Over the weekend my friend and industry colleague JP Rangaswami wrote an insightful post that pondered how we have gone about delivering on customer experiences as connected to our back-end capabilities. Specifically, he explored an issue that is increasingly challenging many of the large-company CIOs I speak with these days: That the present rates of change demanded of the accumulation of 20-30 years of legacy business systems is greatly exceeding the ability of our enterprises and associated software “stacks” to deliver on them, particularly as cloud, social, and mobile dramatically transform computing today.

The problem lies in our classical views of enterprise architecture and business architecture both. But JP puts it more poetically:

Development backlogs are endemic, as the sheer complexity of the grown-like-Topsy stack slows the process of change and makes it considerably more expensive to change. The stack has begun to fossilise, just at the time when businesses are hungrier for growth, when the need to deliver customer-facing, often customer-touching, applications is an imperative.

Which makes me wonder. What Tainter wrote about societies, what Shirky wrote about companies, are we about to witness something analogous in the systems world? A collapse of a monolith, consumed by its own growth and complexity? As against the simpler, fractal approach of ecosystems?

This fractal aspect of user systems, Web 2.0, and SOA is one that I deeply explored in the 2005-2007 timeframe, and my ideas on this even made the cover story of the SOA/Web Services Journal at one point. This is when we began to see successful, composite systems sprouting up “in the wild” that were eminently natural and highly successful, such as Amazon’s, Flickr’s, and many others. Innovative new open API-based businesses had definitively emerged and shown us — in fact, virtually proved to anyone willing to observe — that business/technology ecosystems could be routinely produced that were far more successful than the ones virtually any traditional business had managed to produce for itself internally with methods like SOA. It was clear something important and new was happening and we tried to learn from it.

Traditional Closed Business Stack vs. Open Networked Supply Chain API

The signature lesson in this time period was that being “in the business” of ecosystems, at multiple levels, was the key to resilience, growth, and sustainability. It’s still a far cry from how most organizations are structured today, but I do see the first hints that this is starting to change. Companies like Best Buy and Sears have been getting the message among the old guard firms and looking at integration, partnership, and engagement as an open network activity.

The Web has also shown us that complexity, as important as it is to address and resolve the many inherent vagaries of dealing with the real world, is largely the enemy, whenever it exists needlessly. Our assumptions of where the complexity should be, in the transports for example, was wrong. It was in the ecosystem itself and we paid dearly for half a decade (and running) based on that misapprehension. But we’ve learned. SOAP usage has almost completely shifted to REST, complex WS-* stacks have largely moved on to simple standards and methods like Web-Oriented Architecture.

But technical discussions obscure the very important truths about what we’ve learned in the interim between the “aha” moment when we realized that what was happening so successfully “out there” was something that would make us just as successful inside our organizations, for internal and customer-facing needs both. After all, this holistic ‘integratedness’ — usually triggered via our connections to the Internet — was a part of our businesses increasingly and we simply had to realize this and engage as Internet natives. But back then, Andrew McAfee worried that focusing on the plumbing was the wrong emphasis to have on the exciting changes taking place, and he was largely right from a communication perspective.

However, building fractal architectures that addresses the old and the new isn’t as easy as just deciding to cast off the old, less effecive ways, as JP notes:

What I’d established in my own mind was a growing belief that the issue was to do with rates of change and costs of change. Vertical integration paid off when the rate of change was low. Networked small-pieces approaches paid off when the rate of change was high.

We are clearly living in times of much greater rates of change. Unfortunately, the old systems, the old architectures, the old business “stack” is often running the core of the business. It’s usually deeply vertically integrated, and not made of small networked pieces, loosely joined that are truly agile, deeply harness innovation on the edge, and turn IT into a profit center instead of a cost center.

So, while I have a lot more to say about this, because I believe that social business context will be at the root of the future of our how we design our products, services, processes, and workplaces, below are a few basic rules you can take away that if you stay true to, can indeed help you make the transition to the future. Note that customer and worker experiences are outside the scope of this, this has more to do with the new business stack below the social layer:

4 Ways to Create Sustainable Business Ecosystems

If resilient, networked, recomposable, open business capabilities are the future of business and IT architecture, getting there then requires a substantial change in stance:

  • Open Your Business Stack. To everyone, internally and externally, generally in the order of the systems and data that are most often requested for integration. Make it easy for anyone to onboard themselves and use the simplest and most egalitarian technology possible to deeply connect the world to your business. Self-onboarding is crucial because it enables the killer asymmetric business advantage of systems that respond to external engagement best: Export most of the change effort to outside your organization and impose it on those that wish to participate and integrate with you. Enterprises that close the “clue gap” with Big Data will have an enormous advantage.
  • Go Into the Business of Ecosystems. Stand behind your open supply chain: Invest in it for the long term, market it, evangelize it, support it, and stand behind it like your customers — again, internally or externally — will be building world changing products and services from them (if you do this, the track record is that they will.) Be fair and generous with your IP and enterprise data; you are under-utilizing it by several orders of magnitude until you do this, leaving much or even most of the potential of your business on the floor. Worse, until you treat it like a business, you won’t get external contributions to enrich your ecosystem that will drive down the cost of change while greatly increasing the velocity of new solutions and local adaptations.
  • Begin Fractal Reconciliation of Your Classical Systems. While hanging a trendy Web API off an ancient COBOL accounting system might be a step in the right direction, long-term success will mean going well behind the API. The existing business/IT landscape must be converged and rationalized so they themselves are made of “networked small pieces” that are faster to change as well as more resilient and responsive to being so much more connected (and therefore useful and valuable) to the world.
  • Spin Up New 21st Century Business Models. All new possibilities, many of which will be foreign to non-digital native firms, are reachable once you have created a fractal enterprise stack. Dominating classes of data, metered business knowledge, all new monetization methods, and more become possible once the three transformations above have started.

Unfortunately, after having advised and directly assisted many large companies in doing much of what I’m describing here over the years, it’s clear that most organizations are at a major disadvantage in making this transformation. The organizational will required is actually quite low, since most of this is fairly easy to do now with cloud technologies, increasingly capable technology partners, and the latest development stacks. Instead, it’s the understanding of how important it is to kill excessive complexity and focus on ways of thinking about business and technical architecture that will propel companies into bright futures.

We must leave behind vertical integration and radically collapse the size of our seams (APIs) in our businesses, while radically increasing the number of useful endpoints such that we look at them as the very fundament of our enterprises. The benchmark for success is how many others are depending on your ecosystem for their own success. Starting now, the activity in your organization must be focused on optimizing for growth of this number. That is the only way we’ll ensure we’re building businesses that are the most relevant, meaningful, and valuable to us going forward. The enterprises that don’t simply won’t survive in their present form. I believe that much is at stake here. Unfortunately, the “cultural gap” John Hagel wrote of back when this conversation started between the enterprise crowd and the Internet crowd is still much too large and is still hindering the move forward for many organizations.

I look forward to continuing this discussion and exploring just how much we’ve recently learned about the future of business and enterprise architecture.

The K-factor Lesson: How Social Ecosystems Grow (Or Not)

Recently for some work that I'm doing I had to revisit the techniques for creating successful online social environments.  This is a surprisingly deep and nuanced topic that we as Web application architects or enterprise social computing practitioners are just now fully beginning to grasp.  The subject matter itself runs the gamut from key conceptual underpinnings — esoteric topics like systems theory and network effects — to the daily grind of understanding and managing the needs/expectations of an often difficult-to-control community of actual, live people.

In general, I've found that the ideas behind social systems themselves are clean and elegant while dealing with their practical realities can definitely be messier and many find them annoyingly unpredictable as well.  In the end, online social ecosystems are invariably a fascinating mix of the classic vagaries of technology and people.  However, despite the apparent science, making them grow into something undeniably successful is still very much an art form. 

Social Surface Area and K-Factor PotentialInterestingly, there's no real name for this skill yet and it's an important — even vitally strategic one — for any organization that has to engage with a lot of people over a network. And that's increasingly most of us in these days of the ever-present Facebook news feed, Twitter microblog, and workplace Enterprise 2.0 environment.  It also means that creating a workable online community requires a good dose of hard-nosed engineering as well as highly effective "soft" skills in UX design, social architecture, and community management. For it to really work — to have a vibrant and growing community — you have to seamlessly connect both of these worlds: a well-crafted social environment together with the people that will use it. The rewards for doing it successfully speak for themselves: Ultimately, businesses and communities are groups of people, and if they produce more value for each other together than they can individually, then there is something in it for everyone. And the online world lets us create these entities far easier, more quickly, and with larger populations than ever before in history.

Related: Network effects are just one of several dozen "power laws" that social architects must know today.

We used to call this process "founding a business" or "creating an organization".  We would call the people that did this entrepreneurs or occasionally philanthropists if their goal was non-commercial. But this terminology doesn't seem to apply as much to what's happening now.  For one thing, communities are organized differently and frequently have other motivations for participation than the usual one for traditional businesses: the worker/employer relationship.  Second, the output of large distributed online communities — especially when focused on discrete outcomes — can greatly exceed the results produced by densely concentrated single institutions.  While we are certainly in the very early days of this phenomenon, I've frequently pointed to numerous examples of these new models for creating shared value. 

So putting aside the socialism vs. capitalism arguments for now (they are increasingly brought up in this discussion, though why they don't seem to apply here is the subject of a future post) the 21st century networked economy — powered by people and knowledge connected together globally at virtually no cost — has set free fundamentally new ways of innovating and collaborating for mutual benefit.  Specifically, it's the rules for how these new mechanisms thrive and create value that is the object of discussion here.

Viral Growth of Social Systems: The K-Factor Lesson

For my own part, I've been fortunate to encounter ways to reduce the concepts for creating growing social ecosystems to a short list, the key ones which I'll present here. Please be warned, some jargon is necessary, but I will explain it along the way.

How To Create Self-Sustaining Social Ecosystems: The K-factor Lesson

Like my 50 Essential Strategies for Web 2.0 Products, this overview cannot possibly be exhaustive.  It does however highlight the central idea behind all successful communities: They are either busy growing or they're busy dying.  Gaining critical mass early on is another important lesson that we've garnered from the early Web 2.0 pioneers.  Finally, the aforementioned and mysterious K-factor is introduced and explained below.  

  • Fiat and network effects are the two primary ways that social ecosystems grow.  The first is based on an explicit or implicit mandate of some kind.  An example of a fiat is when an organization requires participation in a particular social group, perhaps a committee or online community.  The second, a network effect, is when a social group has more value the more that other people are involved in it too (which is usually, but not always the case.) Keep in mind that the degree of value in each new member, or their contributions, is based on the structure or design of the social system and will vary greatly.  Also note that the fiat approach that is common in business today is a top down effort to "push" participation, while network effects are a "pull" method and tend to be more bottom-up and organic.  Background: Read more about push vs. pull systems. Critically, the degree to which a network effect is realized can be influenced by many factors not the least including the intrinsic nature of the social ecosystem itself and how it is connected to the rest of the network (the Web or your intranet.)  In general, network effects will ultimately be more successful than fiat for most purposes, especially since the use of social systems are so often made optional, even in business environments.
  • Network effects can be encouraged by promoting self-sustaining growth via feedback loops (aka virality).  One of the myths of the online world is that viral growth can't be engineered. While it's sometimes not straightforward, it can indeed be created intentionally. While cynical uses of virtuous growth cycles won't produce results for long, the basics work best: letting any user improve the community by contributing knowledge, often to a lightly structured data set or encouraging them to invite their friends and colleagues.  In fact, any good social ecosystem will make it easy and rewarding to do so, such as allowing users to discuss, converse, jointly edit, or otherwise work together on common goals.  Stated another way, the future of software applications that don't provide more value when more people are present in an interaction is probably going to be fairly short. A good place for more background on the lifecycles of systems (social or otherwise) is William Varey's Unlimited Growth.
  • Measure your K-factor and keep it above 1.  The K-factor of an ecosystem is a statement of whether your network effect is self-sustaining or not.  A K-factor of less than one means that your organic growth level is falling (exponentially) and a K-factor of greater than one means that it's growing naturally and again, exponentially.  How to increase your community's K-factor? There are literally countless ways but allowing users to contribute to hard-to-recreate data set, adding user distributable widgets that provide useful offsite functionality, as well as integrating meaningfully with 3rd party social networks through their application model are a few of the more popular and effective ones.  I explored some of these distribution approaches in more detail a while back in my overview of the new distribution models of the Web.
  • The higher the social feedback loop intensity, the higher the breakthrough factor.  Feedback loops are created for example when a user invites another user to the community and then that user invites their friends and so on.  However, the quality of the feedback loop is a measure of how often it reaches outside of the community and how effective that outreach is (often stated as  k = e * i, where “e” is the efficiency of the feedback loop and “i” is the average number of invites per user.)  An individual user's ability to affect the quality and intensity of the feedback loop is usually a measure of their social surface area, which I generally consider to be the size of their social graph x the number of online channels they use x the frequency of participation x the amount of influence they exert. If you have a social ecosystem with a lot of influencers or users that are very active online, they will clearly fuel the response to your feedback loops more effectively. The design of the social environment itself also has a major effect depending on whether it makes it easy to invite others or otherwise create feedback loops.  A breakthrough factor is achieved when the feedback loops rise above a certain threshold of attention, such as when multiple invites come from multiple persons over a short interval of time.  When this threshold is crossed for a prolonged period, the K-factor increases much more rapidly.  This rise can be self-sustaining (the so-called virtuous cycle) and can lead to a permanent and dramatic growth climb as evidenced by many of the popular social networks early on, such as MySpace and Facebook.
  • External "high impulse events" can be used in the short term as a catalyst to reach the breakthrough factor.  This was famously the case with MySpace which used a database of 50 million e-mail addresses to drive initial users to the site, whereupon their feedback loops soon created a particulary strong breakthrough factor.  Traditional marketing is often used as the seed to drive the initial population of communities, but unless the K-factor soon rises above 1, the long-term cost of this approach is usually prohibitive.  It's worth noting that it all my research, I've never encountered a community that was successful long-term without naturally self-sustaining growth.  This is the K-factor lesson of this post's title: You can grow any community in the short term using artificial means but it will only last and continue to grow on its own because of the inherent quality of the people and the knowledge that has accumulated.  That said, you must get your community growth primed in some way.  High impulse events can include (by no means exhaustive) traditional marketing, paid incentives (expensive but Paypal was very successful with this model in its early days), and — particularly for business communities — seeding initial high value content.
  • Growth in a social ecosystem is not endlessly exponential and eventually hits a ceiling. Plan for it.  Typically described by the S-curve, it is encountered when you either fight an existing network effect or when resources are exhausted on the network.  While hitting the upper bound of the S-curve is desirable and denotes a successful system (Facebook is just hitting this now), it can also be an indicator of a failure or a bottle-neck in the feedback loop design.  A false S-curve is most often evident in my research when a community undergoes a major UX redesign and users have to find their way again, often unable to find and use the features that drive growth and sustain the commuity. Be aware of this and guard against the potential causes of premature S-curves.

I've written in the past about deliberately creating emergent phenomenon and then capturing some kind of value from it.  Like most efforts on a fixed scale of zero to the maximum possible result, there is a reverse bell curve effect, meaning that most people will get middling results, some will get very poor, and some will hit it out of the park.  From the projects I've been involved in, I find that the most successful social efforts are ones that are highly agile and willing to capture lessons learned early and often and then make changes quickly and do it all over again the next week.  The Web favors those who experiment, adapt, and evolve and thus should go your social ecosystems.

Good luck with your social media and Enterprise 2.0 efforts.  Please don't hesitate to ask questions below or contribute your own wisdom.

 

Building Modern Web Apps? Better Have A Deep Competency in Web 2.0, Open APIs, Widgets, Social Apps, and More

The Web has an interesting property that those building Web applications and online businesses usually encounter soon after they first launch: It has its own unique and unforgiving rules for success and failure.  Appreciating them requires a certain level of understanding of the intrinsic nature of the Web and how it works.  Actually leveraging those rules requires an even deeper and more profound understanding of the Web. The challenge these days? The Web competency bar is climbing fast.

To drive the right decisions in what they do product designers, marketing teams, software architects, developers, strategy officers, and other key roles in today's generation of online businesses need to have a solid handle on an extensive array of Web topics.  This ranges from appreciating why plain old HTTP is so good at underpinning the Web to more sophisticated topics like modern application architecture, the latest in online user experiences, next generation computing models (grid/cloud/utility/SaaS/PaaS), cost-effective scalability, user identity, network effects, Jakob's Law, analytics, operations, user community, as well as the many compelling new distribution models that are nearly mandatory in the first release of most products. 

This extensive set of competencies is what's required nowadays to deliver a credible online product to a receptive user base and it has dramatic implications for both uptake and overall cost/time-to-market.  Worse, this body of knowledge has become extensive enough that many Web startups frequently fall far short of what they need to know in order to be successful with these far flung practice areas. 

Web Product Distribution Models - Web 2.0, Widgets, Social Apps, Open APIs

Does this complex body of knowledge mean the era of the two-to-five person Web startup is coming to a close? Not at all, at least not yet. The productivity level of the latest tools and techniques remains almost astonishing though the level of knowledge required of these teams is creeping up and up.  And as we'll see, new models for product distribution are pushing the capability envelope of the typical Internet startup team to the point we may very well see the day soon that they won't have all the skills necessary to deliver a fully-scoped modern Web application.  It is also one reason why fewer and fewer Web startups have the goods to be all around hits out of the gate.

Certainly, varying depths in subject matter are required depending one's exact role in a Web business, but Web-oriented products are fundamentally shaped the vagaries of the network itself.  Tim O'Reilly himself still has the best quote on the subject: "Winners and losers will be designated by who figures out how to use the network." And as we'll see, the Web is driving the evolution of a major new generation of online distribution models.

Why Adopting New Distribution Models Is Crucial 

As an example of this, I've been tracking some of the latest discussions around the hot topic du jour in the Web world: Social networking applications.  Specifically, it's been interesting to watch the surprisingly low level of industry attention around the titanic competition brewing between social networking application formats from Web giants Facebook and Google.  Why is this?  Some might say it's because these applications still have largely unproven business models.  Others, like Nick O'Neill at the Social Times recently observed (rightly in my opinion) that the struggle may have to do with a deficit in understanding why these new types of Web applications are so important. Nick notes that these widget and social networking style models for packaging and distributing Web apps often "have more eyeballs looking at their products than television channels have" and the challenge is that too many people just "don’t know what any of this means", despite the major players divvying up the online pie for themselves.  With the size of these next generation distribution audiences, ignorance has an extremely painful price: failure to produce results and growth, poor engagement with the marketplace, and loss of market share.

An excellent summary of the truly massive, but largely underappreciated scale of these new Web application models was last week's TechCrunch piece on the progress of Google's OpenSocial, an increasingly successful model for creating portable social networking applications that will run on any OpenSocial-compliant site.  Erick Schonfeld reported that OpenSocial now has a total reach of an astonishing 350 million users and it will soon be 500 million.  There are over 4,500 OpenSocial apps today, a healthy number for the application format but a small drop in the bucket compared to the number of Web sites in the world. But the key is that these applications are integrated much deeper into the social fabric of an engaged audience, interjecting themselves into the daily personal and work habits of the "captive" users of social sites and even have access to the personal habits and data of users of these sites.  Facebook's story is impressive as well with over 37,000 applications that have been installed over 700 million times.

And social networking applications are just one of many news ways that applications have to be packaged and distributed, yet far too many organizations persist in a very 1990s view of Web experiences, namely that Web sites themselves are the center of online product design.  Many even think that some of these other new distribution models are interesting but not part of their core online product.  Unfortunately, that's very much a parochial view in the present era.  Federated applications, atomized content and functionality, 3rd party product ecosystems through open APIs, and much more are required to establish a strong and resilient network effect which fends off competitors that are themselves bringing these potent new competencies to bear. 

 In fact, one of the things we emphasize over and over again in our conference workshops and in Web 2.0 University is that having a Web site is usually the least interesting things about new products.  Worse, it makes the customer have to find you amongst tens of millions of other sites.  Instead, these new models tend to focus on going to the customer, instead of making them come to you which is a much harder proposition. This can instantly give you the ability to reach millions of potential people with dramatically lower effort and cost, as long as you have something interesting to offer.

Unfortunately, the number of capable practitioners of these new distribution models remains relatively small compared to the large body of experts in traditional Web product development.  Demand is also low for these new skills as most organizations have been painfully slow to appreciate how much online product development has changed.  A quick search of the job aggregator SimplyHired tells the tale: Nearly a thousand Web designer positions are available while only 36 OpenSocial and 40 open API positions are open, for example.  This despite the the latter skills being able to project a product across the Web into hundreds of social sites or create an API that allows the product to be incorporated into countless other products for far less cost per customer than traditional methods.

The lesson here is that these new models still have a lot of fertile, unclaimed territory and many otherwise fierce competitors have not yet become fully aware of these new opportunities.  Get your piece of the pie while there's still time

The new Web 2.0 era distribution models remain largely untapped

I also find that the Web development industry has been slow to change, particularly outside the valley, and there is depressingly scarce information on how to deliver well on things like widgets, open APIs, social networking applications, and even syndication.  To help with this, I've put together a short primer and some good references for those that want to get started.

Because the good news is that there remains tremendous opportunity for growth and success — for both startups and traditional businesses — if they will actively begin incorporating these new product delivery models into their own online capabilities.

Overview of Online Product Delivery Models 

  1. Web sites.  This the classic model for Web presence.  During the early Web, creating a Web site was just about the only option for engaging with those online (e-mail being the other.)  Most early Web sites were used for publishing and not for user participation or peer production.  These days, Web sites are still important, though by no means mandatory, and have their content syndicated via RSS and ATOM (pushing the content to where it's wanted), provide an access point to obtain widgets, and maintain user identity, and create communities of users.  Upshot: They've evolved a lot but Web sites are only part of an extensive set of capabilities that must be brought to bear in the Web 2.0 era.
  2. Syndication. It took ten years for the Web community to figure out a workable syndication model.  Now RSS and ATOM are now the expected models used to distribute content off a single site and across the Web. Countless aggregation services now exist that make a site's information embedded in their services as well as a way to offer users a method for pulling information from a site and experienced in a means of their choosing, from Google Reader and Newsgator to the innovative Yahoo! Pipes.  Most sites still heavily underutilize syndication even for notifications and pushing out frequently changed information to draw attention to it much less the strategic ecosystem and integration opportunities it affords.
  3. Web 2.0 applications.  You might argue that Web 2.0 itself is not a product distribution model but a set of design patterns and business models and that would be a true statement. However, in this context we're referring to the fact that Web 2.0 apps package up the 3rd major type of networked value: user participation.  Before then, Web sites and syndication primarily had only centrally produced content or functionality that they could expose over the network and offer to the marketplace.  In other words, user participation its purest form — sometimes known as peer production –  ultimately results in products like Mechanical Turk and Predictify that provide direct networked access to user participation, but there are many fine gradations to this.  The bottom line, Web 2.0 applications plug the user into the network like never before and are a critical rung in the distribution ladder since it offers access to the largest set of content and information by harnessing collective intelligence.
  4. Open APIs and Web services.  This is one of the most important long-term decisions most online businesses can make.  Offering an open API lets anyone take the online components of a business, from its data and functional capabilities to the users themselves, and makes them open and accessible over the Web to be incorporated into other products and services, sometimes in the form of mashups and sometimes in the form of entire online products.  Amazon, one of the first Web companies in existence and is hence far downrange in terms of the experience curve, has been using this distribution model with notable success recently.  So have hundreds of others.  The real challenge has been how foreign this model is to the original Web model and thus to the various management and development competencies in most organizations.  It's much more an a way to OEM a product and leverage the customers and investments of hundreds of other partners.  However, overall, it affords the potential for much larger business outcomes than could ever be created with point Web presence.  It's now considered a significant oversight not to have an open API available for the typical online product.
  5. Web widgets.  Selecting parts of a Web site and it's data and packaging it up to make it run inside a portable, user distributable widget has been growing more and more popular over the last few years. For example, WidgetBox currently distributes 74,000 different kinds of Web widgets from its partners to over 1.2 million other sites.  Widgets lets users distribute a Web site to other places on the Web at no extra cost and it also creates an ecosystem effect, where other Web sites users become the users of the new site.  The YouTube badge is a notoriously well-known example of this that also helped drive the extraordinarily fast growth of the site.  Like APIs, widgets are now considered a mandatory must-have for new and existing online products. But unlike APIs where it's up to the API users, figuring out users want out of your site's widgets is still an art form.
  6. The Plaxo Pulse Story with OpenSocialSocial networking applications.  Sometimes viewed as an extension of the Web widget model, social networking applications are applications designed to run inside of popular social networking environments and usually have capabilities that tap into and make use of the social graph information resident in a user's social network account.  This is an amazingly fast moving field as you can see from a recent post on the latest happenings on the OpenSocial blog, to the extent it's hard even for well-funded companies to keep up.  However, despite skepticism that large businesses can be built exclusively through a social networking application, it's become ever more essential for a site to make its capabilities accessible usable in these environments.  Not only will users help distribute online products in these formats to their contacts but it also increases the overall usage of the your application including participation and its consequently growth of a site's network effect.  While not yet considered mandatory for online products, the ease with which these social network applications can be created and the large numbers of users they make available makes it a smart distribution option for most Web businesses.  Like widgets, however, figuring out what users will find engaging in a social networking application featuring your online product takes some research and experimentation.  However, the results can be very rewarding and some social networking applications have millions of daily users.  See the Plaxo Pulse story on Mashable for the details of how OpenSocial drove a 5x improvement in traffic in only 3 weeks.
  7. Semantic Web and Web 3.0. The Semantic Web, one of the original visions for the World Wide Web, has taken a while to arrive but it's beginning to look like it may hit critical mass in the next 12-24 months.  Combined with Web 3.0, which takes the architectures of participation at the core of Web 2.0 and drives it through a lens of Semantic Web capabilities.  The benefits can be profound and can greatly increase the value and leverage of information on the Web.  While this is very much not prime time yet, unlike #1-#6 above, it likely will be and smart organizations can get ahead of the learning curve and get an early market lead using these techniques.  For now, however, I recommend that most organizations focus on executing well on the first six items before tackling this and waiting for the technologies to finish emerging and maturing.

The list above should provide good guidance for starting move into the potent new models for distribution on the Web.  I'm seeing, however, that because of the major shifts in strategy and product design emphasis these techniques demand, most organizations take an inordinately long amount of time to become effective with them.  The lesson here: Start small now and build core competency.  Small investments now can pay off later in terms of valuable experience made from early experiments and pilots.  When done right,
these new distribution models can become the dominant channels that the world uses to interact with your business, like they already have with Amazon and Twitter.

I'll be talking about these and other strategic online product design topics in my upcoming Building Next Generation Web Apps Workshop at the inaugural Web 2.0 Expo 2008 NYC next month.  I'll have more details about this deep-dive session in an upcoming post.

Web 2.0′s Real Secret Sauce: Network Effects

A lot of the early descriptions of and speculation around Web 2.0 last year were either chock full of examples or long lists of interesting new phenomena that seemed to be emerging on the Web.  My list of Web 2.0 explanations last year is a good survey of these.  This year however we seem to be zeroing in on the core phenomena underlying the sea changes of the Web and even in society itself.  Sure, some fundamental upgrades to the Web’s infrastructure – including steady and significant improvements to the physics of the Web – have also helped things along (i.e. those great Ajax/RIA applications we love so much like Gmail and Gliffy, download in seconds now.)  But it’s the widespread use of the two-way Web that’s really the harbringer of an increasing sense of disruption.

Watch a TV panel that I was on with Google’s Adam Bosworth on how the improvements to the physics of the Internet have enabled Google’s newer products, particularly around SaaS and Ajax.

I wrote recently about the trend of Web 2.0 reductionism that is helping us get to Web 2.0 fundamentals and assisting us in understanding why it’s such a game changer.  The upshot is that the Web is rapidly evolving and is increasingly being shaped by its users.  If you’re not entirely sure that this is really a big deal though, you have only to look at the example of MySpace.  MySpace is as pure a Web 2.0 play as you’re likely to find and they’ve exploited user generated content and network effects to become the #1 visited site on the Web (as of last week), from out of practically nowhere two years ago.  This is an extremely impressive achievement and shows the power of Web 2.0 techniques to quickly best even the very largest and most established industry leaders, including Google and Yahoo!  This demonstrates that the successful exploitation of what has been labelled "Web 2.0" can be an extremely disruptive force.  People are sitting up and taking notice.

Inducing Network Effects with Web 2.0


If only the keys to doing this were in everyone’s hands, the thinking goes, then disruption could be and would be widespread, even rampant, across business and society.   And certainly, a lot of folks are very interested in taking advantage these techniques (such as maximizing your users’ social surface areas) both out on the Web, and within their own organizations.  This further explains the interest by venture capital investors, startups, and enterprises to get into the game before others use these techniques first to upend the traditional industry leaders in their areas of interest.  Mark my words, this year is only the beginning and those in less regulated industries that aren’t constrained from using Web-based network effects effectively are going to have a very tough time of it.  This will prove to be true particularly in the next 24 months as everyone scrambles to try to figure out where this is all headed.

Fantastically and ironically, the keys to doing this actually are in everyone’s hands today, right now.  Web 2.0 is a truly egalitarian force and though certain core competencies are required (being able to keep up with your own growth being one of them), just about anyone with a innovative idea and a good formulation of the techniques can access and tap into the existing audience of over 1 billion Web users.

Triggering Network Effects: As Simple As Establishing an Effective Viral Feedback Loop?  Surprisingly, yes.

The Web is the fabric upon which an ever increasing amount of our lives is woven into.  This is now most media including newspapers, TV, radio, entertainment, music, arts, etc. as well as what I call lifestyle logistics; e-mailing, IMs, calendaring, travel planning, time/task management, and more.  They are all moving or have
already moved to the Web.  This very habit most of us have of being on the Web so much of the time, along with the easy lure of the hyperlink, which can redirect anyone via any of these ‘channels’ into a new Web 2.0 experience or site.  Thus, if someone loves your new site, they send their friends the link, they send their friends, and so on.  Instant pile-ons involving tens or hundreds of thousands of users overnight are now common.  And good viral feedback loops keep them there and keep them coming back, and bringing their friends with them.

Of course, in a few years the exact design patterns for triggering a new MySpace, Facebook, or similar social juggernaut will become common.  Then most likely balance will be reachieved in the industry and there will be less disruption.  But for now the secret balance of Web 2.0 techniques that powers growth through efficient access to network effects is still an art (read my Notes on Making Good Social Software for an idea of how these design patterns might look however).  The bottom line: the upside and downside potential of Web 2.0 is truly significant.  And it means that in most industries doing nothing is really no longer an option.

But let’s talk about network effects for a minute.  What are they?  I’ve written before that the core description of a network effect  is when a good or service has more value the more that other people have it too.  Easy so far right?  Examples include e-mail, IMing, the blogosphere, and even the Web itself.  But what’s not clear from this description is the raw power that is caught up in and represented by network effects.  Most rigorous studies and mathematical formulations reveal that there is tremendous geometric power in network effects.  Though Robert Metcalfe originally coined Metcalfe’s Law to describe the raw potential of network effects in computer networks, most recent formulations have attempted to capture the exact value of them more precisely.  Most notable has been David Reed with his relatively well-thought out and profound  Reed’s Law, as well as Odlyzko and Tilly with their work.  However, whichever formulation you believe is right (and Reed’s Law, if true, has staggering implications in this regard), the result is clear: At even an early point, the cumulative value of a large number of connected users goes exponentially off the charts.

The Potency of Network Effects and Thus Web 2.0


The end result: If you have a million people visiting your Web site but you’re not leveraging network effects with them (such as by letting them contribute and letting others see and respond to those contributions), then you’re probably squandering the greater part of the value of that million person audience.  Along comes someone else who does exploit the network effects of their users.  They can quickly leapfrog you if they figure out a good way to establish and harness the connections between those same users, again in some kind of viral feedback loop.  The new site will have the combined aggregated output of those users as part of its value proposition to new and existing users and it’s clear which will win.  Examples: Digg bested Slashdot in just such a way.  YouTube is set to do something very similar to network and cable television.  Google is staging itself to best the #1 software company in the world.  There are many others.

So what do you do?  Most of us will soon face situations where the person or organization we’re competing against will be trying to use these same techniques to gain advantage, marketshare, or whatever.  Agile development processes have taught us in recent years that that the tightest closed feedback loops resulted in better products, faster.  The Web is now allowing feedback loops of this kind between staggering numbers of non-centrally coordinated people from a ready -to-tap worldwide audience.  Emergent content, communities, and unintended results are all side effects of systems (and yes, your enterprise) with rampant network effects.  Learning how to channel these feedback loops and the resulting network effects constructively will become a significant competitive advantage and a required core competency across the majority of professional disciplines in the very near future.

Read David Berlind’s excellent overview of Eventful and how they are using Web 2.0 techniques to be successful

But is this really happening?  Is the competitive landscape about to be torn up?  Are companies starting to respond?  It does appear so.  For one thing, I’m increasingly seeing traditional organizations trying to do things like "MySpacing" their current product offerings.  Everything from Starwood Resorts to major national interest groups are looking at creating business communities that are really viral social networks powered by their users.  It is a fascinating time to be building or reinventing a business. And if you haven’t started, now is probably the time to begin the effort.  And worry not if you can’t figure it all out right now, the entire world is still trying to figure out how it applies to them too.

How will you apply Web 2.0 to your life, business, or organization?

Follow

Get every new post delivered to your Inbox.

Join 50 other followers