Sunday Musings: Google’s Identity Struggles, Plus Social Media Bans Around the World

The Web’s missing features for built-in user identity have become a real headache for the industry, and for its users too. It certainly took its toll on market leader Google this week as its “Identity Theater” continued (Source: Kevin Marks.) The issue? It’s turning out that making every single user comply with the Common Names policy isn’t workable for a variety of reasons. Reports of Google deleting accounts en masse are driving a lot of the discussion. Robert Scoble has his own recommendations for Google and while they’re probably the least that would be acceptable to the majority of people, it doesn’t go far enough I think.

It certainly doesn’t have to be this way. Twitter allows companies, bots, and just about every other type of social account and it works quite well in the end. Twitter ran into a similar identity issue in a big way a couple of years back after facing lawsuits and widespread complaints. They managed to muddle through with Verified Accounts.

A growing consensus is that Google should allow user-defined accounts as well, with verified identity for those that want or need it. Personally, I’m not sure I see Google coming around with a response fast enough to prevent some damage to services and impacting Google Plus‘s runaway adoption. But in my analysis, it’s most likely to only hurt the commercialization of the service, not regular usage for most for now.

Social Identity Ownership - Google or Facebook?

Worse, the problem may actually be core to the way Google’s stack is conceived and architected. It may not be easy for them to change course in the short-term without ripples through the way global Google’s services fundamentally operate from a security and identity perspective. It also may not be good for their business model which is almost certainly based on the fact they know who people really are. This issue is one to watch given Google’s pervasiveness. It also has some significant implications for business users of its products, especially now that they seem to be gaining some much needed traction in the social networking wars.

For now, I’d recommend that businesses use Google Plus with an eye towards experimentation while the Web giant gets its philosophy and policies around identity sorted out. Frankly, the bigger industry issue is social Web identity itself. Users and companies increasingly depend on commercial providers like Facebook, Twitter, and Google to provide everything identity-related, from login access to storage and maintenance of their social graph. This is causing key elements of power and control to start to swing away from the open standards that made the Web so successful and essentially fair.

Will the W3C step in and resolve what’s appearing to be an increasingly glaring absence in the Web stack? So far it seems unlikely given the failure of many years of open standard Web identity efforts. The culprit? You have only to look in the mirror. Apathy by users and lack of consensus on the part of Web developers. There’s also a lot at stake financially for those that end up owning a big chunk of Web identity. Consequently, online — and especially social — identity is likely to grow into a full blown brouhaha in the next couple of years as issues, missteps, and abuses inevitably surface. However, we could also decide to put our own house in order before governments step in, the least desirable of all outcomes in most imaginable scenarios. The worst probably being governments owning, issuing, and centrally managing verified Web identity credentials for everyone.

Which brings us to the next subject…

Government Bans Chipping Away At Social Media Freedoms?

A couple of interesting things happened this week with governments aiming their considerable might at social media. While knee-jerk responses to this space were common enough a few years ago, with the U.S. Marines banning social media access for a while for example, these are now generally understood to be counterproductive and unworkable for a long list of reasons.

However, that didn’t stop the German government from banning the Facebook ‘Like’ button on Friday, sure to ignite a small firestorm in that country given that it seems to apply to any site accessible from inside its borders and the fine is a stiff €50,000. The Like button, used on millions of sites around the world to enlist users to leverage their Facebook social network to share content from 3rd party sites (see: k-factor), is significant enough on its own to put German Web businesses at some competitive disadvantage on the global stage. The concern is over privacy and that “all the information was sent to the US company even if someone was not a Facebook member.

In another similar situation, the Missouri state government’s new law preventing teachers from using social media to communicate privately with students, the former who just announced that they are fighting back, is another case in point. There are obvious free speech issues with the law despite the good intent on its face to protect students. The real issue is that the law is that violations are almost impossible to detect and enforce, until its too late, and that it ensures teachers, one of the most collaborative and interaction driven professions with far reaching impact, can’t have much of a social media presence of any kind until the implications are sorted out. It also presumably doesn’t prevent teachers from privately communicating with their students in any number of other digital channels. All of this means the law won’t accomplish a whole lot other than sowing confusion and promoting the use of increasingly obsolete methods in an increasingly fast-changing economic and societal landscape.

The real issue with both of these laws is that they are 1) essentially short-sighted, 2) exhibit such poor understanding of social media as to be essentially useless, and 3) are therefore unlikely to be meaningfully carried out. Worse, they chip away at the edges by introducing step-by-step, largely ineffective government oversight and control over social media, one of the largest economic, cultural, and societal changes of our time. This will become an even hotter topic as the Middle East’s social media coordinated model for uprising spills out of the developing world. In fact, this has already happened in Britain and there are already cries to ban social media in cases of civil unrest.

I should be careful to note here: I’m not by and large suggesting there’s any overarching government scheme to interfere with and control social media. Instead, I’m suggesting we keep a close eye on these developments as social media legislation increasingly (and inevitably) accumulates in bits and pieces on the base of knee-jerk responses to individual situations. This will have a great many unintended and unwanted consequences. The continued growth of laws and regulations in a vital new industry that thrives on inherent openness and trust has the potential to limit it so profoundly that we could lose much of the great promise that social media can provide.

While we must find ways that work to protect our citizens, we must also provide them access to one of the most open, free, and powerful means of interacting that has been invented. Let’s push back on unreasonable measures while also proactively being responsible for solving them. It’s up to us to start finding globally acceptable solutions to privacy, security, and misuse in social media and getting them into the hands of those who don’t understand this space well enough yet to govern it. The options for making this happen are something I’ll explore as soon as I can.

Connecting Agile Business with Social Business

When Jim Highsmith graciously invited me to give the opening keynote at the inaugural Agile Executive Forum in Salt Lake City this week, I had to really sit down and think about what I’ve been working on the last few years, namely social business, as compared the conference theme, agility and business. While agile methods have had many separate and distinct threads within the business and technical worlds over the last 20 years, one of the most active areas has been in software development. For its part, social business is a much newer phenomenon that’s become a top priority for many business leaders in the last couple of years. So, while I’ll cover the details of my presentation — in which I connected agility and social business as drivers of innovation, in another post — I will attempt to more formally to capture the specific similarities here.

In recent years, as agile development has been increasingly borne out as a fundamentally better, more efficient, lower risk, and more cost effective way of doing things, there has been significant and growing effort apply agile lessons to business in general. And, as it turns out, agility and social business, as two major new ways of connecting and organizing people in directed activity, have plenty in common. Perhaps even more importantly, they have key things to learn from each other.

I’ve had quite bit of experience with agile methods personally, having led extreme programming project teams and been closely involved in large, distributed SCRUM projects in years past. I’ve seen agile methods work significantly better than classical processes. This is probably why it’s now the most common development process in software that developers identify with in my experience. Consequently, I’m in a position to see some of the connections between business agility and social business, in all their many flavors. The connection isn’t trivial either. There are hard won lessons learned from agility that social business initiatives could certainly benefit from. Just as there are innovative new approaches to scale, transparency, process, and tooling that social business brings to the table, as extreme and radical as they may appear to agile folks, who are more used to being the harbingers of change.

Comparing Agile Business and Social Business

What’s the point of connecting these two approaches? Because they can learn a great deal from each other. Agile methods can be updated and modernized from what social business brings to the table, and social business can apply some maturity and rigor to what it does, as appropriate. This I believe is a fruitful exercise for both disciplines and is one I summarize below.

Agile Business and Social Business: Side-by-Side

Keeping in mind that some agile process purists are still on the fence about applying the methods more broadly, the focus here is on agile processes of any kind as applied to general people-based business activities. Some processes are more amenable to agility, just as some are more amenable to social business. In general, however, the less collaborative, more rigid, and user-isolated a business activity is, the less applicable either agile or social media methods will be to it. However, if you have a complex, open-ended, and outcome-oriented business process involving many people, especially including those that it most directly affected (typically, the customer, internal or external), then both approaches represent the very best ways that we know of today to deliver successfully on them.

As you’ll see, agility and social have much more in common than they have differences. Here’s my take on how they break down:

  • Coordination Instead of Control. Both agility and social eschew using centralized hierarchies to achieve control. Instead, as Brad Appleton has long recommended, they both work best with autonomous, adaptive, and accountable actors. The first two are something that applies very much to social business, while the latter is something inherent in any social environment that has a strong identity system (which, unfortunately, not all do.) The lesson here is that emergence (an important and prized aspect of Enterprise 2.0) and self-organization are very similar and are shared as core values in both disciplines.
  • Designing for Change/Loss of Control. This is something in which agile is inherently stronger than nascent social business methods, which are just wrapping their heads around this. Not killing emergence requires the acceptance that external change is a desired constant and should be responded to productively to get the right results with the resources at hand. Ignoring that requirements aren’t what the customers need, that the planned outcome of a business process won’t be very useful, and other denying of reality is anathema to both disciplines, but is more formal and well-defined in agile methods. Social business does recognize that the majority of productive output is on the edge of the network and largely outside of formal control, but other than measuring community sentiment, that’s often as far as it goes in terms of responding to new ground truths. The best results in both approaches are when there are tight feedback loops to all stakeholders and that a planned response to that feedback is the central factor in re-engagement with the project or online community in the next cycle. For additional insight, read Tim Leberecht’s great overview of this issue, titled Openness or How Do You Design For the Loss Of Control.
  • Frequent Work Cycles. Agilists call work cycles iterations. Social business doesn’t have as strong a notion of discrete work cycles because it’s essentially continuous and itself emergent, a more extreme version of agile when you look at collaborative work in social media environments such as crowdsourcing efforts or Social CRM. In either case, the project and/or community must assess and respond to change at the end of each iteration, or do it continuously which is more common in the case of social business processes.
  • Open Contribution. Social business works best when the broadest possible invitation is made for stakeholders to get involved and contribute. Agile processes tend to define valid contributors to a smaller audience, though it’s entirely up to the project and varies widely. Social business realizes that the “anyone can contribute” default stance is one of the most powerful concepts in recent business history (as only those that care about the outcome will get involved, yet that’s almost always many more people than you thought.) Agile methods could learn from the extreme openness and fewer contribution boundaries and barriers in social media. I made the point in my speech that open source software has proven this in the real-world better than any a priori speculation about what works best ever could.
  • Working Results. It’s long been the mantra that agile processes value working software as soon and often as possible at any given time in the project. When the requirements are right and/or the budget runs out, you have the best possible output, ready to use. Social business is not yet so disciplined in its directed outcomes, yet by its very nature is always up-to-date with the latest revisions, contributions, or updates.
  • Continuous Processes. While agile business typically recommends iterations, milestones, review steps, and other processes to happen as often as they provide useful course corrections (typically every few days, or weeks at most), social business is even higher velocity and larger scale. Consider real-time processes that run around the clock globally involving tens of thousands and sometimes a million or more simultaneous contributors. This means the scale and velocity of social business often outpaces agile by two to four orders of magnitude. Social business could learn a lot about continuous in the small (builds, releases, work product iterations, etc) while agile can perhaps learn to scale and go even faster in a way it never could before.

This comparison just scratches the surface but is a useful start. I’m happy to be called out on any details anyone feels like I may have gotten wrong. I do believe that agility and social business go hand-in-hand and that we can cross pollinate the two to create far stronger results that either can by themselves today. Put simply, agile business and social business are two sides of the same coin. That may be a controversial statement to some but I believe that as far along as these two disciplines have come in parallel, they will do better with more explicit and effective connection. Our organizations (businesses, organizations, government, etc.) will almost certainly benefit.

What do you see as the commonalities and differences between agility and social?

On Web Strategy

Global Use of Social Networks and E-mailMy old Web 2.0 blog is finally closing due to hosting issues so I’m moving the conversation here going forward. I’m also relocating my large library of old posts and visuals to this blog over the next few weeks. Collectively they’ve had over 12 million views and are witness to an amazing time in the history of the Web, business, and society. 

It’s been a profound era of change by any measure, and one that we’re fortunate to live through. Over the last seven years we’ve seen the rise of social media, Web 2.0, Enterprise 2.0, and now social business. Put simply, the Web-based world has changed nearly everything about the way we globally connect together and create shared value. For now, this blog as well as ZDNet, ebizQ, Dachis Group, and Hinchcliffe.org, will be where I will continue exploring the emerging edge of business and technology, with this blog focusing more on the Web itself and my other channels focusing more on the enterprise aspects.

I’m renaming this blog to On Web Strategy because I’ll continue to focus on the way the Web works, particularly what makes it so powerful for those that understand it. There’s lots of exposition available online about the changes taking place today, but not enough exploring the specifics of how Web-based networks are driving pervasive change. Among the endless information streams available now, there’s still room for more thorough examination of the way the Internet is co-evolving into the single most powerful platform for self-expression in history. I believe this is true whether you’re a person or a business; there is no other place nearly as compelling, innovative, valuable, or relevant today. And the Web itself, far from reaching maturity, remains the single most exciting — and most rapidly moving — place to improve and transform how we live and work.

Knowledge Work Dominates U.S. Labor by SectorWhile my primary interest lies in connecting the two too-often loosely connected roads of business and technology, I think the increasing convergence between them is where much, if not most, of tomorrow’s opportunity resides for those that can successfully overcome the obstacles. For its part, social media has become a leading force for value creation in the world along with the rest of our digital footprints, with which we are now creating the richest and most vibrant record of our times. Visionary enterprises are now seeing how to tap into this and join in partnership with the rest of the world to create entirely new types of products and services together with their customers. From the data, it’s clear that social co-creation and other new and closely related models, such as crowdsourcing, are genuinely changing the nature of human activity, especially value creation, control, ownership and other less-tangible qualities like trust, openness, and understanding.

While some organizations and individuals will continue to debate the actual magnitude of the changes that the new low-barrier, high scale, and virtually free tools of self-expression are fostering today in the large, there’s little doubt looking at the macro trends that momentous things are happening. I have certainly been asked, “are these changes as big as the printing press? As big as mass media? How about personal computing” Yes and yes and yes. And much more significant in terms of actual change wrought. I’ve included here a few recent pieces of my research that illustrate the case that the business, cultural, and societal landscape is being remade right now, and doing it quickly in some cases. This includes the following data points:

Trends and Drivers of Work and Life Today

    Portion of the Web That's Peer Produced
  • Social is how we communicate today. There has been a generational change of communication from point-to-point (e-mail) to social in four short years (see first figure above). Certainly e-mail is in decline and will be with us for a decade or two, but it has dramatically lost its prominence and relevance in recent years. For now, social media is the way we increasingly prefer to connect and work together. Traditional organizations have had some trouble catching up to these trends, but I’m finally seeing evidence that they’re doing so.
  • Knowledge work is the driver of our world economy. People-based activities centered around the creation and exchange of information (financial services, real estate, education, media, governing, etc.) are what modern economies are built on (see second figure above.) Methods that greatly improve the creation and exchange of information will have inordinate value, especially models that optimize for it, i.e. social media and by intent, social business. Knowledge work is about 60% of the labor force today and growing steadily. The less valuable service economy is growing as well and is also a beneficiary of these new and emerging forms of communication and collaboration.
  • Peer production is now the primary motive force for creation and sharing. Centrally controlled models for production in business and government are much less powerful and inordinately more expensive. The creation of information on the “edge”, by individuals, has transformed traditional media as well as the Internet: It’s now made by us, with approximately 80% of information on the Web (see last figure, right) now coming from user generated content. In the future, sensors and other information generators may outpace us, but for now the most important trend is that productive capability has moved decisively into the hands of us out there in the long tail.

What does this all mean? That’s what this blog was developed to figure out. Come back and visit. I’ll be exploring Web strategy and cutting-edge innovations that are likely to have significant impact to the way we run our organizations and live our lives. Please drop me a line if you want to share your ideas, or better yet, contribute them in comments below or in your own blog or social network.

The K-factor Lesson: How Social Ecosystems Grow (Or Not)

Recently for some work that I'm doing I had to revisit the techniques for creating successful online social environments.  This is a surprisingly deep and nuanced topic that we as Web application architects or enterprise social computing practitioners are just now fully beginning to grasp.  The subject matter itself runs the gamut from key conceptual underpinnings — esoteric topics like systems theory and network effects — to the daily grind of understanding and managing the needs/expectations of an often difficult-to-control community of actual, live people.

In general, I've found that the ideas behind social systems themselves are clean and elegant while dealing with their practical realities can definitely be messier and many find them annoyingly unpredictable as well.  In the end, online social ecosystems are invariably a fascinating mix of the classic vagaries of technology and people.  However, despite the apparent science, making them grow into something undeniably successful is still very much an art form. 

Social Surface Area and K-Factor PotentialInterestingly, there's no real name for this skill yet and it's an important — even vitally strategic one — for any organization that has to engage with a lot of people over a network. And that's increasingly most of us in these days of the ever-present Facebook news feed, Twitter microblog, and workplace Enterprise 2.0 environment.  It also means that creating a workable online community requires a good dose of hard-nosed engineering as well as highly effective "soft" skills in UX design, social architecture, and community management. For it to really work — to have a vibrant and growing community — you have to seamlessly connect both of these worlds: a well-crafted social environment together with the people that will use it. The rewards for doing it successfully speak for themselves: Ultimately, businesses and communities are groups of people, and if they produce more value for each other together than they can individually, then there is something in it for everyone. And the online world lets us create these entities far easier, more quickly, and with larger populations than ever before in history.

Related: Network effects are just one of several dozen "power laws" that social architects must know today.

We used to call this process "founding a business" or "creating an organization".  We would call the people that did this entrepreneurs or occasionally philanthropists if their goal was non-commercial. But this terminology doesn't seem to apply as much to what's happening now.  For one thing, communities are organized differently and frequently have other motivations for participation than the usual one for traditional businesses: the worker/employer relationship.  Second, the output of large distributed online communities — especially when focused on discrete outcomes — can greatly exceed the results produced by densely concentrated single institutions.  While we are certainly in the very early days of this phenomenon, I've frequently pointed to numerous examples of these new models for creating shared value. 

So putting aside the socialism vs. capitalism arguments for now (they are increasingly brought up in this discussion, though why they don't seem to apply here is the subject of a future post) the 21st century networked economy — powered by people and knowledge connected together globally at virtually no cost — has set free fundamentally new ways of innovating and collaborating for mutual benefit.  Specifically, it's the rules for how these new mechanisms thrive and create value that is the object of discussion here.

Viral Growth of Social Systems: The K-Factor Lesson

For my own part, I've been fortunate to encounter ways to reduce the concepts for creating growing social ecosystems to a short list, the key ones which I'll present here. Please be warned, some jargon is necessary, but I will explain it along the way.

How To Create Self-Sustaining Social Ecosystems: The K-factor Lesson

Like my 50 Essential Strategies for Web 2.0 Products, this overview cannot possibly be exhaustive.  It does however highlight the central idea behind all successful communities: They are either busy growing or they're busy dying.  Gaining critical mass early on is another important lesson that we've garnered from the early Web 2.0 pioneers.  Finally, the aforementioned and mysterious K-factor is introduced and explained below.  

  • Fiat and network effects are the two primary ways that social ecosystems grow.  The first is based on an explicit or implicit mandate of some kind.  An example of a fiat is when an organization requires participation in a particular social group, perhaps a committee or online community.  The second, a network effect, is when a social group has more value the more that other people are involved in it too (which is usually, but not always the case.) Keep in mind that the degree of value in each new member, or their contributions, is based on the structure or design of the social system and will vary greatly.  Also note that the fiat approach that is common in business today is a top down effort to "push" participation, while network effects are a "pull" method and tend to be more bottom-up and organic.  Background: Read more about push vs. pull systems. Critically, the degree to which a network effect is realized can be influenced by many factors not the least including the intrinsic nature of the social ecosystem itself and how it is connected to the rest of the network (the Web or your intranet.)  In general, network effects will ultimately be more successful than fiat for most purposes, especially since the use of social systems are so often made optional, even in business environments.
  • Network effects can be encouraged by promoting self-sustaining growth via feedback loops (aka virality).  One of the myths of the online world is that viral growth can't be engineered. While it's sometimes not straightforward, it can indeed be created intentionally. While cynical uses of virtuous growth cycles won't produce results for long, the basics work best: letting any user improve the community by contributing knowledge, often to a lightly structured data set or encouraging them to invite their friends and colleagues.  In fact, any good social ecosystem will make it easy and rewarding to do so, such as allowing users to discuss, converse, jointly edit, or otherwise work together on common goals.  Stated another way, the future of software applications that don't provide more value when more people are present in an interaction is probably going to be fairly short. A good place for more background on the lifecycles of systems (social or otherwise) is William Varey's Unlimited Growth.
  • Measure your K-factor and keep it above 1.  The K-factor of an ecosystem is a statement of whether your network effect is self-sustaining or not.  A K-factor of less than one means that your organic growth level is falling (exponentially) and a K-factor of greater than one means that it's growing naturally and again, exponentially.  How to increase your community's K-factor? There are literally countless ways but allowing users to contribute to hard-to-recreate data set, adding user distributable widgets that provide useful offsite functionality, as well as integrating meaningfully with 3rd party social networks through their application model are a few of the more popular and effective ones.  I explored some of these distribution approaches in more detail a while back in my overview of the new distribution models of the Web.
  • The higher the social feedback loop intensity, the higher the breakthrough factor.  Feedback loops are created for example when a user invites another user to the community and then that user invites their friends and so on.  However, the quality of the feedback loop is a measure of how often it reaches outside of the community and how effective that outreach is (often stated as  k = e * i, where “e” is the efficiency of the feedback loop and “i” is the average number of invites per user.)  An individual user's ability to affect the quality and intensity of the feedback loop is usually a measure of their social surface area, which I generally consider to be the size of their social graph x the number of online channels they use x the frequency of participation x the amount of influence they exert. If you have a social ecosystem with a lot of influencers or users that are very active online, they will clearly fuel the response to your feedback loops more effectively. The design of the social environment itself also has a major effect depending on whether it makes it easy to invite others or otherwise create feedback loops.  A breakthrough factor is achieved when the feedback loops rise above a certain threshold of attention, such as when multiple invites come from multiple persons over a short interval of time.  When this threshold is crossed for a prolonged period, the K-factor increases much more rapidly.  This rise can be self-sustaining (the so-called virtuous cycle) and can lead to a permanent and dramatic growth climb as evidenced by many of the popular social networks early on, such as MySpace and Facebook.
  • External "high impulse events" can be used in the short term as a catalyst to reach the breakthrough factor.  This was famously the case with MySpace which used a database of 50 million e-mail addresses to drive initial users to the site, whereupon their feedback loops soon created a particulary strong breakthrough factor.  Traditional marketing is often used as the seed to drive the initial population of communities, but unless the K-factor soon rises above 1, the long-term cost of this approach is usually prohibitive.  It's worth noting that it all my research, I've never encountered a community that was successful long-term without naturally self-sustaining growth.  This is the K-factor lesson of this post's title: You can grow any community in the short term using artificial means but it will only last and continue to grow on its own because of the inherent quality of the people and the knowledge that has accumulated.  That said, you must get your community growth primed in some way.  High impulse events can include (by no means exhaustive) traditional marketing, paid incentives (expensive but Paypal was very successful with this model in its early days), and — particularly for business communities — seeding initial high value content.
  • Growth in a social ecosystem is not endlessly exponential and eventually hits a ceiling. Plan for it.  Typically described by the S-curve, it is encountered when you either fight an existing network effect or when resources are exhausted on the network.  While hitting the upper bound of the S-curve is desirable and denotes a successful system (Facebook is just hitting this now), it can also be an indicator of a failure or a bottle-neck in the feedback loop design.  A false S-curve is most often evident in my research when a community undergoes a major UX redesign and users have to find their way again, often unable to find and use the features that drive growth and sustain the commuity. Be aware of this and guard against the potential causes of premature S-curves.

I've written in the past about deliberately creating emergent phenomenon and then capturing some kind of value from it.  Like most efforts on a fixed scale of zero to the maximum possible result, there is a reverse bell curve effect, meaning that most people will get middling results, some will get very poor, and some will hit it out of the park.  From the projects I've been involved in, I find that the most successful social efforts are ones that are highly agile and willing to capture lessons learned early and often and then make changes quickly and do it all over again the next week.  The Web favors those who experiment, adapt, and evolve and thus should go your social ecosystems.

Good luck with your social media and Enterprise 2.0 efforts.  Please don't hesitate to ask questions below or contribute your own wisdom.

 

How Social Could Disrupt Search

The inimitable Fred Wilson concluded yesterday in "Why Social Beats Search", that "[m]achines can help us find what is good. But with the help of machines, our friends and trusted sources can and will do that even better."  This statement capped off a fresh debate over the weekend about automated, keyword-driven "McContent" creation that started when Michael Arrington posted about "the end of hand crafted content". Richard MacManus also explored the same issues in "Content Farms: Why Media, Blogs, and Google Should Be Worried".

I find this discussion very intriguing because it's nearly a mirror-image to the still-unfolding story of the last big change in this space: How the volume and timeliness of social media has disrupted traditional media.  I explored this subject in-depth recently on ZDNet about how this same transformation is now happening more broadly to other industries as well.  Now we're full circle already: What went around with social media is coming around again rather quickly with McContent. The machines are in the upstart role this time and have the potential to displace social media "moms and pops" who might not be able to match the volume and speed at which automated content can be created.  TWill Social Surpass Search?hat of course, depends on if you believe that machines can match the quality of handmade content.  And indeed, if quality ultimately matters as much as volume and timeliness. There's a balance here that I'm not sure we fully understand yet but I'm betting there's probably room for the full spectrum.  This will only be true, however, if we are prepared to accept that the online landscape and current ways of doing business are going to continue to evolve rapidly.

The premise of today's information abundance reaching an unsustainable place isn't a new one. Information overload is a  rapidly growing subject that a lot of smart folks are talking about these days.  One bright area however, and this is the point that Fred Wilson touches on, is that social systems might actually provide an effective filter that will separate the wheat from the chaff by decentralizing the expertise and work of content curation into a sort of crowdsourced collaborative process (an increasingly widespread approach.)  This could make it both scalable and sustainable and I do believe that social content curation is an important trend.  But it's only one step in the right direction as we head into the future dominated by truly vast information abundance. 

One holy grail of search is "search that finds you" just as and when you actually need it.  Encouragingly, I'm now starting to see this happen with social environments like Twitter where I've received more and more tweets lately in the vein of "@dhinchcliffe Thanks for the link, was just looking for this 10 mins ago!" This is the seed of a trend that could be exploited by a very smart company that created the right product design that systematically optimized social recommendations and content referrals into something so much more than it is today: ad hoc serendipity.  Will the company that does this be the ones with the largest active social networks, such as Twitter or Facebook?  Or perhaps Google will figure it out as a component of real-time search?  Or will it just as likely be someone that no one has heard of yet? If the history of the Web is any guide, it will come from a place we won't anticipate.

I also suspect that other forces are in the running and may end up limiting the impact of distributed social curation, or more likely co-opting it.  Emerging trends like Web Squared and its autonomic filters and recommendation systems powered by data shadows as well as advanced forms of Enterprise 2.0 BI are just as likely to provide the solution in the medium to long-term.  Either way, search is only going to get better and social will certainly improve it.  That's not to say social won't disrupt search, but it may only complement the changes happening more broadly.  One big question is whether social can be made to scale enough to be routinely effective for most users.  In the end, that's the big question in my mind: The output of machines can always exceed that of people and that's not necessarily a bad thing as long as we still get access to both results when we need them.

Web Squared Emerges To Refine Web 2.0

It’s been five years now that we’ve begun to understand what Web 2.0 is, starting way back in 2003.  It’s been a fairly impressive if winding road as a new online generation was born.  But far from getting long in the tooth, along the way Web 2.0 became vitally important — even central in some cases — to the very future of global culture and business.  Oh certainly, sometimes we get tired of the term itself, and admittedly it doesn’t describe something necessarily new anymore, but what we just do these days.  But the concepts identified as Web 2.0 have proved to be highly insightful, even prescient, and are used around the world daily to guide everything from product development to the future of government.

It has been today’s commonplace use of intensely popular and deeply pervasive social networks, the outright transformation of industries such as media and software, and the growing dominance of 2.0 techniques such as user generated content, open business methods, crowdsourcing, Enterprise 2.0, open APIs — in the business world and elsewhere — which has been striking, even if there’s a long way to go in some quarters.  Indeed, Web 2.0 is not just here to stay, it seems to be a revolution that just keeps on rolling and giving us, as I often say in my talks, an amazingly resourceful and resilient new lens with which to 1) look at the world and 2) the network that now surrounds it, and 3) the opportunities between the two.

Now, to paraphrase Churchill, “Now this is not the end. … But it is, perhaps, the end of the beginning” of Web 2.0, many are starting to perceive deeper patterns and concepts within Web 2.0 practices.  We can perhaps now see more clearly the next steps towards what some would like to call Web 3.0, and which Tim O’Reilly and John Battelle have decided to dub Web Squared, the deeper explanation of which you can find here on the Web 2.0 Summit site.  Whether we need a new term and whether Web Squared will be as accepted, or at least as widely repeated, as Web 2.0 is perhaps unlikely but it nevertheless is a cogent, if necessarily, incomplete next progression.  What is certain is, that like Web 2.0, the ideas within it are useful new strategic ones that are emerging from the countless practical experiences of Internet practitioners everywhere in the world’s largest living laboratory: The Internet. 

At its core, the message is that the Web is becoming more autonomic, reflective, real-time, generative, and open while at the same time far more deeply embedded everywhere in the fabric of our environment.  And like what came before it, Web Squared is likely to have profound impact to the societies and organizations, either way, that choose to understand it or ignore it.

What is Web Squared?

While this is my personal take, Web Squared articulates a broader fusion between the world-at-large, the Web, and the people connected to it.  It’s a more extreme view of Web 2.0 while at the same time hinting that while social computing has been a major transformative force recently in the consumer world and beyond, the relentless growth of devices, network connectivity, and sensors into our lives across our homes, workplaces, and external environment is casting an growing “information shadow” that is increasingly hard to ignore.  At first glance this can seem to be an impersonal and inhuman concept as the network expands to surround everything and dominate the participation that so far at least is still driven (for a little bit longer anyway) by what people do and contribute online.  However, this bleak vision is tempered by the realization that far from being pushed to the side, we collectively must be the feedback loop that guides Web Squared through billions of daily interactions that makes it possible in the first place.  It’s the full environment, including us, which makes it all work. 

Comparing Web 1.0, Web 2.0, and Web Squared

The comparison above gives a cleaner, most succinct sense of what Web Squared is by comparing it to Web 1.0 and classic Web 2.0.  It’s not necessarily a generation beyond Web 2.0 since many of the concepts are simply more refined or focused.  But the “knobs” on many Web 2.0 ideas like collective intelligence, feedback loops, network effects, and so on are turned up quite a bit more and are fueled more directly by our interactions with the world as well as our synthesis of it.  We know now how to sharpen the scalpels that we use to design our online businesses a good bit better and Web Squared reflects this.

For my own part, it’s a useful evolution of Web 2.0 even if it’s not quite as dramatically transformative culturally.  However, the implications in terms of the types of new businesses that will be created have the potential to be as potent as Google (or even more so) when it comes to cornering the market on new classes of data and therefore entire industries, since most winners in on the online space have outsized dominance of their sectors.  You want to use Web 2.0 ideas for the most impact? Think in terms of Web Squared.

I’ll be exploring the concepts of Web Squared in further depth here as I am able, but it’s clear that we need to take what we’ve learned in the Web 2.0 era and focus on emerging techniques that seem particular promising.  In particular, I think the key aspect of Web Squared will lie in teaching our applications to “learn inferentially” from our online product’s strategic sets of data and drive out previously hidden value.  That’s where being able to create powerful new autonomic, environmentally-connected, database-driven applications, often from ecosystems of partners that will provide access to their data (for a price), has the potential to become a dominant new growth model for modern Web apps.  Whether this is video search engines built on top of YouTube’s content, near real-time language translation using peer production in social communities, or just better product/content recommendation engines remains to be seen.

But the chances are just as likely that entirely new types of break-out products will be created that we can only barely imagine today, like we did with Web 2.0 just a few short years ago.  These new applications will fully harness the Web of data powered by insightful and potent cooperative algorithms and novel integration strategies that are deeply connected to the world we live in.  Developing these generative algorithms will be the central challenge for a new era of product designer.  These product designers will be experts at developing interactive systems for drawing insight, collecting data, and creating value from instantaneous Web input.  We can look to examples such as the Netflix Prize as successful models for distributed, open design “studios” for creating, validating, and identifying the winners.  How we look at fostering innovation and harvesting it is being reshaped by this vision already.

One thing is for certain, however, the way we look at the network is in the process of taking another big leap.

I’d like to get together a list of Web Squared example applications and ideas.  Please leave your suggestions and comments below.

50 Essential Strategies For Creating A Successful Web 2.0 Product

I am fortunate enough to spend a lot of time looking at various online products and services in the development stage, mostly of the Web 2.0 variety, meaning they use one or more of the principles in the Web 2.0 set of practices.  It’s been going on 4 years now and what’s fascinating to me, despite the enormous amount of knowledge that we’ve accumulated on how to create modern Web applications, is how many of the same lessons are learned over and over again.

Wouldn’t it be handy if we had a cheat sheet that combined many of these lessons into one convenient list?  In this vein of thinking, I decided to sit down recently to capture are some of the most important lessons I’ve learned over the last few years along with some of the thinking that went into them.

The Web Community Gets Smarter Every Time It Builds A Product

If there’s one thing that the Web has taught us it’s that the network gets smarter by virtue of people using it and product development is no exception.  Not only do we have examples of great online applications and systems to point to and use for best practices, but the latest tools, frameworks, development platforms, APIs, widgets, and so on, which are largely developed today in the form of open source over the Internet, tend to accumulate many of these new best practices. I’ve lauded everything from frameworks like Rails, Cake PHP, and Grails to online community platforms like Drupal and Joomla as examples of guiding solutions that can be vital springboards for the next great Web product or service.  

However, most of the success of an online product, Web 2.0 or otherwise, comes from two things: Its software architecture and its product design.  It’s also the case that the story of any product is a story of ten of thousands of little decisions made throughout the life of the product, of which only a key — and heartbreakingly small — set will make much of a difference to its success on the network.  The list of strategies below tells part of the story of which decisions will make that critical difference.

What then is software architecture and product design when it comes to today’s Web applications?  The good news: They’re often the same as they’ve always been, albeit just a bit more extreme, though there are some additions for the 2.0 era as well: 


Software architecture determines a Web application’s fundamental structure and properties: Resilience, scalability, adaptability, reliability, changeability, maintainability, extensibility, security, technology base, standards compliance, and other key constraints, and not necessarily in that order. 

Product design determines a Web application’s observable function: Usability, audience, feature set, capabilities, functionality, business model, visual design, and more.  Again, not necessarily in priority order. 

Doing both of these top-level product development activities well, striking a healthy balance between them (one often dominates the other), and doing it with a small team, requires people with deep and multidisciplinary backgrounds in creating successful products across this extensive set of practice areas.  These people are often hard to find and extremely valuable.  This means it’s also not likely you’ll be able to easily put together a team with all the capabilities that are needed from the outset. 

Be prepared from the outset for on-the-job learning and study, relying on tools and products that embody best practices, and replicating only the best designs and ideas (while being very conscientious not to steal IP.)

Balancing Software Architecture and Product Design in Web 2.0 Applications

In this way, I’ve collected a set of strategies that address the most common issues that I see come up over and over again as online products go to market.  I’ve decided to share these with you so we can continue to teach the network, and consequently ourselves, a little bit more about how to make extraordinary Web applications that can really make a difference in the marketplace.

This of course is just my experience and is not intended to be a complete list of Web 2.0 strategies.  However, I think most people will find it a valuable perspective and useful cross check in their product design and development. And please keep in mind this list is for Web 2.0 applications, not necessary static Web sites, or traditional online Web presence, though there is much that here that can be applied to them to make them more useful and successful as well.

Finally, a good number of these strategies are not specifically Web 2.0 concepts.  They are on the list because they are pre-requisites to many Web 2.0 approaches and to any successful product created with software and powered by people.

Please add your own strategies in comments below for anything that I’ve missed.

50 Strategies For Creating A Successful Web 2.0 Product

1. Start with a simple problem.  All of the most successful online services start with a simple premise and execute on it well with great focus.  This could be Google with it’s command-line search engine, Flickr with photo sharing, Digg with user generated news.  State your problem simply: “I make it easier to do X”.  Focus on solving it elegantly and simply, only add features carefully.  Over time, complexity will become the enemy of both your product design and your software architecture, so start with as much focus as you can muster.

2. Create prototypes as early as possible.  Get your idea into a working piece of software as quickly as possible.  The longer you take to go through one entire cycle, the more unknown work you have ahead of you.  Not producing software also means that you are not getting better and better at turning the work of your team into the most important measurable output: Functioning software.  Throughout the life of your product, turning your ideas into software as quickly and inexpensively as possible will be one of the most important activities to get right.

3. Get people on the network to work with the product prototype rapidly and often.  The online world today is fundamentally people-centric.  If your product isn’t about them and how it makes their lives better, your product really doesn’t matter.  And if they’re not using your Web application as soon as possible, you just don’t know if you are building the right product.  Constant, direct feedback from real people is the most important input to our product design after your idea.  Don’t wait months for this to happen; get a beta out to the world, achieve marketplace contact in weeks, or at most a few months, and watch carefully what happens. This approach is sometimes called Web 2.0 Development .

4. Release early and release often.  Don’t get caught up in the massive release cycle approach, no matter how appealing it may be.  Large releases let you push off work tomorrow that should be done today.  It also creates too much change at once and often has too many dependencies, further driving an increase in the size of the release.  Small releases almost always work better, are easier to manage, but can require a bit more operations overhead. Done right, your online product will iterate smoothly as well as improve faster and more regularly than your competitors. Some online products, notably Flickr, have been on record as saying they make new releases to production up to several times a day.  This is a development velocity that many new startups have trouble appreciating or don’t know how to enable. Agile software development processes are a good model to start with and and these and even more extreme methods have worked well in the Web 2.0 community for years.

5. Manage your software development and operations to real numbers that matter.  One often unappreciated issue with software is its fundamentally intangible nature.  Combine that with human nature, which is to manage to what you can see, and you can have a real problem.  There is a reason why software development has such a variable nature in terms of time, budget, and resources.  Make sure you have as many real numbers as possible to manage to: Who is making how many commits a week to the source repository, how many registered users are there on a daily basis, what does the user analytics look like, which product features are being used most/least this month, what are the top 5 complaints of customers, and so on.  All of these are important key performance indicators that far too many startups don’t manage and respond to as closely as they should.

6. Gather usage data from your users and input it back into product design as often as possible.  Watch what your users do live with your product, what they click on, what do they try to do with it, what they don’t use, and so on.  You will be surprised; they will do things you never expected, have trouble with features that seem easy to you, and not understand parts of your product that seemed obvious.  Gather this data often and feed it back into your usability and information architecture processes.  Some Web applications teams do this almost daily, others look at click stream analytics once a quarter, and some don’t it at all.  Guess who is  shaping their product faster and in the right direction?

7. Put off irreversible architecture and product design decisions as long as possible. Get in the habit of asking “How difficult will it be to change our mind about this later?” Choosing a programming language, Web framework, relational database design, or a software interface tend to be one-way decisions that are hard to undo.  Picking a visual design, logo, layout, or analytics tool generally is not.  Consequently, while certain major decisions must be made up front, be vigilant for seemingly innocuous decisions that will be difficult to reverse.  Not all of these will be a big deal, but it’s all too often a surprise to many people when they discover their architecture isn’t malleable in the places that they want it to be.  Reduce unpleasant surprises by always asking this question.

8. Choose the technologies later and think carefully about what your product will do first.  First, make sure your ideas will work on the Web. I’ve seen too many startups with ideas that will work in software but not on the Web.  Second, Web technologies often have surprising limits, Ajax can’t do video or audio, Flash is hard to get to work with SEO for example.  Choosing a technology too early will constrain what is possible later on.  That being said, you have to choose as rapidly as you can within this constraint since you need to build prototypes and the initial product as soon as you are able.

9.  When you do select technologies, consider current skill sets and staff availability.  New, trendy technologies can have major benefits including higher levels of productivity and compelling new capabilities, but it also means it’ll be harder to find people who are competent with them.  Having staff learn new technology on the job can be painful, expensive, and risky.  Older technologies are in a similar boat; you can find people that know them but they’ll most likely not want to work with them.  This means the middle of the road is often the best place to be when it comes to selecting technology, though you all-too-often won’t have a choice depending on what your staff already knows or because of the pre-requisites of specific technologies that you have to use.

10. Balance programmer productivity with operational costs.  Programming time is the most expensive part of product creation up front while operations is after you launch.  Productivity-oriented platforms such as Ruby on Rails are very popular in the Web community to drive down the cost of product development but can have significant run-time penalties later when you are supporting millions of users.  I’ve previously discussed the issues and motivations around moving to newer programming languages and platforms designed for the modern Web, and I encourage you to read it. Productivity-oriented platforms tend to require more operational resources during run-time, and unlike traditional software products, the majority of the cost of operations falls upon the startup.  Be aware of the cost and scale of the trade-offs since every dollar you save on the development productivity side translates into a run-time cost forever after on the operations side.

11. Variability in the productivity amongst programmers and development platforms each varies by an order of magnitude.  Combined together and your choice of programming talent and software development platforms can result in a 100x overall effect on product development productivity.  This means that some teams can ship product in as little as 3 months and some projects won’t ship ever, at least not without truly prohibitive time and resource requirements.  While there are a great many inputs to an Internet startup that will help or hinder it (take a look at Paul Graham’s great 18 Mistakes That Kill Startups for a good list), these are two of the most central and variable: Who is developing the product and what development platform they are using. Joel Spolsky’s write-up on programmer productivity remains one of the best at understanding this issue.  It usually turns out that paying a bit more for the right developer can often mean tremendous output gains.  One the other side of the coin, choosing a development platform not designed for creating modern Web applications is another decision that can sap your team of productivity while they spend months retrofitting it for the features they’ll need to make it work properly in today’s Internet world. 

12. Plan for testing to be a larger part of software development process than non-Web applications.  Cross browser testing, usability, and performance/load testing are much bigger issues with Web applications than many non-Web applications.  Having to do thorough testing in a half-dozen to a dozen browser types can be an unexpected tax on the time and cost of creating a Web product.  Doing adequate load testing is another item that often waits until the end, the very worst time to find where the bottlenecks in your architecture are.  Plan to test more than usual.  Insist on automated unit and integration tests that build up over time and run without having to pay developers or testers to do it manually.

13. Move beyond traditional application hosting. Single Web-server hosting models are not going to suffice for your 2.0 applications.  Reliability, availability, and scalability are essential and must be designed into your run-time architecture and supported by your hosting environment.  Solutions like 3Tera, Amazon’s Elastic Compute Cloud, and Google’s App Engine are three compelling, yet very different solutions to the hosting problem.  Either way, grid and cloud approaches to hosting will help you meet your growth and scalability requirements while managing your costs.

14. Have an open source strategy.  This has two important aspects.  One, developing and hosting a product built with open source software (the ubiquitious LAMP stack) is almost always much less expensive than using commercial software and is what most online products use.  There are certainly commercial licenses that have fair terms for online services, but almost none of them will match the cost of free.  This is one reason why you won’t find Windows or Oracle embedded in very many Web 2.0 services.  Two, you’ll have to decide whether to open source or commercial open source your product.  This has entirely to do with what your product does and how it does it, but an increasing number of Web 2.0 hosted products are releasing their offerings as open source to appeal to customers, particularly if they are business customers. Done right, open sourcing can negate arguments about the size of your company while enlisting many 3rd party developers to help enrich and make your product better.

15. Consider mobile users as important as your regular browser customers.  Mobile devices will ultimately form the majority of your user base as the capability and adoption of smartphones, Internet tablets, laptops, and netbooks ushers in mobile Web use as the dominant model.  Having an application strategy as well as well-supported applications for the iPhone, Android, and RIM platforms is essential for most Web products these days.  By the time you get to market, mobile will be even more important than it is now.  Infoworld confirmed today, in fact, that wireless enterprise development will be one of 2009′s bright spots.

16. Search is the new navigation, make it easy to use in your application.  You have 5-10 seconds for a new user to find what they want from your site or application.  Existing users want to directly access what they need without going through layers of menu items and links.  Search is the fastest way to provide random access navigation.  Therefore, offer search across data, community, and help at a minimum.  A search box must be on the main page and indeed, every page of the modern Web application. 

17. Whenever users can provide data to your product, enable them.  Harnessing collective intelligence is the most central high-level principle of Web 2.0 applications.  To be a major online competitor, getting your millions of users to build a valuable data set around the clock is the key to success. Many product designers look at this too narrowly and usually at a small set of data.  Keep a broad view of this and look for innovative ways to get information from explicit contributions to the database of intentions can form your architecture of participation.

18. Offer an open API so that your Web application can be extended by partners around the world.  I’ve covered this topic many times in the past and if you do it right, your biggest customers will soon become 3rd party Web applications building upon your data and functionality.  Critically, offering an API converts your online product into an open platform with an ecosystem of 3rd party partners.  This is just one of many ways to realize Jakob’s law, as is the next item.

19. Make sure your product can be spread around the Web by users, provide widgets, badges, and gadgets.  If your application has any success at all, your users will want to take it with them and use your features elsewhere.  This is often low-effort but can drive enormous growth and adoption; think about YouTube’s badge. 

20. Create features to make the product distribute virally.  The potency of this is similar to widgets above and everything from simple e-mail friend invites to importing contact lists and social graphs from other Web apps are critical ways to ensure that a user can bring the people they want into the application to drive more value for them and you.   

21. The link is the fundamental unit of thought on the Web, therefore richly link-enable your applications.  Links are what make the Web so special and fundamentally makes it work.  Ensuring your application is URL addressable in a granular way, especially if you have a rich user experience, is vital to participate successfully on the Web.  The Web’s link ecosystem is enormously powerful and is needed for bookmarking, link sharing/propagation, advertising, makes SEO work, drives your page rank, and much more.  Your overall URL structure should be thought out and clean, look to Flickr and del.cio.us for good examples.

22. Create an online user community for your product and nurture it.  Online communities are ways to engage passionate users to provide feedback, support, promotion, evangelism and countless other useful outcomes.  While this is usually standard fare now with online products, too many companies don’t start this early enough or give it enough resources despite the benefits it confers in terms of customer support, user feedback, and free marketing, to name just three benefits.  Investing in online community approaches is ultimately one of the least expensive aspects of your product, no matter the upfront cost. Hire a good community manager and set them to work.

23. Offer a up-to-date, clean, compelling application design.  Attractive applications inherently attract new customers to try them and is a pre-requisite to good usability and user experience.  Visual and navigational unattractiveness and complexity is also the enemy of product adoption.  Finally, using the latest designs and modes provides visual cues that conveys that the product is timely and informed.  A good place to start to make sure you’re using the latest user experience ideas and trends is Smashing Magazine’s 2009 Web Design survey.

24. Load-time and responsiveness matter, measure and optimize for them on a regular basis.  This is not a glamorous aspect of Web applications but it’s a fundamental that is impossible to ignore.  Every second longer a key operation like main page load or a major feature interaction takes, the more likely a customer is to consider finding a faster product.  On the Web, time is literally money and building high speed user experiences is essential.  Rich Internet Application technologies such as Ajax and Flash, albeit used wisely, can help make an application seem as fast as the most responsive desktop application. Using content distribution networks and regional hosting centers.

25. User experience should follow a “complexity gradient.”  Novice users will require a simple interface but will want an application’s capabilities to become more sophisticated over time as they become more skilled in using it.  Offering more advanced features that are available when a user is ready but are hidden until they are allows a product to grow with the user and keeps them engaged instead of looking for a more advanced alternative. 

26. Monetize every page view.  There is no excuse for not making sure every page is driving bottom-line results for your online business.  Some people will disagree with this recommendation and advertising can often seem overly commercial early in a product’s life.  However, though a Web application should never look like a billboard, simple approaches like one line sponsorships or even public service messages are good ideas to maximize the business value of the product and there are other innovation approaches as well.

27. Users’ data belongs to them, not you.  This is a very hard strategy for some to accept and you might be able to get away with bending this rule for a while, that is, until some of your users want to move their data elsewhere.  Data can be a short-term lock-in strategy, but long-term user loyalty comes from treating them fairly and avoiding a ‘Roach Motel’ approach to user data (“they can check-in their data, but they can’t check out.”) Using your application should be a reversible process and users should have control of their data.  See DataPortability.org for examples of how to get started with this.

28. Go to the user, don’t only make them come to you.  The aforementioned APIs and widgets help with this but are not sufficient.  The drive strong user adoption, you have to be everywhere else on the Web that you can be.  This can mean everything from the usual advertising, PR, and press outreach but it also means creating Facebook applications, OpenSocial gadgets, and enabling use from mashups. These methods can often be more powerful than all the traditional ways combined.

29. SEO is as important as ever, so design for it.  One of the most important stream of new users will be people coming in from search engines looking for exactly what you have.  This stream is free and quite large if you are ensuring your data is URL addressable and can be found via search engine Web crawlers.  Your information architecture should be deeply SEO-friendly and highly granular.

30. Know thy popular Web standards and use them.  From a consumer or creator standpoint, the data you will exchange with everyone else will be in some format or another.  And the usefulness of that data or protocol will be in inverse proportion to how well-known and accepted the standard is.  This generally means using CSS, Javascript, XHTML, HTTP, ATOM, RSS, XML, JSON, and so on. Following open standards enables the maximum amount of choice, flexibility, time-to-market, access to talent pools, and many other benefits over time to both you and your customers.

31. Understand and apply Web-Oriented Architecture (WOA). The Web has a certain way that it works best and understanding how HTTP works at a deep level is vital for getting the most out of the unique power that the Internet has to offer.  But HTTP is just the beginning of this way of thinking about the Web and how to use its intrinsic power to be successful with with it.  This includes knowing why and how link structure, network effects, SEO, API ecosystems, mashups, and other aspects of the Web are key to making your application flourish.  It’s important to note that your internal application architecture is likely not fundamentally Web-oriented itself (because most software development platforms are not Web-oriented) and you’ll have to be diligent in enabling a WOA model in your Web-facing product design.  The bottom line: Non-Web-oriented products tend not to fare very well by failing to take advantage of the very things that have made the Web itself so successful.

32. Online products that build upon enterprise systems should use open SOA principles. Large companies building their first 2.0 products will often use existing IT systems and infrastructure that already have the data and functionality they need.  Although they will often decouple and cache them for scalability and performance, the connectedness itself is best done using the principles of SOA. That doesn’t necessarily mean traditional SOA products and standards, although it could, often using more Web-oriented methods works better.  What does this really mean? Stay away from proprietary integration methods and use the most open models you can find, understanding that the back-end of most online products will be consumed by more than just your front-end (see API discussion above for a fuller exploration).

33. Strategically use feeds and syndication to enable deep content distribution.  This is another way to use Jakob’s Law to increase unintended uses and consumption of an application from other sites and ecosystems.  Feeds enable many beneficial use cases such as near real-time perception of fresh data in your application from across the Web in feed readers, syndication sites, aggregators, and elsewhere.  Like many other techniques here, knee-jerk use of feeds won’t drive much additional usage and adoption, but carefully designing feeds to achieve objectives like driving new customers back to the application directly from the feed can make a big difference.  Failing to offer useful feeds is one of the easiest ways to miss out on business opportunities while giving your competitors an edge.

34. Build on the shoulders of giants; don’t recreate what you can source from elsewhere.  Today’s Internet application usually require too much functionality to be cost-effectively built by a single effort.  Typically, an application will actually source dozens of components and external functionality from 3rd parties.  This could be off-the-shelf libraries or it could be the live use of another site’s API, the latter which has become one of the most interesting new business models in the Web 2.0 era.  The general rule of thumb: Unless it’s a strategic capability of your application, try hard to source it from elsewhere before you build it; 3rd parties sources are already more hardened, robust, less expensive, and lower defect than any initial code could that you could produce. Get used to doing a rapid build vs. buy evaluation for each major component of your application.

35. Register the user as soon as possible.  One of the most valuable aspects of your onine product will be the registered user base.  Make sure you application gives them a good reason to register and that the process is as painless as possible.  Each additional step or input field will increase abandonment of the process and you can always ask for more information later. Consider making OpenID the default login, with your local user database a 2nd tier, to make the process even easier and more comfortable for the user.

36. Explicitly enable your users to co-develop the product.  I call this concept Product Development 2.0 and it’s one of the most potent ways to create a market-leading product by engaging the full capabilities of the network.  The richest source of creative input you will have is your audience of passionate, engaged users.  This can be enabled via simple feedback forms, harvested from surveys and online community forums, via services such as GetSatisfaction, or as the ingredients to mashups and user generated software. As you’ll see below, you can even open the code base or provide a plug-in approach/open APIs to allow motivated users and 3rd parties to contribute working functionality.  Whichever of these you do, you’ll find that the innovation and direction to be key to making your product the richest and most robust it can be.  A significant percentage of the top online products in the world take advantage of this key 2.0 technique.

37. Provide the legal and collaborative foundations for others to build on your data and platform.  A good place to start is to license as much of your product as you can via Creative Commons or another licensing model that is less restrictive and more open than copyright or patents. Unfortunately, this is something for which 20th century business models around law, legal precedent, and traditional product design are ill-equipped to support and you’ll have to look at what other market leaders are doing with IP licensing that is working.  Giving others explicit permission up-front to repurpose and reuse your data and functionality in theirs can be essential to drive market share and success. Another good method is to let your users license their data as well and Flickr is famous for doing this.  It’s important to understand that this is now the Some Right Reserved era, not the All Rights Reserved era.  So openly license what your have for others to use; the general rule of thumb is that the more you give away, the more you’ll get back, as long as you have a means of exercising control.  This is why open APIs have become as popular as they have, since they are essentially “IP-as-a-service” and poorly behaving partner/licensees can be dealt with quickly and easily.

38. Design your product to build a strong network effect.  The concept of the network effect is something I’ve covered here extensively before and it’s one of the most important items in this list.  At their most basic, Web 2.0 applications are successful because they explicitly leverage network effects successfully. This is the underlying reason why most of the leading Internet companies got so big, so fast.  Measuring network effects and driving them remains one of the most poorly understood yet critical aspects of competing successfully online.  The short version: It’s extremely hard to fight an established network effect (particularly because research has shown them to be highly exponential).  Instead, find a class of data or a blue ocean market segment for your product and its data to serve.

39. Know your Web 2.0 design patterns and business models.  The fundamental principles of Web 2.0 were all identifid and collected together for a good reason.  Each principle is something that must be considered carefully in the design of your product given how they can magnify your network effect. Your development team must understand them and know why they’re important, especially what outcomes they will drive in your product and business.  It’s the same with Enterprise 2.0 products: There is another, related set of design principles (which I’ve summarized as FLATNESSES) that makes them successful as well.  And as with everything on this list, you don’t apply 2.0 principles reflexively; they need to be intelligently used for good reason.

40. Integrate a coherent social experience into your product.  Social systems tend to have a much more pronounced network effect (Reed’s Law) than non-social systems. Though no site should be social without a good reason, it turns out that most applications will benefit from having a social experience.  What does this mean in practice? In general, social applications let users perceive what other users are doing and actively encourage them to interact, work together, and drive participation through social encouragement and competition.  There is a lot of art to the design of the social architecture of an online product, but there is also an increasing amount of science.  Again, you can look at what successful sites are doing with their social interaction but good places to start are with user profiles, friends lists, activity streams, status messages, social media such as blogs and microsharing, and it goes up from there.  Understand how Facebook Connect and other open social network efforts such as OpenSocial can help you expand your social experience.

41. Understand your business model and use it to drive your product design.  Too many Web 2.0 applications hope that they will create large amounts of traffic and will then find someone interested in acquiring them.  Alternatively, some products charge too much up front and prevent themselves from reaching critical mass. While over-thinking your exit strategy or trying to determine your ultimate business model before you do anything isn’t good either, too many startups don’t sit down and do the rigorous thinking around how to make their business a successful one in the nearer term.  Take a look at Andrew Chen’s How To Create a Profitable Freemium Startup for a good example of the framework on how to do some of the business model planning.  Taking into account the current economic downturn and making sure you’re addressing how you offering can help people and businesses in the current business climate will also help right now.

42. Embrace emergent development methods.  While a great many of the Web’s best products had a strong product designer with a clear vision that truly understood his or her industry, the other half of the equation that often gets short shrift is the quality of emergent design through open development.  This captures the innate crowdsourcing aspects of ecosystem-based products, specifically those that have well-defined points of connectedness with external development inputs and 3rd party additions.   Any Web application has some emergent development if it takes development inputs or extensibility with via 3rd party plug-ins, widgets, open APIs, open source contributions, and so on.  The development (and a good bit of the design) of the product then “emerges” as a function of multiple inputs.  Though there is still some top-down control, in essence, the product becomes more than the sum total of its raw inputs.  Products like Drupal and Facebook are good examples of this, with thousands of plug-ins or 3rd party apps that have been added to them by other developers.

43. It’s all about usability, usability, and usability.  I’ve mentioned usability before in this list but I want to make it a first class citizen.  Nothing will be a more imposing barrier to adoption that people not understanding how your product works.  Almost nothing on this list will work until the usability of your application is a given.  And hands down the most common mistake I see are Web developers creating user experiences in isolation.  If you’re not funded to have a usability lab (and you probably should be, at some level), then you need to grab every friend and family member you have to watch how they use your application for the first time.  Do this again for every release that makes user experience changes.  You will change a surprising number of assumptions and hear feedback that you desperately need to hear before you invest any more in a new user experience approach.  This now true even if you’re developing enterprise applications for the Web.

44. Security isn’t an afterthought.  It’s a sad fact that far too much of a successful startup’s time will be spent on security issues.  Once you are popular, you will be the target of every so-called script kiddie with a grudge or with the desire to get at your customer data, etc. Software vulnerability are numerous and the surface area of modern Web apps large. You not only have your own user experience but also your API, widgets, semantic Web connections, social networking applications, and other points of attack.  Put aside time and budget for regular vulnerability assessments.  You can’t afford a public data spill or exploit due to a security hole that will compromise your user’s data, or you may well find yourself with a lot of departing customers.Web 2.0 applications also need unique types of security systems, from rate limiters to prevent valuable user-generated data from being systematically scraped from the site (this is vital to “maintaining control of unique and hard-to-re-create datasets”) to monitoring software that will screen for objectionable or copyrighted contributions.

45. Stress test regularly and before releases.  It’s a well known saying in the scalability business that your next bottleneck is hiding just behind your last high water mark. Before your launch, data volumes and loads that work fine in the lab should be tested to expected production volumes before launch.  The Web 2.0 industry is rife with examples of companies that went down the first time they got a good traffic spike.  That’s the very worst time to fail, since it’s your best chance of getting a strong initial network effect and may forever reduce your ultimate success.  Know your volume limits and ceilings with each release and prepare for the worst.

46. Backup and disaster recovery, know your plan.  This is another unglamorous but essential aspect for any online product.  How often are backups being made of all your data? Are the backups tested? Are they kept offsite? If you don’t know the answers, the chances that you’ll survive a major event is not high.

47. Good Web products understand that there is more than the Web.  Do you have a desktop widget for Vista or the Mac? Might you benefit from offering an Adobe AIR client version of your application? How about integration and representation in vitual worlds and games?  How about linkages to RFID or GPS sensors? Startups thinking outside the box might even create their own hardware device if it makes sense (see Chumby and the iPhone/iPod for examples).  If one thing that is certain is that the next generation of successful Web startups will only partially resemble what we see today.  Successful new online products will take advantage of “software above the level of a single device” and deliver compelling combinations of elements into entirely new products that are as useful and innovative as they are unexpected.  A great Web 2.0 product often has a traditional Web application as only part of its overall design, see the Doritos Crash the Superbowl campaign for just one small example of this.

48. Look for emerging areas on the edge of the Web.  These are the spaces that have plenty of room for new players and new ideas, where network effects aren’t overly established and marketshare is for the taking. What spaces are these? The Semantic Web seems to be coming back with all new approaches (I continue to be amazed at how much appears about this topic on http://delicious.com/popular/web3.0 these days.) Open platform virtual worlds such as Second Life were hot a few years ago and may be again.  Mobile Web applications are extremely hot today but slated to get over crowded this year as everyone plans a mobile application for phone platforms.  What is coming after this?  That is less clear but those that are watching closely will benefit the most.

49.  Plan to evolve over time, for a long time.  The Web never sits still.  Users change, competitors improve, what’s possible continues to expand as new capabilities emerge in the software and hardware landscape.  In the Perpetual Beta era, products are never really done.  Never forget that, continue to push yourself, or be relegated to a niche in history.

50. Continually improve yourself and your Web 2.0 strategies. While process improvement is one of those lip-service topics that most people will at least admit to aspire to, few have the time and resources to carry it out on a regular basis.  But without that introspection on our previous experience we wouldn’t have many of the “aha” moments that drove forward our industry at various points in term.  Without explicit attempts at improvement, we might not have developed the ideas that became object-oriented languages, search engine marketing, Web 2.0, or even the Internet itself.  This list itself is about that very process and encapsulates a lot of what we’ve learned in the last 4 years.  Consequently, if you’re not sitting down and making your own list from your own experiences, you’re much more likely to repeat past history, never mind raising the bar.  Like I’m often fond of saving; civilization progresses when we make something that was formerly hard to do and make it easy to do.  Take the time, capture your lessons learned, and improve your strategies.

What else is missing here? Please contribute your own 2.0 strategies in comments below:

You can also get help with these strategies from a Web 2.0 assessment, get a deeper perspective on these ideas at Web 2.0 University, or attend our upcoming Economics 2.0 workshop at Web 2.0 Expo SF on March 31st, 2009.

The Social Graph: Issues and Strategies in 2008

One of the hottest topics in the online world in the last couple of years has been the growth of social networking services such as Facebook and MySpace, as well as the addition of a social element to existing user experiences.  Despite riding several waves of hype, it's now clear that the social networking space will only get hotter in 2008 according to most watchers.  Social software has come fully into its own as of 2008 — for all appearances permanently — and understanding the reasons for this rapid rise as well as figuring out how to leverage it best is the job of everyone who wants to make the most of the Web 2.0 era.

Gaining a deeper insight to the social networking phenomenon, now exhibited by the tens of millions of users employing them globally on a daily basis for both personal and businesses uses, currently means understanding the fundamental unit of the social network, also one of the biggest new buzzphrases of the year: the social graph.  Fortunately, that's simple enough despite the term's oblique reference to graph theory, which it is heavily based upon.

Social Graphs - The pattern of social relationships between people

Simply put, a social graph is a set of people, referred to as nodes, that are connected together by vertices — better known as links or connections — that reflect their social relationships.  You can see a conceptual social graph above, showing the typical distinction of social networks to reflect whether a connection with another person is direct or indirect.  For example, the popular business social networking service LinkedIn, uses this model and sorts a member's social graph into different degrees of separation, which you can see a typical example of below and taken from my LinkedIn profile:

 

Organizing Social Graphs - Degress of separation is popular

Also becoming popular is the burgeoning field of social analytics, such as the Socalistics application in Facebook and the Interactive Friends Graph, though there are also commercial standalone products here or on the way for the enterprise and open Web spaces from companies like KnowNow and Bravadosoft.  The Interactive Friends Graph is a nice, simple example anyone can try on their own and you can see mine from Facebook below.  Hovering over nodes in the live version in your Facebook profile allows you to see who is connected to others in your network and begin to gain insight and understanding of the relationships in your network.

 Social Graph Example - One of many way to depict a social graph

But what are the top issues one must understand about the social graph in 2008?  As I've seen social networks become common on corporate intranets and in daily use on the Web, some of the issues are rapidly becoming clear.  However, the full story will certainly continue to unfold for the next several years at least.  Here's what we're seeing at the moment:

Strategies and Issues for the Social Graph – Circa 2008

  • The social graph is poised to replace the address book and contact list as the preferred organizing structure for personal and business relationships. This was one of my Web 2.0 predictions for 2008 and it won't fully come true for the majority of users for at least several years since there's such an installed base of traditional tools for managing relationship information.  What's the difference?  Social networks are usually opt-in, two-ways for one.  And they are social for another, meaning they tend to encourage communication and collaboration, such as through user profile event streams and status messages.  They also offer up and actively make use of the deeper insight into the full graph's social surface area beyond direct contacts, such as LinkedIn's introduction service.
  • Ownership of the social graph is going to be a ground zero issue in 2008.  Robert Scoble's widely covered attempt recently to use Plaxo Pulse to export his 5,000 Facebook contacts recently got him banned temporarily from the service.  But as users begin to realize that the contact lists they are building using online Web tools might not be portable, this will become a growing concern, particularly since two-way opt-in makes a social graph more valuable (and accurate) but significantly harder to recreate on demand elsewhere. This takes us to our next subject…
  • Many social networking services will adopt open data initiatives.  Both Google and Facebook recently showed support for DataPortability.org and Google has an interesting play in their OpenSocial initiative.  This is welcome news that will resolve some of the concerns around who owns the graph but interestingly, traditional corporations will be the slowest get this and will rarely let workers take their hard won social graphs and user profiles with them elsewhere as they move to new jobs.  Public social networking sites Web sites are leading the way here and this will only drive more business users to the open Web, where they at least have some control over their social graph.  Smart organizations will provide their workers with some form of open social graph support, lest they lose control completely as workers keep more and more of their graph in Facebook, LinkedIn, and Plaxo and not in prescribed relationship management tools.
  • Attempts to monetize social graphs will drive interest in regulation and legislation.  Social networking is now a global Internet phenomenon and that the information contained within them is highly central to everyone's lives.  This will make everything from protecting children to individual privacy of social graphs a hot issue for some local and federal governments.  All it will take is one or two widely covered exploits to make this happen.  Expect the European Union and the U.S. government to begin seriously examining the issue this year with many other governments following suite.  Good citizenship of sites that manage social graphs will be essential to prevent excessive government involvement.
  • The line is blurring between personal and business use of social graphs.  We're all rapidly getting one large social graph each already, with everyone we know in them.  Most public social networking sites do a poor job of separating different subgroups of our social networks, such as allowing pictures and status messages to only go to a specific subgroups (work messages to business, family message to family, friends messages to friend, etc.)  This actually works a little bit better in enterprise social networks, but not much, since it largely consists of a Contact Type field.  Segmentation of social graphs will be an increasingly requested feature by users struggling with their use.  The social graph management services that make this distinction and enable its leverage may do very well indeed.
  • Open Web identity, which will ultimately form the global "primary key" for social graph nodes, will not get anywhere soon.  This despite it being needed badly but the users of the Web have not yet felt compelled to demand it.  Data portability of social graphs will begin to drive adoption of user controlled Web identity, and hopefully government regulation will not.  See Dare Obasanjo's deep exploration of using openid to enable social graph interoperability as an example of what will need to happen, despite there being little incentive currently for sites to use other site's openids.
  • Making social networking "gardening" and administration easier will drive new innovations.  Most individual social graphs are primarily tended by hand today, although a growing number of products, such as Visible Path, do all the tedious work for you by watching your social interaction online such as through tight integration through e-mail and instant messaging, building a rich graph for you (even sending invitations) as you go about your daily social activities.  New innovations like these will make social graphs easier to maintain and richer in overall information while also driving adoption through ease of use.
  • The optional two-way confirmation of a social graph link becoming standard.  Many social graph management platforms (Facebook and Linked for example) require confirmation from the other side of the connection before adding a person to your graph.  Sites like Spock, which make it optional, will ultimately be more practical for managing a social graph while still allowing discernment of two way confirmations, which tend to be more valuable and convey key information about the trust and real extent of a social relationship.
  • Social networking fatigue will not set in as perceived constraints such as Dunbar's limit do not prove to be universal.  While there are many theories on how big a social graph can get before it become unmanageable and sees diminishing returns on growth (note that both Facebook and LinkedIn encourage ceilings), the fact is that the are many different purposes for a social graph, from data mining and historical research, to marketing and customer relationship management.  

What else is going to be key to dealing with the social graph in 2008?  Please leave in comments below and I'll update this post with any good submissions.

Ten Aspects of Web 2.0 Strategy That Every CTO and CIO Should Know

Over the last year I’ve worked with organizations around the world that are attempting to grapple with Web 2.0 and the growing external marketplace pressure being exerted for the change and transformation of their businesses. Along the way, I’ve been fortunate enough to be able to identify and assemble a working list of some consistent recurring issues and themes around Web 2.0 strategy.  I’ve provided them below at a high level. Your comments and additions are very welcome as we try to frame up a consistent picture of what’s happening in the marketplace.

It used to be a little surprising how long it’s taken for Web 2.0 to begin to have serious impact on or even high-level interest in the business world.  However, the ideas have had staying power and have also largely been validated; there are now fundamentally different and very powerful new models for engaging with customers, designing our products, and applying technology in general to our business that are proven and have growing bodies of knowledge.  The Web has become the single most important driving force in many fields of endeavor as well as the leading source of both innovation and potent new modes for communicating, collaborating, socializing, and working together. It’s taken a few years but businesses are now feeling the change in the air.

 

The Web 2.0 Transformation and Change Management Process for Business and Enterprises for CTOs and CIOs

 

However, as I’ve said a number of times in my various discussions of Web 2.0, the power of the network has deep roots in some profound shifts in society and culture, particularly the singular move from push-based systems (the 1.0 era going way, way back until right around now) to pull-based systems (the 2.0 era from roughly a few years into this century and going forward).  That this shift is well under way is clear if you look at the sudden explosion of the blogosphere, social networking, social media, open source software, online communities, and peer production in virtually all things.  The good news (or bad news, depending on how you look at it) is that despite the remaking of more than a few industries already — including media, software, advertising — this shift is only just beginning.

This all raises the question of how to make the transition from 1.0 to 2.0 safely and non-disruptively with your business largely intact, perhaps even with a superior competitive position.  That this transition can actually be accomplished by most businesses is still far from clear though some early transitions have met with varying degrees of success.  This list represents some of what we’ve learned so far  about 2.0 transformation but it’s something that strikes at the very heart of most businesses today: The rules for success are not-so-gradually changing and the marketplace is driving it in an often-subversive grassroots, bottom-up way.  The question now is no longer about “if” but increasingly about thriving long-term, period: What are you willing to do to adapt to a new business world?

This list is aimed primarily at CTOs and CIOs since they are mostly likely to be located at the convergence of traditional business thinking and the wave of 2.0 change coming in off the network. However these ideas apply to anyone looking at how to embrace 2.0 transformation in their organization and take advantage of it.  This is one of the most exciting eras to be in businesses since so many directions are in flux and the outcomes, players, and market leaders of the near future are far from certain.  Those who can see the new opportunities clearly through the lens of 2.0 transformation not only have a fighting chance, but are able to seize them with once-in-a-generation ease.

Note: I’ve dropped the “Web” in Web 2.0 for this discussion because one of the big lessons is that many traditional business thinkers turn off when they hear the word, even though Web 2.0 design patterns and business models have truly profound implications across any business today.  Consequently, hat the Web is driving most of these changes is being considered incidental for this discussion (though it’s absolutely the opposite when actually executing on these new models.) Instead, this is targeted a discussion about the transformative models themselves (such as who creates the products and where, how they are used, who supports them, how are they remixed, syndicated, franchised, licensed, IP protected, etc) in a strategic businesses sense. At the core of this discussion is how 1.0 business models of the 20th century are very much being eroded, transformed, and frequently dethroned by the immense motive forces that lie in the pervasive, open networked systems we have today, which are taking us deeply into a very new place: the 2.0 era.

Ten Key Aspects of Web 2.0 Strategy

  1. It’s not about technology, it’s about the changes it enables.  While technology is a close second (and ultimately makes 2.0 business models possible), the real discussion is about the disruptive new opportunities it creates.  Instead the discussion should be focused more around strategies such as harnessing millions of customers over the network to co-create products through peer production, engaging in mass customer self-service, customer communities, and open supply chains to thousands of ad hoc partners with open APIs. These are just some of the examples of using the network to create far richer and more profound results than could be created in the 1.0 era.  Don’t get caught up in the technology of 2.0 at first other than to understand the business possibilities it affords.  Avoid technology-first discussions like the plague.  Premature monetization discussions around 2.0 are also to be avoided, they tend to have a negative impact on process if done too early.
  2. The implications of 2.0 stands many traditional views on their head and so change takes more time than usual.  In the 2.0 world customers and partners have a much closer, more sustained relationship because of social interaction and tightly integrated online supply chains, to name just two reasons.  The shift of control from institutions to communities of users takes a lot of getting used to.  Just understanding how and why intellectual property is better covered by Creative Commons instead of copyright will take the legal department years (if not decades).  Each part of the organization will have its miniature 2.0 revolution.  These take time to happen and sort themselves out.  This means getting these new ideas into people’s heads is one of the first steps…
  3. Get the ideas, concepts, and vocabulary out into the organization and circulating.  If you’re trying to affect 2.0 change in an organization, there’s no better solution that exposing people to it.  Demographics can be a problem in this situation depending on the industry.  Younger workers tend to live and breath 2.0 while older workers may be aware of it but don’t think it applies to them.  I use point education where change needs to happen either first or quickly and then internal communities that bring the discussion of change, innovation, and transformation to the entire organization.  Either way, learning and education around 2.0 are a vital trigger to begin change and should be started early and non-disruptively.
  4. Existing management methods and conventional wisdom are a hard barrier to 2.0 strategy and transformation.  You don’t have to get far into discussions about the Perpetual Beta or Product Development 2.0 before existing management methods seem outdated, inflexible, and ineffective.  This is one of the more difficult aspects of adopting 2.0 models and the implications is that we’ll have to do a lot of rethinking how we manage businesses driven by 2.0 models, where the boundaries of organizations are less clear, the ownership is much more community-based, and the outcomes are far more diverse and spread out, making them less trackable, controllable, and directed.  Overhauling management practices and techniques will be a core activity in a 2.0 transformation and will be hard to achieve quickly enough due to the Innovator’s Dilemma.
  5. Avoiding external disruption is hard but managing self-imposed risk caused by 2.0 is easier.  The great fear than many businesses have is facing a fast-growth competitor that takes these ideas and either wrests away market share rapidly and aggressively or cuts them off at the pass with entirely new products.  YouTube did this to the broadcast and cable industry, which responded with Hulu.  Apple did this with iTunes to the recording industry and the blogosphere did the same to the newspaper industry.  Other industries are next likely including the financial services industry, real estate, and others.  Internally, however, risk management is still a challenge but is much more manageable.  The big implication for this is that starting internally first with things like Enterprise 2.0 initiatives and prediction markets to learn the ropes on how to deal with unexpected outcomes and results can help organizations climb the maturity curve.
  6. Incubators and pilots projects can help create initial environments for success with 2.0 efforts.  Too much contact with the traditional support environment of an existing, primarily 1.0 organization makes it hard for 2.0 efforts to succeed; everything gets done in the traditional way instead of the new ways that are required.  The traditional tools, processes, and skills just aren’t there or are just too slow and burdened with unnecessary overhead.  Creating dedicated incubators that are designed to use the strengths of the organization while being isolated from its weaknesses can help.  Incubators are at risk of becoming too isolated however, and won’t inform or change the greater organization unless care is taken to roll the lessons and capability back in.
  7. Irreversible decisions around 2.0 around topics such as brand, reputation, and corporate strategy can be delayed quite a while, and sometime forever. Most organizations get paralysis around change and transformation because of concerns around decisions that can’t be reversed.  Concern over damaging the company’s brand is one of the top issues I run into and it’s a valid concern.  The good news is that many organizations are discovering they can safely leverage the advantages of their organization (such as their extensive customer base to drive initial growth of 2.0 engagement and adoption of new products and services) without dragging their brand into it whatsoever.  New 2.0 products from major companies are now often released under new brands entirely. This enables serious experimentation with 2.0 while taking little risk to the organization.
  8. The technology competence organizations have today are inadequate for moving to 2.0.  This is key if you’re a CTO or CIO today; your organization is almost certainly not ready to handle the development, management, scalability, identity, governance, and openness issues around 2.0.  If you’re not sure, just ask your IT staff.  Examples include cloud computing, open APIs, mashups, rich user experiences, Web-Oriented-Architecture, community platforms, Enterprise 2.0, 2.0-era computing stacks like Rails and Django, are all disciplines that are considerable in their own right, of rapidly growing importance to organizations in the 2.0 era.  These are all likely to be things your staff needs to come up the learning curve on in significant ways and with the rate of change on the network what it is presently, falling behind is too easy to do.  Note: The existing technology landscape of most organizations will have to change as well which is where Web-Oriented Architecture (WOA) is getting quite a bit of attention today.  And the Web products themselves have moved far beyond the model of the Web page and most enterprises are very far behind.
  9. The business side requires 2.0 competence as well.  This includes how to design, build, launch, market, support, and maintain 2.0 products and services as well as the ways that workers should use the tools and concepts to work together.  I recently suggested that learning how to be effective in working within and directing communities of workers/users/partners to accomplish large-scale outcomes will be a vital skill in the very near future.  All of this requires both a new perspective as well as a hard-headed effort at skill building and a re-orientation of existing work habits and processes.
  10. Start small, think big.  We have discovered that the leverage the network can give us is almost unlimited.  It’s ability to scale ideas, products, and communities of users as fast as they are able to is one of the aspects that makes it so attractive to business.  2.0 products tends to be very simple at heart, and though there is certainly challenges and complications growing, small ideas can become big very, very quickly.  Getting to the right solutions, not-overinvesting (which leads to complication and heavyweight management and processes) and letting customers and partners take the seeds of great ideas and run with them is what makes sudden success turn into a large-scale success.  On the Web, starting small, and thinking big can take you a long, long way.  Read more about network effects driven by architectures of participation .

Please share your ideas around what else is essential in a Web 2.0 strategy below.

Building Modern Web Apps? Better Have A Deep Competency in Web 2.0, Open APIs, Widgets, Social Apps, and More

The Web has an interesting property that those building Web applications and online businesses usually encounter soon after they first launch: It has its own unique and unforgiving rules for success and failure.  Appreciating them requires a certain level of understanding of the intrinsic nature of the Web and how it works.  Actually leveraging those rules requires an even deeper and more profound understanding of the Web. The challenge these days? The Web competency bar is climbing fast.

To drive the right decisions in what they do product designers, marketing teams, software architects, developers, strategy officers, and other key roles in today's generation of online businesses need to have a solid handle on an extensive array of Web topics.  This ranges from appreciating why plain old HTTP is so good at underpinning the Web to more sophisticated topics like modern application architecture, the latest in online user experiences, next generation computing models (grid/cloud/utility/SaaS/PaaS), cost-effective scalability, user identity, network effects, Jakob's Law, analytics, operations, user community, as well as the many compelling new distribution models that are nearly mandatory in the first release of most products. 

This extensive set of competencies is what's required nowadays to deliver a credible online product to a receptive user base and it has dramatic implications for both uptake and overall cost/time-to-market.  Worse, this body of knowledge has become extensive enough that many Web startups frequently fall far short of what they need to know in order to be successful with these far flung practice areas. 

Web Product Distribution Models - Web 2.0, Widgets, Social Apps, Open APIs

Does this complex body of knowledge mean the era of the two-to-five person Web startup is coming to a close? Not at all, at least not yet. The productivity level of the latest tools and techniques remains almost astonishing though the level of knowledge required of these teams is creeping up and up.  And as we'll see, new models for product distribution are pushing the capability envelope of the typical Internet startup team to the point we may very well see the day soon that they won't have all the skills necessary to deliver a fully-scoped modern Web application.  It is also one reason why fewer and fewer Web startups have the goods to be all around hits out of the gate.

Certainly, varying depths in subject matter are required depending one's exact role in a Web business, but Web-oriented products are fundamentally shaped the vagaries of the network itself.  Tim O'Reilly himself still has the best quote on the subject: "Winners and losers will be designated by who figures out how to use the network." And as we'll see, the Web is driving the evolution of a major new generation of online distribution models.

Why Adopting New Distribution Models Is Crucial 

As an example of this, I've been tracking some of the latest discussions around the hot topic du jour in the Web world: Social networking applications.  Specifically, it's been interesting to watch the surprisingly low level of industry attention around the titanic competition brewing between social networking application formats from Web giants Facebook and Google.  Why is this?  Some might say it's because these applications still have largely unproven business models.  Others, like Nick O'Neill at the Social Times recently observed (rightly in my opinion) that the struggle may have to do with a deficit in understanding why these new types of Web applications are so important. Nick notes that these widget and social networking style models for packaging and distributing Web apps often "have more eyeballs looking at their products than television channels have" and the challenge is that too many people just "don’t know what any of this means", despite the major players divvying up the online pie for themselves.  With the size of these next generation distribution audiences, ignorance has an extremely painful price: failure to produce results and growth, poor engagement with the marketplace, and loss of market share.

An excellent summary of the truly massive, but largely underappreciated scale of these new Web application models was last week's TechCrunch piece on the progress of Google's OpenSocial, an increasingly successful model for creating portable social networking applications that will run on any OpenSocial-compliant site.  Erick Schonfeld reported that OpenSocial now has a total reach of an astonishing 350 million users and it will soon be 500 million.  There are over 4,500 OpenSocial apps today, a healthy number for the application format but a small drop in the bucket compared to the number of Web sites in the world. But the key is that these applications are integrated much deeper into the social fabric of an engaged audience, interjecting themselves into the daily personal and work habits of the "captive" users of social sites and even have access to the personal habits and data of users of these sites.  Facebook's story is impressive as well with over 37,000 applications that have been installed over 700 million times.

And social networking applications are just one of many news ways that applications have to be packaged and distributed, yet far too many organizations persist in a very 1990s view of Web experiences, namely that Web sites themselves are the center of online product design.  Many even think that some of these other new distribution models are interesting but not part of their core online product.  Unfortunately, that's very much a parochial view in the present era.  Federated applications, atomized content and functionality, 3rd party product ecosystems through open APIs, and much more are required to establish a strong and resilient network effect which fends off competitors that are themselves bringing these potent new competencies to bear. 

 In fact, one of the things we emphasize over and over again in our conference workshops and in Web 2.0 University is that having a Web site is usually the least interesting things about new products.  Worse, it makes the customer have to find you amongst tens of millions of other sites.  Instead, these new models tend to focus on going to the customer, instead of making them come to you which is a much harder proposition. This can instantly give you the ability to reach millions of potential people with dramatically lower effort and cost, as long as you have something interesting to offer.

Unfortunately, the number of capable practitioners of these new distribution models remains relatively small compared to the large body of experts in traditional Web product development.  Demand is also low for these new skills as most organizations have been painfully slow to appreciate how much online product development has changed.  A quick search of the job aggregator SimplyHired tells the tale: Nearly a thousand Web designer positions are available while only 36 OpenSocial and 40 open API positions are open, for example.  This despite the the latter skills being able to project a product across the Web into hundreds of social sites or create an API that allows the product to be incorporated into countless other products for far less cost per customer than traditional methods.

The lesson here is that these new models still have a lot of fertile, unclaimed territory and many otherwise fierce competitors have not yet become fully aware of these new opportunities.  Get your piece of the pie while there's still time

The new Web 2.0 era distribution models remain largely untapped

I also find that the Web development industry has been slow to change, particularly outside the valley, and there is depressingly scarce information on how to deliver well on things like widgets, open APIs, social networking applications, and even syndication.  To help with this, I've put together a short primer and some good references for those that want to get started.

Because the good news is that there remains tremendous opportunity for growth and success — for both startups and traditional businesses — if they will actively begin incorporating these new product delivery models into their own online capabilities.

Overview of Online Product Delivery Models 

  1. Web sites.  This the classic model for Web presence.  During the early Web, creating a Web site was just about the only option for engaging with those online (e-mail being the other.)  Most early Web sites were used for publishing and not for user participation or peer production.  These days, Web sites are still important, though by no means mandatory, and have their content syndicated via RSS and ATOM (pushing the content to where it's wanted), provide an access point to obtain widgets, and maintain user identity, and create communities of users.  Upshot: They've evolved a lot but Web sites are only part of an extensive set of capabilities that must be brought to bear in the Web 2.0 era.
  2. Syndication. It took ten years for the Web community to figure out a workable syndication model.  Now RSS and ATOM are now the expected models used to distribute content off a single site and across the Web. Countless aggregation services now exist that make a site's information embedded in their services as well as a way to offer users a method for pulling information from a site and experienced in a means of their choosing, from Google Reader and Newsgator to the innovative Yahoo! Pipes.  Most sites still heavily underutilize syndication even for notifications and pushing out frequently changed information to draw attention to it much less the strategic ecosystem and integration opportunities it affords.
  3. Web 2.0 applications.  You might argue that Web 2.0 itself is not a product distribution model but a set of design patterns and business models and that would be a true statement. However, in this context we're referring to the fact that Web 2.0 apps package up the 3rd major type of networked value: user participation.  Before then, Web sites and syndication primarily had only centrally produced content or functionality that they could expose over the network and offer to the marketplace.  In other words, user participation its purest form — sometimes known as peer production –  ultimately results in products like Mechanical Turk and Predictify that provide direct networked access to user participation, but there are many fine gradations to this.  The bottom line, Web 2.0 applications plug the user into the network like never before and are a critical rung in the distribution ladder since it offers access to the largest set of content and information by harnessing collective intelligence.
  4. Open APIs and Web services.  This is one of the most important long-term decisions most online businesses can make.  Offering an open API lets anyone take the online components of a business, from its data and functional capabilities to the users themselves, and makes them open and accessible over the Web to be incorporated into other products and services, sometimes in the form of mashups and sometimes in the form of entire online products.  Amazon, one of the first Web companies in existence and is hence far downrange in terms of the experience curve, has been using this distribution model with notable success recently.  So have hundreds of others.  The real challenge has been how foreign this model is to the original Web model and thus to the various management and development competencies in most organizations.  It's much more an a way to OEM a product and leverage the customers and investments of hundreds of other partners.  However, overall, it affords the potential for much larger business outcomes than could ever be created with point Web presence.  It's now considered a significant oversight not to have an open API available for the typical online product.
  5. Web widgets.  Selecting parts of a Web site and it's data and packaging it up to make it run inside a portable, user distributable widget has been growing more and more popular over the last few years. For example, WidgetBox currently distributes 74,000 different kinds of Web widgets from its partners to over 1.2 million other sites.  Widgets lets users distribute a Web site to other places on the Web at no extra cost and it also creates an ecosystem effect, where other Web sites users become the users of the new site.  The YouTube badge is a notoriously well-known example of this that also helped drive the extraordinarily fast growth of the site.  Like APIs, widgets are now considered a mandatory must-have for new and existing online products. But unlike APIs where it's up to the API users, figuring out users want out of your site's widgets is still an art form.
  6. The Plaxo Pulse Story with OpenSocialSocial networking applications.  Sometimes viewed as an extension of the Web widget model, social networking applications are applications designed to run inside of popular social networking environments and usually have capabilities that tap into and make use of the social graph information resident in a user's social network account.  This is an amazingly fast moving field as you can see from a recent post on the latest happenings on the OpenSocial blog, to the extent it's hard even for well-funded companies to keep up.  However, despite skepticism that large businesses can be built exclusively through a social networking application, it's become ever more essential for a site to make its capabilities accessible usable in these environments.  Not only will users help distribute online products in these formats to their contacts but it also increases the overall usage of the your application including participation and its consequently growth of a site's network effect.  While not yet considered mandatory for online products, the ease with which these social network applications can be created and the large numbers of users they make available makes it a smart distribution option for most Web businesses.  Like widgets, however, figuring out what users will find engaging in a social networking application featuring your online product takes some research and experimentation.  However, the results can be very rewarding and some social networking applications have millions of daily users.  See the Plaxo Pulse story on Mashable for the details of how OpenSocial drove a 5x improvement in traffic in only 3 weeks.
  7. Semantic Web and Web 3.0. The Semantic Web, one of the original visions for the World Wide Web, has taken a while to arrive but it's beginning to look like it may hit critical mass in the next 12-24 months.  Combined with Web 3.0, which takes the architectures of participation at the core of Web 2.0 and drives it through a lens of Semantic Web capabilities.  The benefits can be profound and can greatly increase the value and leverage of information on the Web.  While this is very much not prime time yet, unlike #1-#6 above, it likely will be and smart organizations can get ahead of the learning curve and get an early market lead using these techniques.  For now, however, I recommend that most organizations focus on executing well on the first six items before tackling this and waiting for the technologies to finish emerging and maturing.

The list above should provide good guidance for starting move into the potent new models for distribution on the Web.  I'm seeing, however, that because of the major shifts in strategy and product design emphasis these techniques demand, most organizations take an inordinately long amount of time to become effective with them.  The lesson here: Start small now and build core competency.  Small investments now can pay off later in terms of valuable experience made from early experiments and pilots.  When done right,
these new distribution models can become the dominant channels that the world uses to interact with your business, like they already have with Amazon and Twitter.

I'll be talking about these and other strategic online product design topics in my upcoming Building Next Generation Web Apps Workshop at the inaugural Web 2.0 Expo 2008 NYC next month.  I'll have more details about this deep-dive session in an upcoming post.

Follow

Get every new post delivered to your Inbox.

Join 66 other followers