The Social Graph: Issues and Strategies in 2008

January 16, 2009 4 Comments

One of the hottest topics in the online world in the last couple of years has been the growth of social networking services such as Facebook and MySpace, as well as the addition of a social element to existing user experiences.  Despite riding several waves of hype, it's now clear that the social networking space will only get hotter in 2008 according to most watchers.  Social software has come fully into its own as of 2008 — for all appearances permanently — and understanding the reasons for this rapid rise as well as figuring out how to leverage it best is the job of everyone who wants to make the most of the Web 2.0 era.

Gaining a deeper insight to the social networking phenomenon, now exhibited by the tens of millions of users employing them globally on a daily basis for both personal and businesses uses, currently means understanding the fundamental unit of the social network, also one of the biggest new buzzphrases of the year: the social graph.  Fortunately, that's simple enough despite the term's oblique reference to graph theory, which it is heavily based upon.

Social Graphs - The pattern of social relationships between people

Simply put, a social graph is a set of people, referred to as nodes, that are connected together by vertices — better known as links or connections — that reflect their social relationships.  You can see a conceptual social graph above, showing the typical distinction of social networks to reflect whether a connection with another person is direct or indirect.  For example, the popular business social networking service LinkedIn, uses this model and sorts a member's social graph into different degrees of separation, which you can see a typical example of below and taken from my LinkedIn profile:

 

Organizing Social Graphs - Degress of separation is popular

Also becoming popular is the burgeoning field of social analytics, such as the Socalistics application in Facebook and the Interactive Friends Graph, though there are also commercial standalone products here or on the way for the enterprise and open Web spaces from companies like KnowNow and Bravadosoft.  The Interactive Friends Graph is a nice, simple example anyone can try on their own and you can see mine from Facebook below.  Hovering over nodes in the live version in your Facebook profile allows you to see who is connected to others in your network and begin to gain insight and understanding of the relationships in your network.

 Social Graph Example - One of many way to depict a social graph

But what are the top issues one must understand about the social graph in 2008?  As I've seen social networks become common on corporate intranets and in daily use on the Web, some of the issues are rapidly becoming clear.  However, the full story will certainly continue to unfold for the next several years at least.  Here's what we're seeing at the moment:

Strategies and Issues for the Social Graph – Circa 2008

  • The social graph is poised to replace the address book and contact list as the preferred organizing structure for personal and business relationships. This was one of my Web 2.0 predictions for 2008 and it won't fully come true for the majority of users for at least several years since there's such an installed base of traditional tools for managing relationship information.  What's the difference?  Social networks are usually opt-in, two-ways for one.  And they are social for another, meaning they tend to encourage communication and collaboration, such as through user profile event streams and status messages.  They also offer up and actively make use of the deeper insight into the full graph's social surface area beyond direct contacts, such as LinkedIn's introduction service.
  • Ownership of the social graph is going to be a ground zero issue in 2008.  Robert Scoble's widely covered attempt recently to use Plaxo Pulse to export his 5,000 Facebook contacts recently got him banned temporarily from the service.  But as users begin to realize that the contact lists they are building using online Web tools might not be portable, this will become a growing concern, particularly since two-way opt-in makes a social graph more valuable (and accurate) but significantly harder to recreate on demand elsewhere. This takes us to our next subject…
  • Many social networking services will adopt open data initiatives.  Both Google and Facebook recently showed support for DataPortability.org and Google has an interesting play in their OpenSocial initiative.  This is welcome news that will resolve some of the concerns around who owns the graph but interestingly, traditional corporations will be the slowest get this and will rarely let workers take their hard won social graphs and user profiles with them elsewhere as they move to new jobs.  Public social networking sites Web sites are leading the way here and this will only drive more business users to the open Web, where they at least have some control over their social graph.  Smart organizations will provide their workers with some form of open social graph support, lest they lose control completely as workers keep more and more of their graph in Facebook, LinkedIn, and Plaxo and not in prescribed relationship management tools.
  • Attempts to monetize social graphs will drive interest in regulation and legislation.  Social networking is now a global Internet phenomenon and that the information contained within them is highly central to everyone's lives.  This will make everything from protecting children to individual privacy of social graphs a hot issue for some local and federal governments.  All it will take is one or two widely covered exploits to make this happen.  Expect the European Union and the U.S. government to begin seriously examining the issue this year with many other governments following suite.  Good citizenship of sites that manage social graphs will be essential to prevent excessive government involvement.
  • The line is blurring between personal and business use of social graphs.  We're all rapidly getting one large social graph each already, with everyone we know in them.  Most public social networking sites do a poor job of separating different subgroups of our social networks, such as allowing pictures and status messages to only go to a specific subgroups (work messages to business, family message to family, friends messages to friend, etc.)  This actually works a little bit better in enterprise social networks, but not much, since it largely consists of a Contact Type field.  Segmentation of social graphs will be an increasingly requested feature by users struggling with their use.  The social graph management services that make this distinction and enable its leverage may do very well indeed.
  • Open Web identity, which will ultimately form the global "primary key" for social graph nodes, will not get anywhere soon.  This despite it being needed badly but the users of the Web have not yet felt compelled to demand it.  Data portability of social graphs will begin to drive adoption of user controlled Web identity, and hopefully government regulation will not.  See Dare Obasanjo's deep exploration of using openid to enable social graph interoperability as an example of what will need to happen, despite there being little incentive currently for sites to use other site's openids.
  • Making social networking "gardening" and administration easier will drive new innovations.  Most individual social graphs are primarily tended by hand today, although a growing number of products, such as Visible Path, do all the tedious work for you by watching your social interaction online such as through tight integration through e-mail and instant messaging, building a rich graph for you (even sending invitations) as you go about your daily social activities.  New innovations like these will make social graphs easier to maintain and richer in overall information while also driving adoption through ease of use.
  • The optional two-way confirmation of a social graph link becoming standard.  Many social graph management platforms (Facebook and Linked for example) require confirmation from the other side of the connection before adding a person to your graph.  Sites like Spock, which make it optional, will ultimately be more practical for managing a social graph while still allowing discernment of two way confirmations, which tend to be more valuable and convey key information about the trust and real extent of a social relationship.
  • Social networking fatigue will not set in as perceived constraints such as Dunbar's limit do not prove to be universal.  While there are many theories on how big a social graph can get before it become unmanageable and sees diminishing returns on growth (note that both Facebook and LinkedIn encourage ceilings), the fact is that the are many different purposes for a social graph, from data mining and historical research, to marketing and customer relationship management.  

What else is going to be key to dealing with the social graph in 2008?  Please leave in comments below and I'll update this post with any good submissions.

Filed under Social Business, Social Graph, Social Media, Social Technology & Standards, Strategy Tagged with , , , ,

Ten Aspects of Web 2.0 Strategy That Every CTO and CIO Should Know

September 14, 2008 Leave a comment

Over the last year I’ve worked with organizations around the world that are attempting to grapple with Web 2.0 and the growing external marketplace pressure being exerted for the change and transformation of their businesses. Along the way, I’ve been fortunate enough to be able to identify and assemble a working list of some consistent recurring issues and themes around Web 2.0 strategy.  I’ve provided them below at a high level. Your comments and additions are very welcome as we try to frame up a consistent picture of what’s happening in the marketplace.

It used to be a little surprising how long it’s taken for Web 2.0 to begin to have serious impact on or even high-level interest in the business world.  However, the ideas have had staying power and have also largely been validated; there are now fundamentally different and very powerful new models for engaging with customers, designing our products, and applying technology in general to our business that are proven and have growing bodies of knowledge.  The Web has become the single most important driving force in many fields of endeavor as well as the leading source of both innovation and potent new modes for communicating, collaborating, socializing, and working together. It’s taken a few years but businesses are now feeling the change in the air.

 

The Web 2.0 Transformation and Change Management Process for Business and Enterprises for CTOs and CIOs

 

However, as I’ve said a number of times in my various discussions of Web 2.0, the power of the network has deep roots in some profound shifts in society and culture, particularly the singular move from push-based systems (the 1.0 era going way, way back until right around now) to pull-based systems (the 2.0 era from roughly a few years into this century and going forward).  That this shift is well under way is clear if you look at the sudden explosion of the blogosphere, social networking, social media, open source software, online communities, and peer production in virtually all things.  The good news (or bad news, depending on how you look at it) is that despite the remaking of more than a few industries already — including media, software, advertising — this shift is only just beginning.

This all raises the question of how to make the transition from 1.0 to 2.0 safely and non-disruptively with your business largely intact, perhaps even with a superior competitive position.  That this transition can actually be accomplished by most businesses is still far from clear though some early transitions have met with varying degrees of success.  This list represents some of what we’ve learned so far  about 2.0 transformation but it’s something that strikes at the very heart of most businesses today: The rules for success are not-so-gradually changing and the marketplace is driving it in an often-subversive grassroots, bottom-up way.  The question now is no longer about “if” but increasingly about thriving long-term, period: What are you willing to do to adapt to a new business world?

This list is aimed primarily at CTOs and CIOs since they are mostly likely to be located at the convergence of traditional business thinking and the wave of 2.0 change coming in off the network. However these ideas apply to anyone looking at how to embrace 2.0 transformation in their organization and take advantage of it.  This is one of the most exciting eras to be in businesses since so many directions are in flux and the outcomes, players, and market leaders of the near future are far from certain.  Those who can see the new opportunities clearly through the lens of 2.0 transformation not only have a fighting chance, but are able to seize them with once-in-a-generation ease.

Note: I’ve dropped the “Web” in Web 2.0 for this discussion because one of the big lessons is that many traditional business thinkers turn off when they hear the word, even though Web 2.0 design patterns and business models have truly profound implications across any business today.  Consequently, hat the Web is driving most of these changes is being considered incidental for this discussion (though it’s absolutely the opposite when actually executing on these new models.) Instead, this is targeted a discussion about the transformative models themselves (such as who creates the products and where, how they are used, who supports them, how are they remixed, syndicated, franchised, licensed, IP protected, etc) in a strategic businesses sense. At the core of this discussion is how 1.0 business models of the 20th century are very much being eroded, transformed, and frequently dethroned by the immense motive forces that lie in the pervasive, open networked systems we have today, which are taking us deeply into a very new place: the 2.0 era.

Ten Key Aspects of Web 2.0 Strategy

  1. It’s not about technology, it’s about the changes it enables.  While technology is a close second (and ultimately makes 2.0 business models possible), the real discussion is about the disruptive new opportunities it creates.  Instead the discussion should be focused more around strategies such as harnessing millions of customers over the network to co-create products through peer production, engaging in mass customer self-service, customer communities, and open supply chains to thousands of ad hoc partners with open APIs. These are just some of the examples of using the network to create far richer and more profound results than could be created in the 1.0 era.  Don’t get caught up in the technology of 2.0 at first other than to understand the business possibilities it affords.  Avoid technology-first discussions like the plague.  Premature monetization discussions around 2.0 are also to be avoided, they tend to have a negative impact on process if done too early.
  2. The implications of 2.0 stands many traditional views on their head and so change takes more time than usual.  In the 2.0 world customers and partners have a much closer, more sustained relationship because of social interaction and tightly integrated online supply chains, to name just two reasons.  The shift of control from institutions to communities of users takes a lot of getting used to.  Just understanding how and why intellectual property is better covered by Creative Commons instead of copyright will take the legal department years (if not decades).  Each part of the organization will have its miniature 2.0 revolution.  These take time to happen and sort themselves out.  This means getting these new ideas into people’s heads is one of the first steps…
  3. Get the ideas, concepts, and vocabulary out into the organization and circulating.  If you’re trying to affect 2.0 change in an organization, there’s no better solution that exposing people to it.  Demographics can be a problem in this situation depending on the industry.  Younger workers tend to live and breath 2.0 while older workers may be aware of it but don’t think it applies to them.  I use point education where change needs to happen either first or quickly and then internal communities that bring the discussion of change, innovation, and transformation to the entire organization.  Either way, learning and education around 2.0 are a vital trigger to begin change and should be started early and non-disruptively.
  4. Existing management methods and conventional wisdom are a hard barrier to 2.0 strategy and transformation.  You don’t have to get far into discussions about the Perpetual Beta or Product Development 2.0 before existing management methods seem outdated, inflexible, and ineffective.  This is one of the more difficult aspects of adopting 2.0 models and the implications is that we’ll have to do a lot of rethinking how we manage businesses driven by 2.0 models, where the boundaries of organizations are less clear, the ownership is much more community-based, and the outcomes are far more diverse and spread out, making them less trackable, controllable, and directed.  Overhauling management practices and techniques will be a core activity in a 2.0 transformation and will be hard to achieve quickly enough due to the Innovator’s Dilemma.
  5. Avoiding external disruption is hard but managing self-imposed risk caused by 2.0 is easier.  The great fear than many businesses have is facing a fast-growth competitor that takes these ideas and either wrests away market share rapidly and aggressively or cuts them off at the pass with entirely new products.  YouTube did this to the broadcast and cable industry, which responded with Hulu.  Apple did this with iTunes to the recording industry and the blogosphere did the same to the newspaper industry.  Other industries are next likely including the financial services industry, real estate, and others.  Internally, however, risk management is still a challenge but is much more manageable.  The big implication for this is that starting internally first with things like Enterprise 2.0 initiatives and prediction markets to learn the ropes on how to deal with unexpected outcomes and results can help organizations climb the maturity curve.
  6. Incubators and pilots projects can help create initial environments for success with 2.0 efforts.  Too much contact with the traditional support environment of an existing, primarily 1.0 organization makes it hard for 2.0 efforts to succeed; everything gets done in the traditional way instead of the new ways that are required.  The traditional tools, processes, and skills just aren’t there or are just too slow and burdened with unnecessary overhead.  Creating dedicated incubators that are designed to use the strengths of the organization while being isolated from its weaknesses can help.  Incubators are at risk of becoming too isolated however, and won’t inform or change the greater organization unless care is taken to roll the lessons and capability back in.
  7. Irreversible decisions around 2.0 around topics such as brand, reputation, and corporate strategy can be delayed quite a while, and sometime forever. Most organizations get paralysis around change and transformation because of concerns around decisions that can’t be reversed.  Concern over damaging the company’s brand is one of the top issues I run into and it’s a valid concern.  The good news is that many organizations are discovering they can safely leverage the advantages of their organization (such as their extensive customer base to drive initial growth of 2.0 engagement and adoption of new products and services) without dragging their brand into it whatsoever.  New 2.0 products from major companies are now often released under new brands entirely. This enables serious experimentation with 2.0 while taking little risk to the organization.
  8. The technology competence organizations have today are inadequate for moving to 2.0.  This is key if you’re a CTO or CIO today; your organization is almost certainly not ready to handle the development, management, scalability, identity, governance, and openness issues around 2.0.  If you’re not sure, just ask your IT staff.  Examples include cloud computing, open APIs, mashups, rich user experiences, Web-Oriented-Architecture, community platforms, Enterprise 2.0, 2.0-era computing stacks like Rails and Django, are all disciplines that are considerable in their own right, of rapidly growing importance to organizations in the 2.0 era.  These are all likely to be things your staff needs to come up the learning curve on in significant ways and with the rate of change on the network what it is presently, falling behind is too easy to do.  Note: The existing technology landscape of most organizations will have to change as well which is where Web-Oriented Architecture (WOA) is getting quite a bit of attention today.  And the Web products themselves have moved far beyond the model of the Web page and most enterprises are very far behind.
  9. The business side requires 2.0 competence as well.  This includes how to design, build, launch, market, support, and maintain 2.0 products and services as well as the ways that workers should use the tools and concepts to work together.  I recently suggested that learning how to be effective in working within and directing communities of workers/users/partners to accomplish large-scale outcomes will be a vital skill in the very near future.  All of this requires both a new perspective as well as a hard-headed effort at skill building and a re-orientation of existing work habits and processes.
  10. Start small, think big.  We have discovered that the leverage the network can give us is almost unlimited.  It’s ability to scale ideas, products, and communities of users as fast as they are able to is one of the aspects that makes it so attractive to business.  2.0 products tends to be very simple at heart, and though there is certainly challenges and complications growing, small ideas can become big very, very quickly.  Getting to the right solutions, not-overinvesting (which leads to complication and heavyweight management and processes) and letting customers and partners take the seeds of great ideas and run with them is what makes sudden success turn into a large-scale success.  On the Web, starting small, and thinking big can take you a long, long way.  Read more about network effects driven by architectures of participation .

Please share your ideas around what else is essential in a Web 2.0 strategy below.

Filed under Social Business, Social Media, Strategy, The Enterprise

Building Modern Web Apps? Better Have A Deep Competency in Web 2.0, Open APIs, Widgets, Social Apps, and More

August 24, 2008 Leave a comment

The Web has an interesting property that those building Web applications and online businesses usually encounter soon after they first launch: It has its own unique and unforgiving rules for success and failure.  Appreciating them requires a certain level of understanding of the intrinsic nature of the Web and how it works.  Actually leveraging those rules requires an even deeper and more profound understanding of the Web. The challenge these days? The Web competency bar is climbing fast.

To drive the right decisions in what they do product designers, marketing teams, software architects, developers, strategy officers, and other key roles in today's generation of online businesses need to have a solid handle on an extensive array of Web topics.  This ranges from appreciating why plain old HTTP is so good at underpinning the Web to more sophisticated topics like modern application architecture, the latest in online user experiences, next generation computing models (grid/cloud/utility/SaaS/PaaS), cost-effective scalability, user identity, network effects, Jakob's Law, analytics, operations, user community, as well as the many compelling new distribution models that are nearly mandatory in the first release of most products. 

This extensive set of competencies is what's required nowadays to deliver a credible online product to a receptive user base and it has dramatic implications for both uptake and overall cost/time-to-market.  Worse, this body of knowledge has become extensive enough that many Web startups frequently fall far short of what they need to know in order to be successful with these far flung practice areas. 

Web Product Distribution Models - Web 2.0, Widgets, Social Apps, Open APIs

Does this complex body of knowledge mean the era of the two-to-five person Web startup is coming to a close? Not at all, at least not yet. The productivity level of the latest tools and techniques remains almost astonishing though the level of knowledge required of these teams is creeping up and up.  And as we'll see, new models for product distribution are pushing the capability envelope of the typical Internet startup team to the point we may very well see the day soon that they won't have all the skills necessary to deliver a fully-scoped modern Web application.  It is also one reason why fewer and fewer Web startups have the goods to be all around hits out of the gate.

Certainly, varying depths in subject matter are required depending one's exact role in a Web business, but Web-oriented products are fundamentally shaped the vagaries of the network itself.  Tim O'Reilly himself still has the best quote on the subject: "Winners and losers will be designated by who figures out how to use the network." And as we'll see, the Web is driving the evolution of a major new generation of online distribution models.

Why Adopting New Distribution Models Is Crucial 

As an example of this, I've been tracking some of the latest discussions around the hot topic du jour in the Web world: Social networking applications.  Specifically, it's been interesting to watch the surprisingly low level of industry attention around the titanic competition brewing between social networking application formats from Web giants Facebook and Google.  Why is this?  Some might say it's because these applications still have largely unproven business models.  Others, like Nick O'Neill at the Social Times recently observed (rightly in my opinion) that the struggle may have to do with a deficit in understanding why these new types of Web applications are so important. Nick notes that these widget and social networking style models for packaging and distributing Web apps often "have more eyeballs looking at their products than television channels have" and the challenge is that too many people just "don’t know what any of this means", despite the major players divvying up the online pie for themselves.  With the size of these next generation distribution audiences, ignorance has an extremely painful price: failure to produce results and growth, poor engagement with the marketplace, and loss of market share.

An excellent summary of the truly massive, but largely underappreciated scale of these new Web application models was last week's TechCrunch piece on the progress of Google's OpenSocial, an increasingly successful model for creating portable social networking applications that will run on any OpenSocial-compliant site.  Erick Schonfeld reported that OpenSocial now has a total reach of an astonishing 350 million users and it will soon be 500 million.  There are over 4,500 OpenSocial apps today, a healthy number for the application format but a small drop in the bucket compared to the number of Web sites in the world. But the key is that these applications are integrated much deeper into the social fabric of an engaged audience, interjecting themselves into the daily personal and work habits of the "captive" users of social sites and even have access to the personal habits and data of users of these sites.  Facebook's story is impressive as well with over 37,000 applications that have been installed over 700 million times.

And social networking applications are just one of many news ways that applications have to be packaged and distributed, yet far too many organizations persist in a very 1990s view of Web experiences, namely that Web sites themselves are the center of online product design.  Many even think that some of these other new distribution models are interesting but not part of their core online product.  Unfortunately, that's very much a parochial view in the present era.  Federated applications, atomized content and functionality, 3rd party product ecosystems through open APIs, and much more are required to establish a strong and resilient network effect which fends off competitors that are themselves bringing these potent new competencies to bear. 

 In fact, one of the things we emphasize over and over again in our conference workshops and in Web 2.0 University is that having a Web site is usually the least interesting things about new products.  Worse, it makes the customer have to find you amongst tens of millions of other sites.  Instead, these new models tend to focus on going to the customer, instead of making them come to you which is a much harder proposition. This can instantly give you the ability to reach millions of potential people with dramatically lower effort and cost, as long as you have something interesting to offer.

Unfortunately, the number of capable practitioners of these new distribution models remains relatively small compared to the large body of experts in traditional Web product development.  Demand is also low for these new skills as most organizations have been painfully slow to appreciate how much online product development has changed.  A quick search of the job aggregator SimplyHired tells the tale: Nearly a thousand Web designer positions are available while only 36 OpenSocial and 40 open API positions are open, for example.  This despite the the latter skills being able to project a product across the Web into hundreds of social sites or create an API that allows the product to be incorporated into countless other products for far less cost per customer than traditional methods.

The lesson here is that these new models still have a lot of fertile, unclaimed territory and many otherwise fierce competitors have not yet become fully aware of these new opportunities.  Get your piece of the pie while there's still time. 

The new Web 2.0 era distribution models remain largely untapped

I also find that the Web development industry has been slow to change, particularly outside the valley, and there is depressingly scarce information on how to deliver well on things like widgets, open APIs, social networking applications, and even syndication.  To help with this, I've put together a short primer and some good references for those that want to get started.

Because the good news is that there remains tremendous opportunity for growth and success — for both startups and traditional businesses — if they will actively begin incorporating these new product delivery models into their own online capabilities.

Overview of Online Product Delivery Models 

  1. Web sites.  This the classic model for Web presence.  During the early Web, creating a Web site was just about the only option for engaging with those online (e-mail being the other.)  Most early Web sites were used for publishing and not for user participation or peer production.  These days, Web sites are still important, though by no means mandatory, and have their content syndicated via RSS and ATOM (pushing the content to where it's wanted), provide an access point to obtain widgets, and maintain user identity, and create communities of users.  Upshot: They've evolved a lot but Web sites are only part of an extensive set of capabilities that must be brought to bear in the Web 2.0 era.
  2. Syndication. It took ten years for the Web community to figure out a workable syndication model.  Now RSS and ATOM are now the expected models used to distribute content off a single site and across the Web. Countless aggregation services now exist that make a site's information embedded in their services as well as a way to offer users a method for pulling information from a site and experienced in a means of their choosing, from Google Reader and Newsgator to the innovative Yahoo! Pipes.  Most sites still heavily underutilize syndication even for notifications and pushing out frequently changed information to draw attention to it much less the strategic ecosystem and integration opportunities it affords.
  3. Web 2.0 applications.  You might argue that Web 2.0 itself is not a product distribution model but a set of design patterns and business models and that would be a true statement. However, in this context we're referring to the fact that Web 2.0 apps package up the 3rd major type of networked value: user participation.  Before then, Web sites and syndication primarily had only centrally produced content or functionality that they could expose over the network and offer to the marketplace.  In other words, user participation its purest form — sometimes known as peer production —  ultimately results in products like Mechanical Turk and Predictify that provide direct networked access to user participation, but there are many fine gradations to this.  The bottom line, Web 2.0 applications plug the user into the network like never before and are a critical rung in the distribution ladder since it offers access to the largest set of content and information by harnessing collective intelligence.
  4. Open APIs and Web services.  This is one of the most important long-term decisions most online businesses can make.  Offering an open API lets anyone take the online components of a business, from its data and functional capabilities to the users themselves, and makes them open and accessible over the Web to be incorporated into other products and services, sometimes in the form of mashups and sometimes in the form of entire online products.  Amazon, one of the first Web companies in existence and is hence far downrange in terms of the experience curve, has been using this distribution model with notable success recently.  So have hundreds of others.  The real challenge has been how foreign this model is to the original Web model and thus to the various management and development competencies in most organizations.  It's much more an a way to OEM a product and leverage the customers and investments of hundreds of other partners.  However, overall, it affords the potential for much larger business outcomes than could ever be created with point Web presence.  It's now considered a significant oversight not to have an open API available for the typical online product.
  5. Web widgets.  Selecting parts of a Web site and it's data and packaging it up to make it run inside a portable, user distributable widget has been growing more and more popular over the last few years. For example, WidgetBox currently distributes 74,000 different kinds of Web widgets from its partners to over 1.2 million other sites.  Widgets lets users distribute a Web site to other places on the Web at no extra cost and it also creates an ecosystem effect, where other Web sites users become the users of the new site.  The YouTube badge is a notoriously well-known example of this that also helped drive the extraordinarily fast growth of the site.  Like APIs, widgets are now considered a mandatory must-have for new and existing online products. But unlike APIs where it's up to the API users, figuring out users want out of your site's widgets is still an art form.
  6. The Plaxo Pulse Story with OpenSocialSocial networking applications.  Sometimes viewed as an extension of the Web widget model, social networking applications are applications designed to run inside of popular social networking environments and usually have capabilities that tap into and make use of the social graph information resident in a user's social network account.  This is an amazingly fast moving field as you can see from a recent post on the latest happenings on the OpenSocial blog, to the extent it's hard even for well-funded companies to keep up.  However, despite skepticism that large businesses can be built exclusively through a social networking application, it's become ever more essential for a site to make its capabilities accessible usable in these environments.  Not only will users help distribute online products in these formats to their contacts but it also increases the overall usage of the your application including participation and its consequently growth of a site's network effect.  While not yet considered mandatory for online products, the ease with which these social network applications can be created and the large numbers of users they make available makes it a smart distribution option for most Web businesses.  Like widgets, however, figuring out what users will find engaging in a social networking application featuring your online product takes some research and experimentation.  However, the results can be very rewarding and some social networking applications have millions of daily users.  See the Plaxo Pulse story on Mashable for the details of how OpenSocial drove a 5x improvement in traffic in only 3 weeks.
  7. Semantic Web and Web 3.0. The Semantic Web, one of the original visions for the World Wide Web, has taken a while to arrive but it's beginning to look like it may hit critical mass in the next 12-24 months.  Combined with Web 3.0, which takes the architectures of participation at the core of Web 2.0 and drives it through a lens of Semantic Web capabilities.  The benefits can be profound and can greatly increase the value and leverage of information on the Web.  While this is very much not prime time yet, unlike #1-#6 above, it likely will be and smart organizations can get ahead of the learning curve and get an early market lead using these techniques.  For now, however, I recommend that most organizations focus on executing well on the first six items before tackling this and waiting for the technologies to finish emerging and maturing.

The list above should provide good guidance for starting move into the potent new models for distribution on the Web.  I'm seeing, however, that because of the major shifts in strategy and product design emphasis these techniques demand, most organizations take an inordinately long amount of time to become effective with them.  The lesson here: Start small now and build core competency.  Small investments now can pay off later in terms of valuable experience made from early experiments and pilots.  When done right,
these new distribution models can become the dominant channels that the world uses to interact with your business, like they already have with Amazon and Twitter.

I'll be talking about these and other strategic online product design topics in my upcoming Building Next Generation Web Apps Workshop at the inaugural Web 2.0 Expo 2008 NYC next month.  I'll have more details about this deep-dive session in an upcoming post.

Filed under Network Effects, Social Business, Social Media, Social Technology & Standards, Strategy, The Web Tagged with , , ,

A Timeless Way of Building Software

January 10, 2006 Leave a comment

Most of my readers know that I’m a software architect by trade.  I’ve been creating software large and small for over twenty years.  And I’ve experienced movement after movement in software design from object-orientation in the 1980s and early 90s to component-based design, distributed objects, Web-based software, service-oriented architecture and too many others to even mention.  I’m pretty jaded at this point because I’ve learned, in general, the problems that you couldn’t solve in the previous generation of technique are often only marginally more solvable in the next generation (which is invariably invented to "fix" the previous problems.)

Alas, a genuinely better mousetrap is really hard to find.

So in the end, if you couldn’t do whatever it is you wanted to do with the previous generation of technique, it’s actually not that likely you’ll succeed in the next.  Certain software problems remain hard, and in general, it mysteriously happens to involve the juncture between technology and people in some way.  To paraphrase this, I could say that the software and techniques get better fairly constantly, but people remain the same.

And please, bear with me because I will attempt to connect a great many dots in a moment. 

Because every once in a long while, something new and big actually does come along.  Or at least something that looks new and big.  One of the new and big things that came along about ten years ago was the concept of design patterns.  It was pretty neat stuff.  It said that despite the current technology we have, the processes that continue to evolve, there are certain timeless solutions to certain software design problems.  It was a revelation at the time.  And the writers of the book that explained this got both famous and very successful.  Why? Because these design patterns really worked is why.  And anyone who has read the books and has ever really built software recognizes these patterns.  And what was strange was that no one really expected it. One day, we just had them.  And the kicker was, they were always there, but now they were in our conscious thought and we had real names for them.  My point: They were in our face all the time but most of us couldn’t see them.

We are in a similar place with the Web right now.  We’ve done this Web stuff enough now that we are just beginning to see the design patterns.  What works, and why, in a specific situations, bounded by forces.  Some folks have had the hubris to give this next generation a name and to tease out these patterns.  Some people are just now going aha, and some people haven’t got it yet, and most of the rest of us either aren’t ready for it or just haven’t heard of it.  But, I will tell you this.  It’s quite real.  The best practices and design patterns of Web software are just starting to become understood.  The strange part is, we’re discovering the same things over again.  What’s old is new again.

Now, before you get all worked up or worse, I bore you and you stop reading, I will give you a nice list of the the forces behind these patterns.  If you recall, design patterns are a solution to a problem in context.  We are starting to get the context and even the outlines of the patterns of this "new" generation of software.  But we have a long way to go still.  The Web is a monstrously big space with big problems, and it’s not getting better.  There are one billion of us out here now.  Clearly understanding what it takes to create great software on the Web that is successful, useful, and vibrant will be an ongoing challenge for a long time.  But it will get easier because we are codifying our knowledge of this exciting and huge place where we now find ourselves.

Comparing SOA, Web 2.0, and a Timeless Way of Building Software
Figure 1:  The driving forces in modern software.
With a rough comparison between SOA
and The Timeless Way (Web 2.0 by any other name).


So, here is where we take a quantum leap to the next level of thinking.  I’m going to use Christopher Alexander’s opening chapter of a Timeless Way of Building and tailor it to describe this old-but-new way of building the Web and software for it.  We are lacking for a little inspiration and this book in particular continues to sell upwards of 10,000 copies a year, 25 years after it was frst published.  And Christopher Alexander, for those of you who may not know, was the person that originally discovered the design pattern.  But it wasn’t for software.  It was for creating great, timeless buildings.  He was one of the first that realized that his field of endeavor has certain elemental, timeless cores, no matter the technique, building material, or the people.  It was an amazing discovery that poured over into the software world with considerable success. 

My assertion is that nothing has really changed in software, we might understand the forces better but they are almost always the same.  People want software that does what they want, is available when they need it.  They want software that grows with them, helps them, teaches them, and lets them do the same with others.  They want software that gets out of their way, disappears, and is more convenient by far than inconvenient.  And they want to pay as little as possible for it, but enough so that it’s worth it.  They are willing to have software get right into the middle of their lives.  If it’s the right software.  And as long as we’ve had software, they’ve always wanted this. But now they might actually start getting it.

In any case, I don’t literally believe every phrase in this take-off, but I do believe the overall concept deeply and profoundly as a software professional.  And I will continue to update the diagram above (clearly marked beta 1) until we have more of the forces in it. And some are definitely missing.  Please, as always, leave any comments and suggestions for improvement below.

And now, without further ado, here is the The Timeless Way of Building Software, with sincere apologies to Christopher Alexander:

The Timeless Way of Building Software
Inspiration For The Next Generation of Web Software


There is one timeless way of building software.  It is decades old and is the same today as it’s always been.  And because it is timeless, it will always remain this way.

The great software of our time has always been created by people who were close to this way.  It isn’t possible to create great software – software that is satisfying, and useful, and makes itself a natural extension of life – except by following this way.  And as you will see, this way will lead anyone who looks for it to elegant, vibrant software which is itself timeless in its form.

It is the process by which the function of a piece of software grows directly from the inner nature of people and naturally out of the raw bits, the otherwise meaningless digital medium, of which it is made.

It is a process which allows the life inside a person, or a group of people, or a community to flourish, openly, in freedom, so vividly that it gives rise, of its own accord, to the natural order which is needed to be contained within it.

It is so powerful and fundamental that with its help you can create software that is as beautiful and enriching as anything else you have ever seen.

Once you understand this way, you yourself will be able to create software that is alive, that is intertwined comfortably with your life and the lives of others. You will design worlds where you and others will want to work, play, and co-exist together; beautiful places where you can sit and dream comfortably.

This way is so powerful, that with its help hundreds or thousands, or even hundreds of thousands of people, can come together together to create software and community which is as alive and vibrant, peaceful and relaxed, as any living experience has ever been.

Without the central control of authorities and experts, if you are working in this timeless way, a genuine place will grow right from underneath your fingertips, as steady as the grass in the fields or the trees in your backyard.

And there is no other way in which a software which is fundamentally good can possibly be made.

That doesn’t mean that all ways of making software are identical.  Quite the contrary. It means that at the core of all successful software and at the core of all successful processes of creation and evolution, there is one fundamental invariant feature which is responsible for their success.  Although this way has taken on a thousand different forms at different times, in different places, still, there is an unavoidable, invariant core to all of them.

Take a look at the some of the great Web software like Google’s search page, Flickr or del.icio.us.  They all have that unique, yet unhurried, grace which comes from perfect ease and natural balance.  But what is it they have in common exactly?  They are beautiful, ordered, harmonious – yes, all of these things.  But especially, and what strikes to the heart, they live.

Each one of us yearns to be able to bring something to life like this. Or just be a part of it somehow.

It is a fundamental human instinct, as much a part of our desire as the desire to be part of something greater than ourselves.  It is, quite simply, the desire to make a part of nature, to complete a world which is already made of mountains, streams, stones, buildings, ourselves, our living systems, and our increasing connectedness together.

Each one of us has, somewhere in our heart, the dream to make a living world, a universe, and place of our own for us to share with others.

Those of us who have trained as software designers have this desire perhaps at the very center of our lives; that one day, somewhere, somehow, we shall build a software experience which is wonderful, beautiful, and breathtaking; a place where people can go and live their dreams.

In some form, every person has some version of this dream; whoever you are, you may have the dream of one day creating a most beautiful place, virtual or otherwise, where you can come together with others and freely share your knowledge, learn, participate in your community or government, and otherwise conduct your daily interaction with the rest of the world.

In some less clear fashion, anyone who is concerned with communities and other large group efforts has this same dream, perhaps for the entire world.

And there is a way that software can actually be brought to life like this.

There is a definable sequence of activities which are the heart of all acts of software design, and it is possible to specify, precisely, under way conditions these activities will generate software which is alive.  All this can be made so explicit that anyone can do it.

And just so, the process by which a group of independent people can make software become alive and create a place as real as any other can equally be made precise.  Again, there is a definable sequence of activities, more complex in this case, which are the heart of all collective processes of software creation.  And it is also possible to specify exactly when these processes will bring things to life.  And once again, these processes can be made so explicit, and so clear, that any group of people can make use of them.

This process is behind the design of community built software like Linux, Apache, Wikipedia, and many others.  It was behind the design of the great virtual places for people to live and work: the Internet, Usenet, and the World Wide Web.  It was behind the creation of simple, satisfying software of the kind that powers the iPod, the Blackberry, and Firefox; of SourceForge, Wikipedia, and BitTorrent.  In an unconscious form, this way has been behind almost all ways of creating software since the beginning.

But it has become possible to identify it, only now, by going to a level of analysis which is deep enough to show what is invariant in all of the different versions of this way.

This hinges on a form of representation which reveals all possible design processes, as versions of one most fundamental set of patterns.

First, we have a way of looking at the ultimate constituents of the environment: the ultimate "things" which a piece of software is made of.  As we shall see, every piece of software is made of certain fundamental entities known as design patterns; and once we understand software in terms of its patterns, we have a way of looking at them, which makes all software, all of their parts and function, all members of the same class of thing.

Second, we have a way of understanding the generative processes which give rise to these patterns: in short, the source from which the ultimate constituents of software come.  These patterns tend to come from certain combinatory processes, which are different in the specific patterns that they generate, but always similar in their overall structure, and in the way they work.  They are essentially like languages.  And again, in terms of these pattern languages, all the different way of building software, although different in detail, become similar in general outline.

At this level of analysis, we can compare many different software creation processes.

Then, once we see their differences clearly, it becomes possible to define the difference between those processes which make software vibrant, alive, and useful, and those which make them the opposite.

And it turns out that, invariant, behind all processes which allow us to make great software, there is a single common process.

This single idea is operational and precise.  It is not merely a vague idea, or a class of processes which we can understand: it is concrete enough and specific enough, so that it functions practically.  It gives us the power to make software and virtual communities live, as concrete as a match gives us the power to make flame.  It is a method of a discipline, which teaches us precisely what we have to do make our software what we want it to be.

But though this method is precise, it cannot be used mechanically.

The fact is, that even when we have seen deep into the processes by which it is possible to make software alive, in the end, it turns out this knowledge only brings us back to that part of ourselves which is forgotten.  Although the process is precise, and can be defined in exact scientific terms, finally it becomes valuable, not so much because it shows us things which we don’t know (though it may do that as well), but instead, because it shows us what we know already.

Of course, this way of building software has never be named.  It’s not service-oriented architecture, or the personal software process, or agile methodology, or the unified process, or CMM, or any of the others.  It’s the actual things that are conceived and done and worried about when software is created and used.  For now, because all software is quickly becoming connected to all other software, and because the Web is becoming the place where more and more of the relevant software is, and finally because it is a more complete reconception of what we thought we knew, we’ll give it a name temporarily.  An unsatisfying name, but one that we can remember for now.

We will call it Web 2.0.

What do you think?  Are we at a place where we can really identify the design patterns in Web 2.0?

Filed under Enterprise Architecture, Social Technology & Standards, Strategy Tagged with , , , , , , , , , , , , , , , , , , ,

Newer posts →