Building Modern Web Apps? Better Have A Deep Competency in Web 2.0, Open APIs, Widgets, Social Apps, and More

To drive the right decisions in what they do product designers, marketing teams, software architects, developers, strategy officers, and other key roles in today's generation of online businesses need to have a solid handle on an extensive array of Web topics.  This ranges from appreciating why plain old HTTP is so good at underpinning the Web to more sophisticated topics like modern application architecture, the latest in online user experiences, next generation computing models (grid/cloud/utility/SaaS/PaaS), cost-effective scalability, user identity, network effects, Jakob's Law, analytics, operations, user community, as well as the many compelling new distribution models that are nearly mandatory in the first release of most products. 

This extensive set of competencies is what's required nowadays to deliver a credible online product to a receptive user base and it has dramatic implications for both uptake and overall cost/time-to-market.  Worse, this body of knowledge has become extensive enough that many Web startups frequently fall far short of what they need to know in order to be successful with these far flung practice areas. 

Does this complex body of knowledge mean the era of the two-to-five person Web startup is coming to a close? Not at all, at least not yet. The productivity level of the latest tools and techniques remains almost astonishing though the level of knowledge required of these teams is creeping up and up.  And as we'll see, new models for product distribution are pushing the capability envelope of the typical Internet startup team to the point we may very well see the day soon that they won't have all the skills necessary to deliver a fully-scoped modern Web application.  It is also one reason why fewer and fewer Web startups have the goods to be all around hits out of the gate.

Certainly, varying depths in subject matter are required depending one's exact role in a Web business, but Web-oriented products are fundamentally shaped the vagaries of the network itself.  Tim O'Reilly himself still has the best quote on the subject: "Winners and losers will be designated by who figures out how to use the network." And as we'll see, the Web is driving the evolution of a major new generation of online distribution models.

Why Adopting New Distribution Models Is Crucial 

As an example of this, I've been tracking some of the latest discussions around the hot topic du jour in the Web world: Social networking applications.  Specifically, it's been interesting to watch the surprisingly low level of industry attention around the titanic competition brewing between social networking application formats from Web giants Facebook and Google.  Why is this?  Some might say it's because these applications still have largely unproven business models.  Others, like Nick O'Neill at the Social Times recently observed (rightly in my opinion) that the struggle may have to do with a deficit in understanding why these new types of Web applications are so important. Nick notes that these widget and social networking style models for packaging and distributing Web apps often "have more eyeballs looking at their products than television channels have" and the challenge is that too many people just "don’t know what any of this means", despite the major players divvying up the online pie for themselves.  With the size of these next generation distribution audiences, ignorance has an extremely painful price: failure to produce results and growth, poor engagement with the marketplace, and loss of market share.

An excellent summary of the truly massive, but largely underappreciated scale of these new Web application models was last week's TechCrunch piece on the progress of Google's OpenSocial, an increasingly successful model for creating portable social networking applications that will run on any OpenSocial-compliant site.  Erick Schonfeld reported that OpenSocial now has a total reach of an astonishing 350 million users and it will soon be 500 million.  There are over 4,500 OpenSocial apps today, a healthy number for the application format but a small drop in the bucket compared to the number of Web sites in the world. But the key is that these applications are integrated much deeper into the social fabric of an engaged audience, interjecting themselves into the daily personal and work habits of the "captive" users of social sites and even have access to the personal habits and data of users of these sites.  Facebook's story is impressive as well with over 37,000 applications that have been installed over 700 million times.

And social networking applications are just one of many news ways that applications have to be packaged and distributed, yet far too many organizations persist in a very 1990s view of Web experiences, namely that Web sites themselves are the center of online product design.  Many even think that some of these other new distribution models are interesting but not part of their core online product.  Unfortunately, that's very much a parochial view in the present era.  Federated applications, atomized content and functionality, 3rd party product ecosystems through open APIs, and much more are required to establish a strong and resilient network effect which fends off competitors that are themselves bringing these potent new competencies to bear. 

 In fact, one of the things we emphasize over and over again in our conference workshops and in Web 2.0 University is that having a Web site is usually the least interesting things about new products.  Worse, it makes the customer have to find you amongst tens of millions of other sites.  Instead, these new models tend to focus on going to the customer, instead of making them come to you which is a much harder proposition. This can instantly give you the ability to reach millions of potential people with dramatically lower effort and cost, as long as you have something interesting to offer.

Unfortunately, the number of capable practitioners of these new distribution models remains relatively small compared to the large body of experts in traditional Web product development.  Demand is also low for these new skills as most organizations have been painfully slow to appreciate how much online product development has changed.  A quick search of the job aggregator SimplyHired tells the tale: Nearly a thousand Web designer positions are available while only 36 OpenSocial and 40 open API positions are open, for example.  This despite the the latter skills being able to project a product across the Web into hundreds of social sites or create an API that allows the product to be incorporated into countless other products for far less cost per customer than traditional methods.

The lesson here is that these new models still have a lot of fertile, unclaimed territory and many otherwise fierce competitors have not yet become fully aware of these new opportunities.  Get your piece of the pie while there's still time. 

I also find that the Web development industry has been slow to change, particularly outside the valley, and there is depressingly scarce information on how to deliver well on things like widgets, open APIs, social networking applications, and even syndication.  To help with this, I've put together a short primer and some good references for those that want to get started.

Because the good news is that there remains tremendous opportunity for growth and success — for both startups and traditional businesses — if they will actively begin incorporating these new product delivery models into their own online capabilities.

Overview of Online Product Delivery Models 

1. Web sites.  This the classic model for Web presence.  During the early Web, creating a Web site was just about the only option for engaging with those online (e-mail being the other.)  Most early Web sites were used for publishing and not for user participation or peer production.  These days, Web sites are still important, though by no means mandatory, and have their content syndicated via RSS and ATOM (pushing the content to where it's wanted), provide an access point to obtain widgets, and maintain user identity, and create communities of users.  Upshot: They've evolved a lot but Web sites are only part of an extensive set of capabilities that must be brought to bear in the Web 2.0 era.
2. Syndication. It took ten years for the Web community to figure out a workable syndication model.  Now RSS and ATOM are now the expected models used to distribute content off a single site and across the Web. Countless aggregation services now exist that make a site's information embedded in their services as well as a way to offer users a method for pulling information from a site and experienced in a means of their choosing, from Google Reader and Newsgator to the innovative Yahoo! Pipes.  Most sites still heavily underutilize syndication even for notifications and pushing out frequently changed information to draw attention to it much less the strategic ecosystem and integration opportunities it affords.
3. Web 2.0 applications.  You might argue that Web 2.0 itself is not a product distribution model but a set of design patterns and business models and that would be a true statement. However, in this context we're referring to the fact that Web 2.0 apps package up the 3rd major type of networked value: user participation.  Before then, Web sites and syndication primarily had only centrally produced content or functionality that they could expose over the network and offer to the marketplace.  In other words, user participation its purest form — sometimes known as peer production –  ultimately results in products like Mechanical Turk and Predictify that provide direct networked access to user participation, but there are many fine gradations to this.  The bottom line, Web 2.0 applications plug the user into the network like never before and are a critical rung in the distribution ladder since it offers access to the largest set of content and information by harnessing collective intelligence.
4. Open APIs and Web services.  This is one of the most important long-term decisions most online businesses can make.  Offering an open API lets anyone take the online components of a business, from its data and functional capabilities to the users themselves, and makes them open and accessible over the Web to be incorporated into other products and services, sometimes in the form of mashups and sometimes in the form of entire online products.  Amazon, one of the first Web companies in existence and is hence far downrange in terms of the experience curve, has been using this distribution model with notable success recently.  So have hundreds of others.  The real challenge has been how foreign this model is to the original Web model and thus to the various management and development competencies in most organizations.  It's much more an a way to OEM a product and leverage the customers and investments of hundreds of other partners.  However, overall, it affords the potential for much larger business outcomes than could ever be created with point Web presence.  It's now considered a significant oversight not to have an open API available for the typical online product.
5. Web widgets.  Selecting parts of a Web site and it's data and packaging it up to make it run inside a portable, user distributable widget has been growing more and more popular over the last few years. For example, WidgetBox currently distributes 74,000 different kinds of Web widgets from its partners to over 1.2 million other sites.  Widgets lets users distribute a Web site to other places on the Web at no extra cost and it also creates an ecosystem effect, where other Web sites users become the users of the new site.  The YouTube badge is a notoriously well-known example of this that also helped drive the extraordinarily fast growth of the site.  Like APIs, widgets are now considered a mandatory must-have for new and existing online products. But unlike APIs where it's up to the API users, figuring out users want out of your site's widgets is still an art form.
6. Social networking applications.  Sometimes viewed as an extension of the Web widget model, social networking applications are applications designed to run inside of popular social networking environments and usually have capabilities that tap into and make use of the social graph information resident in a user's social network account.  This is an amazingly fast moving field as you can see from a recent post on the latest happenings on the OpenSocial blog, to the extent it's hard even for well-funded companies to keep up.  However, despite skepticism that large businesses can be built exclusively through a social networking application, it's become ever more essential for a site to make its capabilities accessible usable in these environments.  Not only will users help distribute online products in these formats to their contacts but it also increases the overall usage of the your application including participation and its consequently growth of a site's network effect.  While not yet considered mandatory for online products, the ease with which these social network applications can be created and the large numbers of users they make available makes it a smart distribution option for most Web businesses.  Like widgets, however, figuring out what users will find engaging in a social networking application featuring your online product takes some research and experimentation.  However, the results can be very rewarding and some social networking applications have millions of daily users.  See the Plaxo Pulse story on Mashable for the details of how OpenSocial drove a 5x improvement in traffic in only 3 weeks.
7. Semantic Web and Web 3.0. The Semantic Web, one of the original visions for the World Wide Web, has taken a while to arrive but it's beginning to look like it may hit critical mass in the next 12-24 months.  Combined with Web 3.0, which takes the architectures of participation at the core of Web 2.0 and drives it through a lens of Semantic Web capabilities.  The benefits can be profound and can greatly increase the value and leverage of information on the Web.  While this is very much not prime time yet, unlike #1-#6 above, it likely will be and smart organizations can get ahead of the learning curve and get an early market lead using these techniques.  For now, however, I recommend that most organizations focus on executing well on the first six items before tackling this and waiting for the technologies to finish emerging and maturing.

The list above should provide good guidance for starting move into the potent new models for distribution on the Web.  I'm seeing, however, that because of the major shifts in strategy and product design emphasis these techniques demand, most organizations take an inordinately long amount of time to become effective with them.  The lesson here: Start small now and build core competency.  Small investments now can pay off later in terms of valuable experience made from early experiments and pilots.  When done right,
these new distribution models can become the dominant channels that the world uses to interact with your business, like they already have with Amazon and Twitter.