Social Technology & Standards | On Digital Strategy

How Digital Collaboration is Fragmenting, and Why It’s a Major Opportunity

June 29, 2015 5 Comments

A significant issue has been developing in digital collaboration for the last several years, and it’s now starting to become somewhat acute. I’m referring here to the pronounced trend towards app, environment, and channel fragmentation. Over the last few of years, I have been speaking with beleaguered IT managers who are struggling to cope with the sheer proliferation of software, systems, and applications that purport to help workers with collaboration. It’s not a new problem, and smart folks like Dave Winer have long worried about it, but it’s now becoming a vital strategic concern.

A variety of factors are contributing to fragmentation: Every department and function now seems to have existing vertical systems — such as their standard HR, sales, or customer care solutions — that have recently added social media, collaboration, sharing, messaging, shared content editing, document attachments, activity streams, rich user profiles, and so on to their feature sets. At the same time, many exciting new applications have emerged on the scene recently that seem nearly must-have to many of us: Dropbox, Box, Slack, even arguably IBM Verse. All of these in turn compete with the officially sanctioned collaboration applications already in the workplace currently, from e-mail and SharePoint to whatever enterprise social network and unified communications platforms have been selected under the CIO’s purview.

Some of these new collaboration applications are brought through the front door in by lines of business that feel they have special needs. Others are so-called “shadow IT” deployments by teams and departments who believe they require certain features or prefer the ease-of-use of alternative collaboration tools, but don’t want to go through the formal hassle of getting blessing. Finally, a good many come in via legacy adoption via mergers/acquisitions or through individual users using their own devices and app stores.

Note: User-driven IT itself isn’t the problem here, it’s actually a key source of opportunity if wielded properly in a network of enabled/supported change agents.

Too Many Collaboration Tools, Not Enough Collaborative Reach

Whatever the source, this trend is creating dozens — and sometime hundreds, in large enterprises — of collaborative silos, where participants and their information are trapped, inaccessible and invisible to a broader range of potential actors. Worse, unlike the original communications tools in the industry — like e-mail — the walled garden trend that started with the great consumer social networks — largely to support business models and not to help users — has decisively shifted to enterprise collaboration software today. Thus, unless you have a license and are using the exact same app, chances are increasingly poor that you can collaborate with someone unless they are using the exact same toolkit and environment.

I’ve pointed this out in the past, the we have an urgent problem with our collaboration tools not talking to each other. I’d say it’s now a critical issue that threatens the very high-value, human-centric activity that we are supposed to be enabling: Better collaboration. In addition to the typical common issues, I can’t tell you how many times I’ve heard that users have moved away from a collaboration tool because they can’t easy work with customers, or business partners, who are usually outside the company and don’t have access to the same tools or environment.

Now the issue is not just that we have barriers, but that we have so many new applications that we are employing, very few of which are interoperable. The result is that we are creating a growing array of inaccessible bubbles of insulated collaboration within our organizations. Much of this is done in the name of achieving worthwhile goals: Accessing powerful new capabilities, modernizing our workplace tools, improving security, and/or making sure we have vendor stability. However, we’re learning that we are often creating a solution that’s potentially much worse than the problem we’re trying to solve.

Commercial Silos of Social and Collaboration

In Social, We’ve Moved Once from Open Federation to Silos. Now We Must Move Back.

One of the challenges of adopting new digital solutions for collaboration has been that they’re often championed by those for whom technology isn’t their primary background. Consequently, federated architectures, open standards-based technology, and interoperability usually aren’t high on the list of sought after features for collaboration tools by most business users. Partially as a result, growing islands of collaboration have become a very real problem today, as lack of decent connection between our collaboration tools is — instead of creating a large and growing body of collective intelligence accessible to all — is actually resulting in parochial backwaters where too few people use the tools to make them worthwhile.

Certainly, some of the latest additions to the industry pool of collaboration options — and yes, I’m talking about Slack here — are designed specifically to address this issue, as it becomes one of the largest — and most ironic — new obstacles to effective collaboration: Too many apps and channels for working, none of which share well with each other. Solutions like Slack make all the knowledge and content flowing through our many individual business applications, visible from one collaboration platform. And that’s certainly one major way to solve the problem. Certainly, approaches like OpenSocial have tried to tackle it in other ways, and now the W3C Social Web Working Group is looking at the issue as well. So perhaps we’ll still end up with an SMTP for collaboration, but we don’t have it yet.

In the past, I’ve exhorted our industry — especially the most strategic and important aspect of it, social collaboration — to stop the fragmentation and create interoperability standards. But vendors — who can only exist when there are enough customers — have little incentive to help others, nor do their users insist on it. In fact, I’ve steadily come to believe that the problem with not be solved by vendors, customers, or even standards bodies, each of which has a) corporate goals contrary to a real solution to making collaboration tools work together, b) don’t fully understand the details of collaborative systems and their management well enough, or c) can’t gain traction because of the first two issues, respectively.

A Quick Back-of-the-Envelope Proof

As a cross-check, especially since I’m seeing the flow of new collaboration applications increase rather than decrease, I took a look (results in first diagram above) at some of the top types of collaboration tools (content/document management, intranets, social collaboration tools, ESNs, unified communication platforms, e-mail, and mobile collaboration tools) and put them on another dimension against various leading corporate functions (marketing, sales, operations, customer care/support, research & development, HR, legal, IT, and supply chain), and I was able to quickly find many apps — often some just newly emerged — that could fit in each and every intersection between the two axes. This would not have been possible 2-3 years ago. In other words, everything we do is quickly becoming collaborative.

And as a result, the risk is that soon little will be, as collaboration is divided across hundreds of isolated systems that we mostly can’t see and don’t have access to internally, sharply limiting the rich results from collaboration that only open technology can uniquely provide: Working out loud, open and transparency business practices, a corporate body of knowledge, and reuse and learning from everything that the company knows, all lying searchable on the corporate network.

The Issue of Collaborative Silos Must Be Solved. And Because They Must, They Will. But When?

Instead, as the issue becomes a top one for many corporations, I now believe it’s more likely that we’ll see inclusive approaches (again like Slack and a few others like Xendo have done) that ensure these barriers don’t form, that are based on market drivers and ultimate customer value. In fact, I now see that some customers are increasingly frustrated that they can’t use their shiny new tools to work with everyone they want, or are cut-off from the communities, channels, and knowledge that they need to do their job. Fortunately, these stakeholders are the ones likely to drive the changes we need to see.

This may perhaps, at least for the smart software companies and open source projects that understand this increasingly urgent issue, be the next big opportunity for them: They must be the integrating force, the unifying center of collaboration for the enterprise, bringing all the major applications, systems, and data pools together — and make it easy for IT or others to bring in their own local apps — so that we no longer have such highly ironic digital isolation.

I fully realize this issue is not one that people can get as passionate about as the main topic of contemporary collaboration. But unless we fully understand what kind of results we are really creating, we’re going to be building as many walls and barriers as we are new modes and venues of digital collaboration. If we want, we can greatly accelerate a new and better way, a more unified way, and push for interoperability that’s as good or better than e-mail has, then truly create the collaborative worlds of our dreams.

Finally, it’s not up to someone else to make sure this takes place. It’s up to us.

Additional Reading:

The digital collaboration industry continues to flourish

Watching digital collaboration evolve: Key events in the last year

How to improve global workforce collaboration

Filed under Analysis, CoIT, Social Collaboration, Social Media, Social Technology & Standards, Strategy, The Enterprise, The Future of Work

How We Gave Up Control Over The Social Web

November 13, 2013 1 Comment

A short but pithy piece over the weekend by Dave Winer titled “Why the Web 2.0 model is obsolete” got me thinking about where we’ve ended up with social media after nearly ten years. Blogs, wikis, and other tools of easily shared self-expression from the early days have given away in recent years to a much less diverse social media monoculture. A few large social networks now control our social identity, content, and behavior, and through their terms of service, often literally own our online existence legally and de facto.

This evolution was perhaps inevitable given the rules through which networks operate and certainly the result has its strengths. It’s far easier for consumers and businesses to adopt a hosted service than set up their own social presence, with all the complicated bits it requires to set up a fully functional social identity on your own these days. It’s also probably more secure, safe, and reliable long term. It’s certainly the shortest route to connecting with the vast captive audiences that the leading social networks now wield.

Yet in the process of making many short term decisions in the name of reach and convenience, many of us have given away our social capital, and along with it much of our online autonomy and freedom. I’ve long since stopped advising companies to drive their traffic to Facebook (disclaimer: I am a shareholder) and build their own online communities and digital ecosystems if they are intending to be strategic about things like social business and open APIs.

The impressive thing is that we’ve largely achieved the original vision of Web 2.0 and it’s just how we do things now. We share by default. We use social media more than any other digital activity. Social media is now woven into so much of what we do today. Yet the majority of all of this user generated content and online community is now centralized in a few large social silos that can no longer talk to each other.

Even worse, if we go the opposite direction that might seem better long term, we’ve discovered issues with that model too. For example, we’ve learned that when we create many smaller, self-controlled, and more autonomous social environments we then create fragmentation. We can’t easily communicate or collaborate with each other across these social islands. Thus, for as many downsides as the monolithic social networks have, they do achieve one important thing: They create a very large single social universe that we can all communicate across.

So what should we collectively do? Should we cave in and trust that the corporate owners of the social world will be benevolent, even when they clearly have business models that are very often at cross purposes to our needs and desires? Or should we find a way to solve the problem of creating our own social corners on the Web and then connecting them together, all while making it very easy to do so? Personally, I’m hoping it’s the latter. Certainly I’ve explored previously how open social standards have a genuine shot at helping with this, even if it might be a bit of a long shot.

The reality is that social media silos are now holding us back, both as individuals and as businesses. We can do much better if we want. But getting there requires a little long-term discipline and plenty of widespread demand. That makes it pretty unlikely in the face of the enormously strong network effects of the largest social networks today. But perhaps there’s a third option to regaining control over our social lives. In fact, I predict the next big breakthrough in social media is likely to come from the need to resolve this tension between the unfortunate long term consequences of centralized social media and the benefits of a much more federated and user controlled model. Unfortunately, recent history has been a steady march towards the former.

So until then, we all need to mull over where our collective decisions are taking us, for as social media is perhaps the greatest communication revolution in history, its intrinsic power cuts both ways.

Filed under Analysis, Network Effects, Social Business, Social Graph, Social Media, Social Technology & Standards, Strategy, The Web, Trends and Statistics, Web 2.0

Social Business Moves to Workflow, Manufacturing, and Money

September 26, 2011 1 Comment

I receive e-mail frequently from PR people promoting the latest IT tools and new Web applications. These days a common thread I see is the addition of social features to software to make it easier for users to share information and collaborate with others. Personally, I believe it’s largely beneficial to 1) find ways to take advantage of the social graphs that users have been building in recent years, and 2) add the techniques and channels of the social world to make traditional software more effective and usable in general.

However, in reality these relatively minor tweaks are just the proverbial paving of the cowpath through the addition of limited social features such as collaborative sharing, persistent chat, and perhaps some deeper integration with activity streams. Unfortunately, these actions easily fail the imagination test, which is essentially this:

If you could completely rethink your work in a social business world, what would it look like? How would it be better?

To me, this is the fundamental question that organizations must be asking themselves today. Yet, I also think they should do this while going about the aforementioned incremental improvements such as adding basic social layers to their IT landscape. One reason is that this will happen inevitably as more and more enterprise applications and platforms add social computing features and companies proceed along that vendor’s upgrade path. So, while social impinging around the edges of enterprise applications is worth dealing with from a strategic perspective, it’s going to happen largely whether organizations plan for it or not. As such, it’s not likely to make a huge competitive or qualitative difference in the way most businesses perform. That is, unless they start the process of deliberate and strategic social business transformation, such as what IBM and a few other large organizations have begun.

This process of social business transformation will require both advances in social technology — such as the innovations below — as well as changes to the way we do business. Fortunately, one of the great attributes of the larger social business community is that it generally focuses as much on the business and cultural changes as it does the enabling technology. Some of the best discussions I’ve seen on the people aspect of the transition to the social enterprise are from folks like Luis Suarez, Sameer Patel, Stowe Boyd, and JP Rangaswami, who are just part of a much larger conversation about how we remake our organizations for the 21st century.

So, while there are certainly some companies not tracking the sea changes in the world right now in terms of the way we are globally transforming the way we live and work, we’re also continuing to see fascinating next-generation innovations in social business. Let’s take a look at some of them.

Rethinking Workflow, Manufacturing, and Money in Social Business Terms

In just the last week I’ve encountered several fascinating offshoots of the mainstream social business thread. Social business frequently focuses either on social engagement externally or internally on collaboration and social interaction between workers. This is a limiting view, but it’s also where most of the activity and uptake is today. However, as more and more business leaders and entrepreneurs become digital natives, I’ve theorized that the power laws and principles of social business will encourage them to rethink their traditional modes of business. At the same time, Web startups and large software vendors often put themselves out 2-3 years ahead of the market by predicting where their customers will arrive once current trends reach a mainstream tipping point. Then they adjust their product roadmaps to align with this schedule. The combination of these two trends is starting to give us some interesting new possibilities.

I say possibilities, because unlike social collaboration or Social CRM, the outlook and growth potential for these innovation is still unknown. However, it does give us a sense of what’s coming next in social business.

Social BPM

Last week while I was speaking at Sibos, I had the pleasure of speaking on the phone with Sandra Moran from OpenText Metastorm, a leading workflow/BPM product that recently announced the addition of social computing features to its capabilities. Metastorm now enables workers to engage in real collaborative process design, takes advantage of social profiles to locate needed expertise to plug workers into processes in essentially real-time, and has matching dashboards to provide BPM and social analytics. OpenText had this to say about the new social capabilities, which Sandra told me is now available to over a thousand major customers as a standard part of the Metastorm suite:

These new product enhancements help organizations successfully implement business process improvement initiatives by empowering users to become more engaged and productive. Metastorm’s social collaboration tools provide businesses with a highly personalized workspace and unparalleled access to top contributors, enabling them to drive innovation and increase collaboration and improve efficiency among employees. These tools help employees find other people within their organization with specific skill sets required to help them complete their work. Companies can also route work to the most appropriate employee based on individual skills and workload – ensuring the most cost-effective strategy for work allocation.

I think this is significant for a few reasons. For one, I find that there’s often not enough focus in social tools in collapsing the walls between business processes and social conversations. They often run in parallel, side-by-side, even when they are being used simultaneously for the same piece of work. Putting social in the flow of work in highly process-intensive environments should lead to some interesting outcomes. I pressed Sandra on if there was leverage in Metastorm of existing social graphs and networks, and she indicated there was. What remains to be seen is how easy it will be to integrate the resulting BPM environment with an enterprise’s other social business efforts.

I’ll be exploring the social features of Metastorm in more detail soon on ZDNet, but I think the combination of social computing and BPM has genuine potential. This isn’t the first time social and workflow have been connected but I think it’ll be impactful given their large customer base and how central and useful the features are to the product. I’m hoping to revisit how their customers are faring in a year or so to see what the result has been. I currently believe social BPM technology, combined with the right business and cultural changes, will help companies attain a higher than average level of social business transformation.

Social On The Shop Floor

Earlier this month Derek Singleton over at the Software Advice blog wrote about social manufacturing, what you could call a new subfield of social business that’s focused on improving how companies turn raw materials into finished goods. Discussing Kenandy’s new announcement for improving the efficiency and productivity of supply chain manufacturing, Derek wrote:

Creating accessible and actionable inter-shop floor communication can only work if an entire supply chain and other manufacturers are members of, and logged into, Chatter. In short, it requires organizational change for effective use. While manufacturers using Kenandy wait for that changeover, Chatter can be a useful tool for project management. For instance, the engineer of an aerospace job shop could notify shop labor that they’ve just finished designing the wing component of an aircraft. The job shop could then begin building the wing while the engineer finishes designing the other components they’ve been contracted to build. This has great implications for just-in-time (JIT) manufacturing – as it frees up labor to work on more value-added activities rather than waiting for the completion of another phase of the production.

In my workshops at Enterprise 2.0 Conference in years past, I’ve had manufacturers and assembly line managers come up to me to say that social tools have been moving into their area of the business, but it’s mostly been horizontal tools or very focused niche solutions. We’re now seeing broader and more strategic use of social tools with the arrival of solutions such as the Kenandy social manufacturing platform, which has garnered attention in the New York Times. I’ll be exploring this further in coming months to see whether social manufacturing leads to tactical or substantive social business transformation.

The Rise of Social Currency

Finally, at Sibos itself last week, I participated in Innotribe, a social media event inside the main financial services conference that explored various aspects of social media in financial services. For a more in-depth look, I wrote up a detailed exploration of the event on ZDNet on Friday. One of the more interesting and visionary topics at the conference was the subject of social currency, the transformation of the very concept of money in social world where reputation, trust, and openness are prized much more than information control, the latter which is how the financial industry is mostly structured to leverage for gain today.

As an experiment, a social currency called Reputone was actually in use at Innotribe, see picture right. In fact, peer-to-peer monetary systems such as Bitcoin were a hot topic at Innotribe and for good reason, it represents a major shift of control in how banking, money transfer, and investment will work in the future. If Paypal was the first generation of digital money, then Bitcoin is the Web 2.0 version. From their Web site:

Bitcoin is a new digital currency that enables instant payments to anyone, anywhere in the world. Bitcoin uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network.

Mark Shead recently provided a good overview to Bitcoin concepts and is worth taking a look at. In the final analysis, Bitcoin falls a bit short of being a true social currency, in that it doesn’t have an explicit capital mechanism based on social graphs or other means that leverages the intrinsic worth of social status and reputation. That doesn’t mean it should be watched closely as money and social reputation appear ready to get deeply intertwined and Bitcoin is at the leading edge of digital currency at the moment. This is a subject that warrants a lot more exploration as companies such as Facebook look at making their global platforms far more relevant from an economic perspective. For additional insight, David Armano posted some useful insights on social currency recently on his Harvard Business blog.

I’ll be exploring all of these concepts in more detail in coming months as social business continues to evolve. I would love your questions and feedback on this emerging social business topics below.

Filed under Analysis, Social Business, Social Graph, Social Media, Social Technology & Standards, Strategy, The Enterprise, Trends and Statistics

Putting Social Business To Work

August 24, 2011 25 Comments

A great post yesterday by Laurie Buczek brought home for me a key issue that I’ve been pondering lately, namely how surprisingly disconnected some social business efforts end up becoming. We know many of the reasons this happens: Not-invented here, political fiefdoms, integration challenges, the tendency of many applications to turn into silos easily, etc. However, social media in the enterprise is about connecting deeply to those around us to improve the way we work. It’s certainly not about isolation, yet that sometimes becomes the state of affairs. How we organize for social business determines much of our success, as emergent as the process is. As Laurie said in her post (her emphasis):

The big failure of social business is a lack of integration of social tools into the collaborative workflow.

I should be clear that it’s not social business as a concept that’s the problem here. It’s that social must be connected to the day-to-day work that takes place. Unfortunately, most work today is done through existing systems that aren’t very social. If we’re lucky, we can forge a link to a piece of enterprise data from within a social tool, a basic requirement for social collaboration. But more likely we have to manually copy information from the systems of record in order to collaborate on it. Even more likely, the social business environment just becomes a parallel silo that’s not connected to the business and is used for light conversation and status updates instead of meaningful, high value line of business activities.

Yes, many large ERP, CRM, and HRM vendors including Oracle, Salesforce, IBM, Saba, and many others have either added or are otherwise incorporating social layers in their products that can help address this. But this is not necessarily the same as making our businesses fundamentally or more meaningfully social. Such duplication of social tools has its own silo issues and ultimately, rolling out social software on its own does not in itself produce results. No, the ladder of social business maturity requires more from us than that.

Instead we need to wrap our businesses in social in a more ambient and deeply connected manner. To work, this must be more than for example merely adding threaded conversations to our systems of record. It’s about weaving collaboration into everything we do, efficiently and simply. The good news is that there’s now hope to readily address what Laurie was referring to and connect social to workflow. With recent advances like real, mature, standardized social integration with OpenSocial 2.0 — with widespread support by enterprise software makers for the first time — there’s a genuine opportunity, right now, for us to connect our daily departmental and enterprise-scale work activities en masse to an overall social fabric that enables real change, real results, and real ROI.

Note: I do not think pure technology can ever be the full answer to this issue. But whenever we have a means of much more easily putting social in the flow of work we must go well beyond paper strategy and employ them.

So it’s up to us to see the importance of doing this and making it happen. Want social business become just a fancy chat tool in your organization? Don’t put social business to work. Do you want to unleash untapped worker potential, including cognitive surplus, peer production, and collective intelligence and all the big strategic buzzwords? Then put social business to work. The big lesson here: Failure to connect social business to work on the ground will pretty quickly result in limited value. We are now in the possessions of techniques to avoid this and we must use them.

See my writings on connecting business software to systems of engagement, social networking applications, and social app stores for more details on this subject. The Social Business CIO Shortlist can help as well.

Filed under Social Business, Social Media, Social Technology & Standards, The Enterprise

Sunday Musings: Google’s Identity Struggles, Plus Social Media Bans Around the World

August 21, 2011 Leave a comment

The Web’s missing features for built-in user identity have become a real headache for the industry, and for its users too. It certainly took its toll on market leader Google this week as its “Identity Theater” continued (Source: Kevin Marks.) The issue? It’s turning out that making every single user comply with the Common Names policy isn’t workable for a variety of reasons. Reports of Google deleting accounts en masse are driving a lot of the discussion. Robert Scoble has his own recommendations for Google and while they’re probably the least that would be acceptable to the majority of people, it doesn’t go far enough I think.

It certainly doesn’t have to be this way. Twitter allows companies, bots, and just about every other type of social account and it works quite well in the end. Twitter ran into a similar identity issue in a big way a couple of years back after facing lawsuits and widespread complaints. They managed to muddle through with Verified Accounts.

A growing consensus is that Google should allow user-defined accounts as well, with verified identity for those that want or need it. Personally, I’m not sure I see Google coming around with a response fast enough to prevent some damage to services and impacting Google Plus‘s runaway adoption. But in my analysis, it’s most likely to only hurt the commercialization of the service, not regular usage for most for now.

Worse, the problem may actually be core to the way Google’s stack is conceived and architected. It may not be easy for them to change course in the short-term without ripples through the way global Google’s services fundamentally operate from a security and identity perspective. It also may not be good for their business model which is almost certainly based on the fact they know who people really are. This issue is one to watch given Google’s pervasiveness. It also has some significant implications for business users of its products, especially now that they seem to be gaining some much needed traction in the social networking wars.

For now, I’d recommend that businesses use Google Plus with an eye towards experimentation while the Web giant gets its philosophy and policies around identity sorted out. Frankly, the bigger industry issue is social Web identity itself. Users and companies increasingly depend on commercial providers like Facebook, Twitter, and Google to provide everything identity-related, from login access to storage and maintenance of their social graph. This is causing key elements of power and control to start to swing away from the open standards that made the Web so successful and essentially fair.

Will the W3C step in and resolve what’s appearing to be an increasingly glaring absence in the Web stack? So far it seems unlikely given the failure of many years of open standard Web identity efforts. The culprit? You have only to look in the mirror. Apathy by users and lack of consensus on the part of Web developers. There’s also a lot at stake financially for those that end up owning a big chunk of Web identity. Consequently, online — and especially social — identity is likely to grow into a full blown brouhaha in the next couple of years as issues, missteps, and abuses inevitably surface. However, we could also decide to put our own house in order before governments step in, the least desirable of all outcomes in most imaginable scenarios. The worst probably being governments owning, issuing, and centrally managing verified Web identity credentials for everyone.

Which brings us to the next subject…

Government Bans Chipping Away At Social Media Freedoms?

A couple of interesting things happened this week with governments aiming their considerable might at social media. While knee-jerk responses to this space were common enough a few years ago, with the U.S. Marines banning social media access for a while for example, these are now generally understood to be counterproductive and unworkable for a long list of reasons.

However, that didn’t stop the German government from banning the Facebook ‘Like’ button on Friday, sure to ignite a small firestorm in that country given that it seems to apply to any site accessible from inside its borders and the fine is a stiff €50,000. The Like button, used on millions of sites around the world to enlist users to leverage their Facebook social network to share content from 3rd party sites (see: k-factor), is significant enough on its own to put German Web businesses at some competitive disadvantage on the global stage. The concern is over privacy and that “all the information was sent to the US company even if someone was not a Facebook member.”

In another similar situation, the Missouri state government’s new law preventing teachers from using social media to communicate privately with students, the former who just announced that they are fighting back, is another case in point. There are obvious free speech issues with the law despite the good intent on its face to protect students. The real issue is that the law is that violations are almost impossible to detect and enforce, until its too late, and that it ensures teachers, one of the most collaborative and interaction driven professions with far reaching impact, can’t have much of a social media presence of any kind until the implications are sorted out. It also presumably doesn’t prevent teachers from privately communicating with their students in any number of other digital channels. All of this means the law won’t accomplish a whole lot other than sowing confusion and promoting the use of increasingly obsolete methods in an increasingly fast-changing economic and societal landscape.

The real issue with both of these laws is that they are 1) essentially short-sighted, 2) exhibit such poor understanding of social media as to be essentially useless, and 3) are therefore unlikely to be meaningfully carried out. Worse, they chip away at the edges by introducing step-by-step, largely ineffective government oversight and control over social media, one of the largest economic, cultural, and societal changes of our time. This will become an even hotter topic as the Middle East’s social media coordinated model for uprising spills out of the developing world. In fact, this has already happened in Britain and there are already cries to ban social media in cases of civil unrest.

I should be careful to note here: I’m not by and large suggesting there’s any overarching government scheme to interfere with and control social media. Instead, I’m suggesting we keep a close eye on these developments as social media legislation increasingly (and inevitably) accumulates in bits and pieces on the base of knee-jerk responses to individual situations. This will have a great many unintended and unwanted consequences. The continued growth of laws and regulations in a vital new industry that thrives on inherent openness and trust has the potential to limit it so profoundly that we could lose much of the great promise that social media can provide.

While we must find ways that work to protect our citizens, we must also provide them access to one of the most open, free, and powerful means of interacting that has been invented. Let’s push back on unreasonable measures while also proactively being responsible for solving them. It’s up to us to start finding globally acceptable solutions to privacy, security, and misuse in social media and getting them into the hands of those who don’t understand this space well enough yet to govern it. The options for making this happen are something I’ll explore as soon as I can.

Filed under Analysis, Social Business, Social Graph, Social Media, Social Technology & Standards, Strategy Tagged with business, control, Facebook, Germany, Google, laws, open identity, privacy, regulation, security, self-regulation, socail networking, social graph, standards, strategy, transparency, trust

The Social Graph: Issues and Strategies in 2008

January 16, 2009 4 Comments

One of the hottest topics in the online world in the last couple of years has been the growth of social networking services such as Facebook and MySpace, as well as the addition of a social element to existing user experiences. Despite riding several waves of hype, it's now clear that the social networking space will only get hotter in 2008 according to most watchers. Social software has come fully into its own as of 2008 — for all appearances permanently — and understanding the reasons for this rapid rise as well as figuring out how to leverage it best is the job of everyone who wants to make the most of the Web 2.0 era.

Gaining a deeper insight to the social networking phenomenon, now exhibited by the tens of millions of users employing them globally on a daily basis for both personal and businesses uses, currently means understanding the fundamental unit of the social network, also one of the biggest new buzzphrases of the year: the social graph. Fortunately, that's simple enough despite the term's oblique reference to graph theory, which it is heavily based upon.

Social Graphs - The pattern of social relationships between people

Simply put, a social graph is a set of people, referred to as nodes, that are connected together by vertices — better known as links or connections — that reflect their social relationships. You can see a conceptual social graph above, showing the typical distinction of social networks to reflect whether a connection with another person is direct or indirect. For example, the popular business social networking service LinkedIn, uses this model and sorts a member's social graph into different degrees of separation, which you can see a typical example of below and taken from my LinkedIn profile:

Also becoming popular is the burgeoning field of social analytics, such as the Socalistics application in Facebook and the Interactive Friends Graph, though there are also commercial standalone products here or on the way for the enterprise and open Web spaces from companies like KnowNow and Bravadosoft. The Interactive Friends Graph is a nice, simple example anyone can try on their own and you can see mine from Facebook below. Hovering over nodes in the live version in your Facebook profile allows you to see who is connected to others in your network and begin to gain insight and understanding of the relationships in your network.

But what are the top issues one must understand about the social graph in 2008? As I've seen social networks become common on corporate intranets and in daily use on the Web, some of the issues are rapidly becoming clear. However, the full story will certainly continue to unfold for the next several years at least. Here's what we're seeing at the moment:

Strategies and Issues for the Social Graph – Circa 2008

The social graph is poised to replace the address book and contact list as the preferred organizing structure for personal and business relationships. This was one of my Web 2.0 predictions for 2008 and it won't fully come true for the majority of users for at least several years since there's such an installed base of traditional tools for managing relationship information. What's the difference? Social networks are usually opt-in, two-ways for one. And they are social for another, meaning they tend to encourage communication and collaboration, such as through user profile event streams and status messages. They also offer up and actively make use of the deeper insight into the full graph's social surface area beyond direct contacts, such as LinkedIn's introduction service.
Ownership of the social graph is going to be a ground zero issue in 2008. Robert Scoble's widely covered attempt recently to use Plaxo Pulse to export his 5,000 Facebook contacts recently got him banned temporarily from the service. But as users begin to realize that the contact lists they are building using online Web tools might not be portable, this will become a growing concern, particularly since two-way opt-in makes a social graph more valuable (and accurate) but significantly harder to recreate on demand elsewhere. This takes us to our next subject…
Many social networking services will adopt open data initiatives. Both Google and Facebook recently showed support for DataPortability.org and Google has an interesting play in their OpenSocial initiative. This is welcome news that will resolve some of the concerns around who owns the graph but interestingly, traditional corporations will be the slowest get this and will rarely let workers take their hard won social graphs and user profiles with them elsewhere as they move to new jobs. Public social networking sites Web sites are leading the way here and this will only drive more business users to the open Web, where they at least have some control over their social graph. Smart organizations will provide their workers with some form of open social graph support, lest they lose control completely as workers keep more and more of their graph in Facebook, LinkedIn, and Plaxo and not in prescribed relationship management tools.
Attempts to monetize social graphs will drive interest in regulation and legislation. Social networking is now a global Internet phenomenon and that the information contained within them is highly central to everyone's lives. This will make everything from protecting children to individual privacy of social graphs a hot issue for some local and federal governments. All it will take is one or two widely covered exploits to make this happen. Expect the European Union and the U.S. government to begin seriously examining the issue this year with many other governments following suite. Good citizenship of sites that manage social graphs will be essential to prevent excessive government involvement.
The line is blurring between personal and business use of social graphs. We're all rapidly getting one large social graph each already, with everyone we know in them. Most public social networking sites do a poor job of separating different subgroups of our social networks, such as allowing pictures and status messages to only go to a specific subgroups (work messages to business, family message to family, friends messages to friend, etc.) This actually works a little bit better in enterprise social networks, but not much, since it largely consists of a Contact Type field. Segmentation of social graphs will be an increasingly requested feature by users struggling with their use. The social graph management services that make this distinction and enable its leverage may do very well indeed.
Open Web identity, which will ultimately form the global "primary key" for social graph nodes, will not get anywhere soon. This despite it being needed badly but the users of the Web have not yet felt compelled to demand it. Data portability of social graphs will begin to drive adoption of user controlled Web identity, and hopefully government regulation will not. See Dare Obasanjo's deep exploration of using openid to enable social graph interoperability as an example of what will need to happen, despite there being little incentive currently for sites to use other site's openids.
Making social networking "gardening" and administration easier will drive new innovations. Most individual social graphs are primarily tended by hand today, although a growing number of products, such as Visible Path, do all the tedious work for you by watching your social interaction online such as through tight integration through e-mail and instant messaging, building a rich graph for you (even sending invitations) as you go about your daily social activities. New innovations like these will make social graphs easier to maintain and richer in overall information while also driving adoption through ease of use.
The optional two-way confirmation of a social graph link becoming standard. Many social graph management platforms (Facebook and Linked for example) require confirmation from the other side of the connection before adding a person to your graph. Sites like Spock, which make it optional, will ultimately be more practical for managing a social graph while still allowing discernment of two way confirmations, which tend to be more valuable and convey key information about the trust and real extent of a social relationship.
Social networking fatigue will not set in as perceived constraints such as Dunbar's limit do not prove to be universal. While there are many theories on how big a social graph can get before it become unmanageable and sees diminishing returns on growth (note that both Facebook and LinkedIn encourage ceilings), the fact is that the are many different purposes for a social graph, from data mining and historical research, to marketing and customer relationship management.

What else is going to be key to dealing with the social graph in 2008? Please leave in comments below and I'll update this post with any good submissions.

Filed under Social Business, Social Graph, Social Media, Social Technology & Standards, Strategy Tagged with "social graph", analytics, network analysis, social business intelligence, software

Building Modern Web Apps? Better Have A Deep Competency in Web 2.0, Open APIs, Widgets, Social Apps, and More

August 24, 2008 Leave a comment

The Web has an interesting property that those building Web applications and online businesses usually encounter soon after they first launch: It has its own unique and unforgiving rules for success and failure. Appreciating them requires a certain level of understanding of the intrinsic nature of the Web and how it works. Actually leveraging those rules requires an even deeper and more profound understanding of the Web. The challenge these days? The Web competency bar is climbing fast.

To drive the right decisions in what they do product designers, marketing teams, software architects, developers, strategy officers, and other key roles in today's generation of online businesses need to have a solid handle on an extensive array of Web topics. This ranges from appreciating why plain old HTTP is so good at underpinning the Web to more sophisticated topics like modern application architecture, the latest in online user experiences, next generation computing models (grid/cloud/utility/SaaS/PaaS), cost-effective scalability, user identity, network effects, Jakob's Law, analytics, operations, user community, as well as the many compelling new distribution models that are nearly mandatory in the first release of most products.

This extensive set of competencies is what's required nowadays to deliver a credible online product to a receptive user base and it has dramatic implications for both uptake and overall cost/time-to-market. Worse, this body of knowledge has become extensive enough that many Web startups frequently fall far short of what they need to know in order to be successful with these far flung practice areas.

Does this complex body of knowledge mean the era of the two-to-five person Web startup is coming to a close? Not at all, at least not yet. The productivity level of the latest tools and techniques remains almost astonishing though the level of knowledge required of these teams is creeping up and up. And as we'll see, new models for product distribution are pushing the capability envelope of the typical Internet startup team to the point we may very well see the day soon that they won't have all the skills necessary to deliver a fully-scoped modern Web application. It is also one reason why fewer and fewer Web startups have the goods to be all around hits out of the gate.

Certainly, varying depths in subject matter are required depending one's exact role in a Web business, but Web-oriented products are fundamentally shaped the vagaries of the network itself. Tim O'Reilly himself still has the best quote on the subject: "Winners and losers will be designated by who figures out how to use the network." And as we'll see, the Web is driving the evolution of a major new generation of online distribution models.

Why Adopting New Distribution Models Is Crucial

As an example of this, I've been tracking some of the latest discussions around the hot topic du jour in the Web world: Social networking applications. Specifically, it's been interesting to watch the surprisingly low level of industry attention around the titanic competition brewing between social networking application formats from Web giants Facebook and Google. Why is this? Some might say it's because these applications still have largely unproven business models. Others, like Nick O'Neill at the Social Times recently observed (rightly in my opinion) that the struggle may have to do with a deficit in understanding why these new types of Web applications are so important. Nick notes that these widget and social networking style models for packaging and distributing Web apps often "have more eyeballs looking at their products than television channels have" and the challenge is that too many people just "don’t know what any of this means", despite the major players divvying up the online pie for themselves. With the size of these next generation distribution audiences, ignorance has an extremely painful price: failure to produce results and growth, poor engagement with the marketplace, and loss of market share.

An excellent summary of the truly massive, but largely underappreciated scale of these new Web application models was last week's TechCrunch piece on the progress of Google's OpenSocial, an increasingly successful model for creating portable social networking applications that will run on any OpenSocial-compliant site. Erick Schonfeld reported that OpenSocial now has a total reach of an astonishing 350 million users and it will soon be 500 million. There are over 4,500 OpenSocial apps today, a healthy number for the application format but a small drop in the bucket compared to the number of Web sites in the world. But the key is that these applications are integrated much deeper into the social fabric of an engaged audience, interjecting themselves into the daily personal and work habits of the "captive" users of social sites and even have access to the personal habits and data of users of these sites. Facebook's story is impressive as well with over 37,000 applications that have been installed over 700 million times.

And social networking applications are just one of many news ways that applications have to be packaged and distributed, yet far too many organizations persist in a very 1990s view of Web experiences, namely that Web sites themselves are the center of online product design. Many even think that some of these other new distribution models are interesting but not part of their core online product. Unfortunately, that's very much a parochial view in the present era. Federated applications, atomized content and functionality, 3rd party product ecosystems through open APIs, and much more are required to establish a strong and resilient network effect which fends off competitors that are themselves bringing these potent new competencies to bear.

In fact, one of the things we emphasize over and over again in our conference workshops and in Web 2.0 University is that having a Web site is usually the least interesting things about new products. Worse, it makes the customer have to find you amongst tens of millions of other sites. Instead, these new models tend to focus on going to the customer, instead of making them come to you which is a much harder proposition. This can instantly give you the ability to reach millions of potential people with dramatically lower effort and cost, as long as you have something interesting to offer.

Unfortunately, the number of capable practitioners of these new distribution models remains relatively small compared to the large body of experts in traditional Web product development. Demand is also low for these new skills as most organizations have been painfully slow to appreciate how much online product development has changed. A quick search of the job aggregator SimplyHired tells the tale: Nearly a thousand Web designer positions are available while only 36 OpenSocial and 40 open API positions are open, for example. This despite the the latter skills being able to project a product across the Web into hundreds of social sites or create an API that allows the product to be incorporated into countless other products for far less cost per customer than traditional methods.

The lesson here is that these new models still have a lot of fertile, unclaimed territory and many otherwise fierce competitors have not yet become fully aware of these new opportunities. Get your piece of the pie while there's still time.

I also find that the Web development industry has been slow to change, particularly outside the valley, and there is depressingly scarce information on how to deliver well on things like widgets, open APIs, social networking applications, and even syndication. To help with this, I've put together a short primer and some good references for those that want to get started.

Because the good news is that there remains tremendous opportunity for growth and success — for both startups and traditional businesses — if they will actively begin incorporating these new product delivery models into their own online capabilities.

Overview of Online Product Delivery Models

Web sites. This the classic model for Web presence. During the early Web, creating a Web site was just about the only option for engaging with those online (e-mail being the other.) Most early Web sites were used for publishing and not for user participation or peer production. These days, Web sites are still important, though by no means mandatory, and have their content syndicated via RSS and ATOM (pushing the content to where it's wanted), provide an access point to obtain widgets, and maintain user identity, and create communities of users. Upshot: They've evolved a lot but Web sites are only part of an extensive set of capabilities that must be brought to bear in the Web 2.0 era.
Syndication. It took ten years for the Web community to figure out a workable syndication model. Now RSS and ATOM are now the expected models used to distribute content off a single site and across the Web. Countless aggregation services now exist that make a site's information embedded in their services as well as a way to offer users a method for pulling information from a site and experienced in a means of their choosing, from Google Reader and Newsgator to the innovative Yahoo! Pipes. Most sites still heavily underutilize syndication even for notifications and pushing out frequently changed information to draw attention to it much less the strategic ecosystem and integration opportunities it affords.
Web 2.0 applications. You might argue that Web 2.0 itself is not a product distribution model but a set of design patterns and business models and that would be a true statement. However, in this context we're referring to the fact that Web 2.0 apps package up the 3rd major type of networked value: user participation. Before then, Web sites and syndication primarily had only centrally produced content or functionality that they could expose over the network and offer to the marketplace. In other words, user participation its purest form — sometimes known as peer production — ultimately results in products like Mechanical Turk and Predictify that provide direct networked access to user participation, but there are many fine gradations to this. The bottom line, Web 2.0 applications plug the user into the network like never before and are a critical rung in the distribution ladder since it offers access to the largest set of content and information by harnessing collective intelligence.
Open APIs and Web services. This is one of the most important long-term decisions most online businesses can make. Offering an open API lets anyone take the online components of a business, from its data and functional capabilities to the users themselves, and makes them open and accessible over the Web to be incorporated into other products and services, sometimes in the form of mashups and sometimes in the form of entire online products. Amazon, one of the first Web companies in existence and is hence far downrange in terms of the experience curve, has been using this distribution model with notable success recently. So have hundreds of others. The real challenge has been how foreign this model is to the original Web model and thus to the various management and development competencies in most organizations. It's much more an a way to OEM a product and leverage the customers and investments of hundreds of other partners. However, overall, it affords the potential for much larger business outcomes than could ever be created with point Web presence. It's now considered a significant oversight not to have an open API available for the typical online product.
Web widgets. Selecting parts of a Web site and it's data and packaging it up to make it run inside a portable, user distributable widget has been growing more and more popular over the last few years. For example, WidgetBox currently distributes 74,000 different kinds of Web widgets from its partners to over 1.2 million other sites. Widgets lets users distribute a Web site to other places on the Web at no extra cost and it also creates an ecosystem effect, where other Web sites users become the users of the new site. The YouTube badge is a notoriously well-known example of this that also helped drive the extraordinarily fast growth of the site. Like APIs, widgets are now considered a mandatory must-have for new and existing online products. But unlike APIs where it's up to the API users, figuring out users want out of your site's widgets is still an art form.
Social networking applications. Sometimes viewed as an extension of the Web widget model, social networking applications are applications designed to run inside of popular social networking environments and usually have capabilities that tap into and make use of the social graph information resident in a user's social network account. This is an amazingly fast moving field as you can see from a recent post on the latest happenings on the OpenSocial blog, to the extent it's hard even for well-funded companies to keep up. However, despite skepticism that large businesses can be built exclusively through a social networking application, it's become ever more essential for a site to make its capabilities accessible usable in these environments. Not only will users help distribute online products in these formats to their contacts but it also increases the overall usage of the your application including participation and its consequently growth of a site's network effect. While not yet considered mandatory for online products, the ease with which these social network applications can be created and the large numbers of users they make available makes it a smart distribution option for most Web businesses. Like widgets, however, figuring out what users will find engaging in a social networking application featuring your online product takes some research and experimentation. However, the results can be very rewarding and some social networking applications have millions of daily users. See the Plaxo Pulse story on Mashable for the details of how OpenSocial drove a 5x improvement in traffic in only 3 weeks.
Semantic Web and Web 3.0. The Semantic Web, one of the original visions for the World Wide Web, has taken a while to arrive but it's beginning to look like it may hit critical mass in the next 12-24 months. Combined with Web 3.0, which takes the architectures of participation at the core of Web 2.0 and drives it through a lens of Semantic Web capabilities. The benefits can be profound and can greatly increase the value and leverage of information on the Web. While this is very much not prime time yet, unlike #1-#6 above, it likely will be and smart organizations can get ahead of the learning curve and get an early market lead using these techniques. For now, however, I recommend that most organizations focus on executing well on the first six items before tackling this and waiting for the technologies to finish emerging and maturing.

The list above should provide good guidance for starting move into the potent new models for distribution on the Web. I'm seeing, however, that because of the major shifts in strategy and product design emphasis these techniques demand, most organizations take an inordinately long amount of time to become effective with them. The lesson here: Start small now and build core competency. Small investments now can pay off later in terms of valuable experience made from early experiments and pilots. When done right,
these new distribution models can become the dominant channels that the world uses to interact with your business, like they already have with Amazon and Twitter.

I'll be talking about these and other strategic online product design topics in my upcoming Building Next Generation Web Apps Workshop at the inaugural Web 2.0 Expo 2008 NYC next month. I'll have more details about this deep-dive session in an upcoming post.

Filed under Network Effects, Social Business, Social Media, Social Technology & Standards, Strategy, The Web Tagged with distribution models, Jakob's Law, Semantic Web, Web 2.0

The 6 Essential Things You Need To Know About Google’s OpenSocial

November 2, 2007 2 Comments

I've spent the last few days keeping track of the seemingly endless stream of news and blog coverage about Google's new OpenSocial model for social networking applications. OpenSocial has been described by some as Google's industry "chess move" to outmaneuver and corner Facebook. This is fascinating set of developments to watch since Google's own growing social networking platform, Orkut, was eclipsed by Facebook in terms of overall traffic back in September.

Unless you've been hiding under a rock lately, you know that Facebook is presently the industry darling in social networking, having largely pushed MySpace off the industry's stage, as it seems to offer a more compelling model for social interaction to users overall. Just as importantly, Facebook also lets any other company that wants to join in party do so by building 3rd party Facebook applications, of which over 7,100 now exist, making Facebook increasingly rich in functionality and content by leveraging the creative capacity at the edge of the Web. In the Web 2.0 era (and in all computing eras before), the central truism is that a platform beats an application every time. This applies here with a vengeance and MySpace and other social networking sites have suddenly rushed to embrace openness and 3rd party widgets and gadgets to such an extent that MySpace has thrown in with Google on OpenSocial.

So the damage is done and in the fickle world of online social networking, Facebook currently has the upper hand. This demonstrates yet again a powerful but counterintuitive aspect of networked software: the more control you give away, the more value you can get back.

Read my ZDNet coverage on how Facebook got ready to overtake MySpace and the challenges of setting up shop inside in Facebook.

However, much of the blogging around OpenSocial would have you believe that has Google now trounced the competition with a strategic move that counters Facebook's open SNS platform move with an open SNS application model that can work everywhere else too. At least, that is, the other social networking sites that support OpenSocial's API.

But as Don Dodge noted in his OpenSocial coverage this isn't going to stop developers from building apps natively for Facebook any time soon and will have little practical effect on existing Facebook users for quite a while. Not to mention the rest of the Web, since not even a single real OpenSocial application yet exists.

That's not to say however that OpenSocial doesn't have its advantages. Joe Kraus, a Director of Product Management at Google, wrote today on the Official Google blog that OpenSocial will make life easier for developers "because it makes it easier for them to focus on making their web apps better; they get lots of distribution with a lot less work. It's good for websites, because they can tap into the creativity of the largest possible developer community (and no longer have to compete with one another for developer attention). And finally, it's good for users, because they get more applications in more places."

So, despite the early beginnings, does OpenSocial make sense from the production side of social networking applications? It still remains to be seen, despite the enormous amount of early partner support for it, if the consumption side in terms of these kinds of applications really generates value. Most of the applications on Facebook provide so little actual utility that they are barely worth installing. While making these mini-apps portable between social networking sites is convenient — and it probably will appreciably increase the total number of available social applications — it's really people and the network effect they represent for a given social networking site that makes the site truly valuable. In other words, if my friends and colleagues aren't on the social networking site I use, then that site is of little or no use to me, even if I can take my apps with me.

It'll be interesting to see what ultimately happens to OpenSocial. I suspect it will actually see fairly good uptake since it's based on the highly successful Google Gadgets model, for which over 23,000 different Gadgets presently exist. But will it change the playing field in the social networking wars? Probably not as much as a federated social identity would. Federated social identity could potentially let you exist and participate simultaneously in all the social networks you wanted to at once using one set of social metadata you control. That's probably a lot closer to the Facebook killer that so many are looking for and things like openid are bring that world closer to reality all the time.

In the meantime, here's the six things you absolutely have to know about OpenSocial to have an opinion about it:

6 Essential Things You Need To Know About Google's OpenSocial

OpenSocial only offers the lowest common denominator, not the full richness of each social networking platform. While application developers can create apps using the OpenSocial model and they will be able to run on dozens of different social networking sites, OpenSocial can't help you leverage the full capabilities of the site it runs on. Social networking site APIs aren't anywhere as complex as say, the Windows APIs, but we've seen this before with platforms such as Java, where the development model can't support the full capabilities of the underlying operating systems. Like Java, write once, test everywhere is the name of the game for OpenSocial and while economies in this model certainly exist, a single universal widget model tends to discourage product differentiation in favor of broad distribution. This means to get at the full richness of the underlying platform and create a competitive product, you have do custom coding for that site and you've just broken the reason to use a common application model.
OpenSocial is largely based on open standards and there's only minor developer lock-in. Overall, it actually seems pretty safe to do a lot of your social application development using OpenSocial. It uses the essential browser open standards of XML, HTML, Javascript, and the data formats are all ATOM and RESTful/WOA. You can even host Flash content and functionality inside the OpenSocial application as long as you don't break the rules. Finally, most of the really popular development platforms, including Ruby on Rails, can support the server-side API. All in all, Google seems to have stuck to a fairly open and non-proprietary model including avoiding crufty proprietary markup. OpenSocial documentation and sample code all uses the Creative Commons licensing and Apache 2.0, and the OpenSocial FAQ says everything will be open sourced at some point. Kudos for this open stance, Google.
OpenSocial is a real doorway to social networking data portability as well as potential security holes. A site that supports OpenSocial applications provides that application with all the people data in that user's account. Their own info as well as their friends. This can be used to export user's social data from sites that don't support themselves directly and it could even be used to knit together a person's social data across other social sites that support OpenSocial, with properly designed 3rd party apps. But it also opens the door to security problems and expect to see that security, cross-site scripting, and exploits become an issue over time, as it always does when platforms open up to the rest of the world. Update: Michael Arrington has reported that the first OpenSocial app has now been hacked.
OpenSocial is simple and straightforward but also capable of developing full-blown, rich Internet applications. And without server-side infrastructure. Developers can simply innovate with a few bits of markup and procedural code and drop it into the OpenSocial ecosystem and leverage the massive audiences and scalable infrastructure of OpenSocial compliant sites. OpenSocial even supports powerful interactive Web user interface models like Ajax explicitly. Like we saw last year, with the new productivity-oriented Web development platforms, this will change what's possible while also creating mountains and mountains of relatively useless, uninteresting apps amongst a few real gems. But a lot more wildflowers will bloom on the OpenSocial landscape and some will likely rise up and show us how useful these applications can be.
OpenSocial is from Google and excessive philanthropy should not be expected. Google almost certainly thinks OpenSocial will ultimately be very good for Google, if not outright bad for a few others (probably Facebook). While the openness is encouraging, if OpenSocial is successful, Google has a plan to make that success work for it. Those plans may not always be to the benefit of everyone playing under the OpenSocial umbrella. User beware.
A new era in competency in social software is being ushered in by models like OpenSocial. A lot more social applications are being created because of open social platforms have become so popular. But building successful social applications is a lot different prospect from building traditional business and consumer applications. Expect that many developers and software designers will fail to build applications successfully until we learn that a different focus and way of thinking is required. I've written before about the basic rules for building good social applications, but these are just the beginning. Understanding people is the key to building effective social networking applications, and that is often the hardest thing for us in an industry obsessed with connecting with each other via 1s and 0s.

What else do we need to know about Google's OpenSocial? Put your ideas in comments below or drop me a line at dion@hinchcliffeandco.com.

Going to Web 2.0 Expo Berlin? I'll be there November 5th and 6th giving two sessions (What is Web 2.0 and The Rise of Widgets) as well as on the show floor at the Reply booth, our European partners for Web 2.0 University.

Filed under Social Media, Social Technology & Standards, The Web

A Timeless Way of Building Software

January 10, 2006 Leave a comment

Most of my readers know that I’m a software architect by trade. I’ve been creating software large and small for over twenty years. And I’ve experienced movement after movement in software design from object-orientation in the 1980s and early 90s to component-based design, distributed objects, Web-based software, service-oriented architecture and too many others to even mention. I’m pretty jaded at this point because I’ve learned, in general, the problems that you couldn’t solve in the previous generation of technique are often only marginally more solvable in the next generation (which is invariably invented to "fix" the previous problems.)

Alas, a genuinely better mousetrap is really hard to find.

So in the end, if you couldn’t do whatever it is you wanted to do with the previous generation of technique, it’s actually not that likely you’ll succeed in the next. Certain software problems remain hard, and in general, it mysteriously happens to involve the juncture between technology and people in some way. To paraphrase this, I could say that the software and techniques get better fairly constantly, but people remain the same.

And please, bear with me because I will attempt to connect a great many dots in a moment.

Because every once in a long while, something new and big actually does come along. Or at least something that looks new and big. One of the new and big things that came along about ten years ago was the concept of design patterns. It was pretty neat stuff. It said that despite the current technology we have, the processes that continue to evolve, there are certain timeless solutions to certain software design problems. It was a revelation at the time. And the writers of the book that explained this got both famous and very successful. Why? Because these design patterns really worked is why. And anyone who has read the books and has ever really built software recognizes these patterns. And what was strange was that no one really expected it. One day, we just had them. And the kicker was, they were always there, but now they were in our conscious thought and we had real names for them. My point: They were in our face all the time but most of us couldn’t see them.

We are in a similar place with the Web right now. We’ve done this Web stuff enough now that we are just beginning to see the design patterns. What works, and why, in a specific situations, bounded by forces. Some folks have had the hubris to give this next generation a name and to tease out these patterns. Some people are just now going aha, and some people haven’t got it yet, and most of the rest of us either aren’t ready for it or just haven’t heard of it. But, I will tell you this. It’s quite real. The best practices and design patterns of Web software are just starting to become understood. The strange part is, we’re discovering the same things over again. What’s old is new again.

Now, before you get all worked up or worse, I bore you and you stop reading, I will give you a nice list of the the forces behind these patterns. If you recall, design patterns are a solution to a problem in context. We are starting to get the context and even the outlines of the patterns of this "new" generation of software. But we have a long way to go still. The Web is a monstrously big space with big problems, and it’s not getting better. There are one billion of us out here now. Clearly understanding what it takes to create great software on the Web that is successful, useful, and vibrant will be an ongoing challenge for a long time. But it will get easier because we are codifying our knowledge of this exciting and huge place where we now find ourselves.

Comparing SOA, Web 2.0, and a Timeless Way of Building Software

Figure 1: The driving forces in modern software.
With a rough comparison between SOA
and The Timeless Way (Web 2.0 by any other name).

So, here is where we take a quantum leap to the next level of thinking. I’m going to use Christopher Alexander’s opening chapter of a Timeless Way of Building and tailor it to describe this old-but-new way of building the Web and software for it. We are lacking for a little inspiration and this book in particular continues to sell upwards of 10,000 copies a year, 25 years after it was frst published. And Christopher Alexander, for those of you who may not know, was the person that originally discovered the design pattern. But it wasn’t for software. It was for creating great, timeless buildings. He was one of the first that realized that his field of endeavor has certain elemental, timeless cores, no matter the technique, building material, or the people. It was an amazing discovery that poured over into the software world with considerable success.

My assertion is that nothing has really changed in software, we might understand the forces better but they are almost always the same. People want software that does what they want, is available when they need it. They want software that grows with them, helps them, teaches them, and lets them do the same with others. They want software that gets out of their way, disappears, and is more convenient by far than inconvenient. And they want to pay as little as possible for it, but enough so that it’s worth it. They are willing to have software get right into the middle of their lives. If it’s the right software. And as long as we’ve had software, they’ve always wanted this. But now they might actually start getting it.

In any case, I don’t literally believe every phrase in this take-off, but I do believe the overall concept deeply and profoundly as a software professional. And I will continue to update the diagram above (clearly marked beta 1) until we have more of the forces in it. And some are definitely missing. Please, as always, leave any comments and suggestions for improvement below.

And now, without further ado, here is the The Timeless Way of Building Software, with sincere apologies to Christopher Alexander:

The Timeless Way of Building Software

Inspiration For The Next Generation of Web Software

There is one timeless way of building software. It is decades old and is the same today as it’s always been. And because it is timeless, it will always remain this way.

The great software of our time has always been created by people who were close to this way. It isn’t possible to create great software – software that is satisfying, and useful, and makes itself a natural extension of life – except by following this way. And as you will see, this way will lead anyone who looks for it to elegant, vibrant software which is itself timeless in its form.

It is the process by which the function of a piece of software grows directly from the inner nature of people and naturally out of the raw bits, the otherwise meaningless digital medium, of which it is made.

It is a process which allows the life inside a person, or a group of people, or a community to flourish, openly, in freedom, so vividly that it gives rise, of its own accord, to the natural order which is needed to be contained within it.

It is so powerful and fundamental that with its help you can create software that is as beautiful and enriching as anything else you have ever seen.

Once you understand this way, you yourself will be able to create software that is alive, that is intertwined comfortably with your life and the lives of others. You will design worlds where you and others will want to work, play, and co-exist together; beautiful places where you can sit and dream comfortably.

This way is so powerful, that with its help hundreds or thousands, or even hundreds of thousands of people, can come together together to create software and community which is as alive and vibrant, peaceful and relaxed, as any living experience has ever been.

Without the central control of authorities and experts, if you are working in this timeless way, a genuine place will grow right from underneath your fingertips, as steady as the grass in the fields or the trees in your backyard.

And there is no other way in which a software which is fundamentally good can possibly be made.

That doesn’t mean that all ways of making software are identical. Quite the contrary. It means that at the core of all successful software and at the core of all successful processes of creation and evolution, there is one fundamental invariant feature which is responsible for their success. Although this way has taken on a thousand different forms at different times, in different places, still, there is an unavoidable, invariant core to all of them.

Take a look at the some of the great Web software like Google’s search page, Flickr or del.icio.us. They all have that unique, yet unhurried, grace which comes from perfect ease and natural balance. But what is it they have in common exactly? They are beautiful, ordered, harmonious – yes, all of these things. But especially, and what strikes to the heart, they live.

Each one of us yearns to be able to bring something to life like this. Or just be a part of it somehow.

It is a fundamental human instinct, as much a part of our desire as the desire to be part of something greater than ourselves. It is, quite simply, the desire to make a part of nature, to complete a world which is already made of mountains, streams, stones, buildings, ourselves, our living systems, and our increasing connectedness together.

Each one of us has, somewhere in our heart, the dream to make a living world, a universe, and place of our own for us to share with others.

Those of us who have trained as software designers have this desire perhaps at the very center of our lives; that one day, somewhere, somehow, we shall build a software experience which is wonderful, beautiful, and breathtaking; a place where people can go and live their dreams.

In some form, every person has some version of this dream; whoever you are, you may have the dream of one day creating a most beautiful place, virtual or otherwise, where you can come together with others and freely share your knowledge, learn, participate in your community or government, and otherwise conduct your daily interaction with the rest of the world.

In some less clear fashion, anyone who is concerned with communities and other large group efforts has this same dream, perhaps for the entire world.

And there is a way that software can actually be brought to life like this.

There is a definable sequence of activities which are the heart of all acts of software design, and it is possible to specify, precisely, under way conditions these activities will generate software which is alive. All this can be made so explicit that anyone can do it.

And just so, the process by which a group of independent people can make software become alive and create a place as real as any other can equally be made precise. Again, there is a definable sequence of activities, more complex in this case, which are the heart of all collective processes of software creation. And it is also possible to specify exactly when these processes will bring things to life. And once again, these processes can be made so explicit, and so clear, that any group of people can make use of them.

This process is behind the design of community built software like Linux, Apache, Wikipedia, and many others. It was behind the design of the great virtual places for people to live and work: the Internet, Usenet, and the World Wide Web. It was behind the creation of simple, satisfying software of the kind that powers the iPod, the Blackberry, and Firefox; of SourceForge, Wikipedia, and BitTorrent. In an unconscious form, this way has been behind almost all ways of creating software since the beginning.

But it has become possible to identify it, only now, by going to a level of analysis which is deep enough to show what is invariant in all of the different versions of this way.

This hinges on a form of representation which reveals all possible design processes, as versions of one most fundamental set of patterns.

First, we have a way of looking at the ultimate constituents of the environment: the ultimate "things" which a piece of software is made of. As we shall see, every piece of software is made of certain fundamental entities known as design patterns; and once we understand software in terms of its patterns, we have a way of looking at them, which makes all software, all of their parts and function, all members of the same class of thing.

Second, we have a way of understanding the generative processes which give rise to these patterns: in short, the source from which the ultimate constituents of software come. These patterns tend to come from certain combinatory processes, which are different in the specific patterns that they generate, but always similar in their overall structure, and in the way they work. They are essentially like languages. And again, in terms of these pattern languages, all the different way of building software, although different in detail, become similar in general outline.

At this level of analysis, we can compare many different software creation processes.

Then, once we see their differences clearly, it becomes possible to define the difference between those processes which make software vibrant, alive, and useful, and those which make them the opposite.

And it turns out that, invariant, behind all processes which allow us to make great software, there is a single common process.

This single idea is operational and precise. It is not merely a vague idea, or a class of processes which we can understand: it is concrete enough and specific enough, so that it functions practically. It gives us the power to make software and virtual communities live, as concrete as a match gives us the power to make flame. It is a method of a discipline, which teaches us precisely what we have to do make our software what we want it to be.

But though this method is precise, it cannot be used mechanically.

The fact is, that even when we have seen deep into the processes by which it is possible to make software alive, in the end, it turns out this knowledge only brings us back to that part of ourselves which is forgotten. Although the process is precise, and can be defined in exact scientific terms, finally it becomes valuable, not so much because it shows us things which we don’t know (though it may do that as well), but instead, because it shows us what we know already.

Of course, this way of building software has never be named. It’s not service-oriented architecture, or the personal software process, or agile methodology, or the unified process, or CMM, or any of the others. It’s the actual things that are conceived and done and worried about when software is created and used. For now, because all software is quickly becoming connected to all other software, and because the Web is becoming the place where more and more of the relevant software is, and finally because it is a more complete reconception of what we thought we knew, we’ll give it a name temporarily. An unsatisfying name, but one that we can remember for now.

We will call it Web 2.0.

What do you think? Are we at a place where we can really identify the design patterns in Web 2.0?

Filed under Enterprise Architecture, Social Technology & Standards, Strategy Tagged with "design patterns", "Timeless Way of Building", architecture, Christopher Alexander, complexity, components, framework, granularity, IT, library, models, openness, participation, patterns, practices, SOA, software architecture, standards, Web 2.0, web services

RSS is the Web 2.0 “Pipe”

November 11, 2005 1 Comment

Problematically, RSS is still not quite a household word yet, and even the software industry is just beginning to realize the importance of this workhorse syndication format. Though at this point it’s already clear RSS will be the key enabler of Web 2.0 and Software As a Service. It will do this both as a notification system and as the actual glue that will eventually hold many Web 2.0 services and mash-ups together.

Dave Winer famously created the current incarnation of RSS but its implications are still rippling through the industry. The folks that can fully appreciate RSS will reap corresponding rewards. Microsoft CTO’s Ray Ozzie is a good example of the folks that "get" the significance of RSS. I love the quote from his much-discussed leaked memo this week and I haven’t been able to stop using it for the last day: "[RSS] is filling a role as the UNIX pipe of the internet as people use it to connect data and systems in unanticipated ways."

And we can’t forget that RSS feeds, storage, synchronization will be a central new feature of the next version of Windows.

So expect to see RSS on every blog, every Web 2.0 service, web site, and piece of desktop software going forward. If you can’t find the feed, you can be sure whatever it is won’t last long.

And for the fully buzzword compliant and for the record, I do fundamentally believe REST/RSS is the new EAI. And the glue of first choice for lightweight SOA as well. And I’m actively starting to see some folks drop their traditional Web services and go RSS wholesale.

Of course, the real problem right now is that most people on the Web still don’t have any idea what RSS is. At best, the average Web user might understand that RSS forms some kind of information "feed". More sophisticated users notice that if they can find an RSS link somewhere (a blog or news site for example) that they can use it like a URL to get updates of information within services like My Yahoo, Bloglines, or something called an RSS reader.

Murkiness and partial examples are the enemy here. Raising awareness and finding clear examples that fully express the potential and power of RSS should be the goal.

Here are some clear, canonical examples that I think convey the full scope of what RSS does for us in a Web 2.0 world:

RSS Use Cases

Notification: Need to inform a lot of people about changes to information? Don’t want central control? Want to enable self-service? Use RSS.

Syndication: Publishing new information regularly? Put it into an RSS feed. This flows out to the world your blog entries, news articles, podcasts, videos, job posts, weather reports, financial updates, bug reports, etc. The software you use should be able to take your information and make it into an RSS feed. If your current software can’t, find new software. It’s that important.

Glue: Need to connect any service to another service on the Web (or anywhere else)? Trying to mash together data? Building supply chains? There is generally no need to ever ask anyone to stand up a new web service. Pull everything you need via its RSS feed. Some software developers will disagree with this and say there are better methods, but to this I point out: 1) RSS is robust enough that it’s all you’ll ever need nine times out of ten and 2) it’s what you’re going to offered automatically anyway, take it and get something done.

RSS creates the Web 2.0 information ecosystem by enabling interconnectedness, network effects, emergent behavior, and much more as well. And RSS doesn’t demand control of the other end of the conversation. This is a big enabler all by itself and is a classic Web 2.0 force. By letting consumers of RSS use any tool or service they want on their side, barriers are eliminated and connectivity is encouraged.

That doesn’t mean that RSS doesn’t have its weaknesses either and certainly there are other ways to create feeds, but RSS has the mindshare, support, and the goods right now. So though it’s not perfect, it’s more than up to the job. Let’s spread the word…

What did I miss? What other canonical examples are there?

Technorati: web2.0, rss

Filed under Social Technology & Standards, The Web, Web 2.0

← Older posts