Friday, October 12, 2007

Major enterprise search development--Vivisimo & social search

Enterprise search vendor Vivisimo announced this week that they will be including a social collaborative component with the next version ("6.0") of their software. Lynda Moulton blogged about it; she is presenting next week at both a Vivisimo-sponsored webinar and also a more private event that I will be attending (disclosure: I may get a free lunch out of the deal). I have heard Lynda speak in the past and have generally been impressed with her depth of knowledge about search.

Accessing Group Smarts Through Tagging and Ranking

Providing some social context, through user tagging, shared user search result ranking, and exposure of other's favorites, has great potential to enhance findability and context inside the firewall, and I think 6.0 is a major development. I have previously posted an introduction to tagging, pivoting in Del.icio.us, and on CommonCraft's "howtoon" video introduction to social tagging.

The leading example of leveraging mass opinion to enhance findability that I know of outside the firewall is Amazon. Amazon uses not just these three methods of social collaboration, but also now has designated key reviewers, user blogs, and much more, such as listing "what do customers ultimately buy after viewing this item?" But the key remains other people's reviews, which let you know if what you are considering buying is any good, and other people's tags, which help you find the products. It is no accident that other major evendors such as Tiger and even brick-and-mortar powerhouses like Sears have jumped on the customer review/ranking bandwagon. It simply works, for the users and the companies.

I've been reading legal scholar Cass Sunstein's Infotopia: How Many Minds Produce Knowledge, which provides a clear explanation for the effect of "group wisdom" on decision making. He is quite hopeful, though not Pollyannish, on our ability to pool information. If people on average are more than 50% likely to rank and tag accurately (substitute "answer a question right" or "make a decision" as needed), then with a large group, ranking will tend to dramatically enhance accuracy, that is, the group will almost always identify "the right answer" or the most relevant page or document for the particular search. The outlier case is the converse; if users are not likely to be accurate (as when asked to guess the distance to the moon, or who will win the next Federal Circuit decision on obviousness leading to patent invalidity), then a large group will be even less likely than an individual to be accurate.

The more people and the more content you throw at it, the better group intelligence will do. By contrast, interpersonal collaboration and intrafirm internal communication becomes much harder the larger (and more geographically scattered) the enterprise gets. Tagging, like other enterprise 2.0 tools, has a way of making the enterprise feel smaller.

Other Social Software Inside The Firewall Vendors

There was some fairly insightful coverage of Vivisimo's announcement in eWeek comparing Vivisimo's offering with a social collaboration module introduced by guided navigation pioneer Endeca. From a professional services perspective, I think it compares more closely with Connectbeam. Connectbeam provides social collaboration inside the firewall, organized around private, group, and company "topics," but does not itself have a search component; their publicity materials indicate, however, that they have connectors and have integrated with "major enterprise search engines" (by which they seem to mean Google and Fast.) Honeywell apparently implemented Connectbeam with Google Enterprise in March 2007.

There was also coverage of 6.0 in Information Week.

Another enterprise social software vendor, this one purely on the "tagging" side, is Cogenz. They have an excellent demo that serves as a good introduction to tagging (see also my previous post on Del.icio.us, a free web tagging service that can be integrated with Cogenz' product) I especially liked their promotion of tagging as the way to "tap into the collective intelligence of an organization by collecting, sharing and connecting around unstructured information."

One question is whether these tools have a connector or can readily be integrated with Microsoft's Sharepoint 2007 intranet / DMS search. Many firms already have it, it's free, and some firms (like Sheppard Mullin) have used it as the basis for intranet searches.

Features of 6.0?

I'll be curious to see if, like Connectbeam, "6.0" will allow users to pivot on other users' tags and favorites, as well as pivot around others' tags. Will it be possible to rank or segregate people's tags on documents based on their role within an organization? A litigation partner might only want to look at sample complaints or settlement agreements that have been used or endorsed by another such person.

I will also look for how 6.0 manages the tension between the organization wanting control over what the tags might be (and perhaps to drive them to certain content and away from other content) and the necessity to free up people to make up tags that make sense to them. I've noticed with my Delicious tags that it is all too easy to add a comma or use a slightly different word and thereby miss much of the context that tagging can provide. Cogenz handles this tension by predicting and displaying others' tags based on what you have started typing. I'm curious if more popular tags will be displayed first.

No comments: