Intelligent Enterprise

David Weinberger, author of Everything is Miscellaneous, spoke at last week's FASTforward conference (ostensibly the user conference for FAST Search & Transfer) about the power of digital disorder, and how we need to unlearn what we think that we know about the best ways to organize information. He feels that we're approaching the end of the age of information — by which he means a focus on rigidly structured information — and a move away from being "informationalized," where we consider everything to be information even if they're just symbolic representations of reality.

He looked at how many projects, typically physical projects, require a much greater degree of control as they increase in size, but contrasts that with the web, which has growth only because of the lack of control. Control doesn't scale; we just thought that it did, and managed to scale with control by eliminating information.

There are two orders for how things are traditionally organized: in the case of physical objects, there's the first order, namely, the physical order in which they are placed; and the second order, which is a catalog of the physical objects, like a card catalog in a library, which uses a limited and pre-determined taxonomy for categorizing the objects.

In the new world of organization, we allow for user-generated folksonomies to generate metadata that we couldn't have possibly thought of for the objects. These multiple layers of metadata online allow for a physical object to virtually be in more than one place at a time; when the concepts are applied to online data rather than physical objects, the division between data and metadata starts to blur. All contents are also connections: everything leads to everything else, creating a wonderfully messy mass of interconnected data.

The Web, of course, excels at creating connections because of the basic premise of linking: we create hypertext links on pages to make connections that are important to us. The user revolution, therefore, is not just about us creating our own content; we also control the links, hence control the connections between content and the organization of that content. Digg, Twitter, your RSS feeds and other socially-created sites create our new "front page," replacing the newspaper of old: why would you read someone else's idea of what's important, rather than self-select what you're interested in reading?

A key thing is not to set arbitrary limits; he cites the example of the Library of Congress recently posting a huge set of historical photos on Flickr to allow open tagging by everyone, which has since run up against Flickr's arbitrary limit of 75 tags per photo. Tagging like this helps to destroy stereotypes: the more data/metadata that you have about someone or something, the less likely you are to make a shallow characterization.

Weinberger's a great speaker, and uses some really funny and compelling images in his slides.