Continued elsewhere

I've decided to abandon this blog in favor of a newer, more experimental hypertext form of writing. Come over and see the new place.

Saturday, June 16, 2007

The Aleph and the Knowledge Worker

or, The nature of knowledge in the wired world.

[warning -- formless braindump]

Borges' story The Aleph tells of a point at which a person can view the entire state of the world, everything concentrated and simultaneous. Typically, it's a metaphysical horror story with uncanny resonances in both present reality and the future. The Aleph was published in 1949, four years after Vannevar Bush's seminal article As We May Think, which described the Memex, the postwar technocrat's version of The Aleph as a realizable device. Did Borges read Bush, or just intuit a zeitgeist that was blowing through both of them?

Bush's article is an amazing mix of insightful prophecy and some laughably wrong technical predictions (ie, that storage would involve photographic processes). But mostly he got it right. What is really startling, however, is that despite the fact that we have information systems orders of magnitude more capable than he dreamed of, the problem still remains:

There is a growing mountain of research. But there is increased evidence that we are being bogged down today as specialization extends. The investigator is staggered by the findings and conclusions of thousands of other workers' conclusions which he cannot find time to grasp, much less to remember, as they appear. Yet specialization becomes increasingly necessary for progress, and the effort to bridge between disciplines is correspondingly superficial.


He anticipated the primacy of search:

The prime action of use is selection, and here we are halting indeed. There may be millions of fine thoughts, and the account of the experience on which they are based, all encased within stone walls of acceptable architectural form; but if the scholar can get at only one a week by diligent search, his syntheses are not likely to keep up with the current scene.

Selection, in this broad sense, is a stone adze in the hands of a cabinetmaker. Yet, in a narrow sense and in other areas, something has already been done mechanically on selection. The personnel officer of a factory drops a stack of a few thousand employee cards into a selecting machine, sets a code in accordance with an established convention, and produces in a short time a list of all employees who live in Trenton and know Spanish.
And user interfaces:
One can consider rapid selection of this form, and distant projection for other purposes. To be able to key one sheet of a million before an operator in a second or two, with the possibility of then adding notes thereto, is suggestive in many ways. It might even be of use in libraries... One might, for example, speak to a microphone, in the manner described in connection with the speech controlled typewriter, and thus make his selections. It would certainly beat the usual file clerk.

And blogging and search trails and annotation (which the web has still not quite got right):


It affords an immediate step, however, to associative indexing, the basic idea of which is a provision whereby any item may be caused at will to select immediately and automatically another. This is the essential feature of the memex. The process of tying two items together is the important thing.

When the user is building a trail, he names it, inserts the name in his code book, and taps it out on his keyboard. Before him are the two items to be joined, projected onto adjacent viewing positions... The user taps a single key, and the items are permanently joined..

Thereafter, at any time, when one of these items is in view, the other can be instantly recalled merely by tapping a button below the corresponding code space. Moreover, when numerous items have been thus joined together to form a trail, they can be reviewed in turn, rapidly or slowly, by deflecting a lever like that used for turning the pages of a book. It is exactly as though the physical items had been gathered together from widely separated sources and bound together to form a new book. It is more than this, for any item can be joined into numerous trails.


So. Now we are at a point where we have huge amounts of information at our fingertips, and reasonably good ways to search for it, and crude but effective ways to link it together. The combination of internet standards, high-speed access, Google and open access scientific publications has made a new world. We should be in knowledge paradise!

But that's not how it feels. The basic human problem of how to deal with information hasn't gone away, it's just been raised to the nth power. Finding documents is easy, deciding which are worth reading, and in how much detail, is as difficult as ever. Where should effort be focused, and how can one narrow down a global curiosity into a manageable subset?

Look at the typical modern knowledge worker, trying desperately to keep track of all the things they want to know, or ought to know. Maybe you aren't in this boat, but I am -- comes of too much intellectual curiosity. My RSS reader has 300 feeds! Categorized variously -- I read the politcs blogs mostly for entertainment (in that they don't generally spur me to action), the philosophic ones for ideas, the tech ones mostly out of obligation.

Even within tech, it's a lost cause. There are about five different programming languages I'm involved with right now -- do I want to keep up with current developments in them? And what about all the components and toolkits; what's going on with Prototyup or Scriptaculous? Or Lisp, Ruby, Python? Then there is the entire universe of Java language, components, and tools like Eclipse, which is a whole sub-universe unto itself.

There's no way I can be an expert in all of this stuff. Can I be an expert in finding out just the right piece of knowledge i need? Well, there's where Google comes in handy. I've had pretty good luck, the last six months or so, googling for answers to obscure or not-so-obscure tech questions. A couple of issues though:

It gives a big advantage to tools with a large user community. For instance, I've had occasion both to use Oracle and the very nice Virtuoso OpenLink, an open-source database/semantic web platform/middleware/kitchen sink. THe problem is, hardly anyone is uing Virtuoso so my stupid questions do not have a stupid answer available wit a quick Google. Instead, I need to mail the developers and maybe they get back to me in a day or so, or maybe not.

So where was I? Oh yeah, trying to think about Google and Borges and Memexes and focus and what it means to know everything...got distracted...

The point is it's impossible to know everything, and not even that useful. But what should an informational omnivore know? How to use Google effectively, sure....but there seems to be a deeper idea lurking somewhere in there.

The nature of knowing is going to be different in the future. I was trying to get at this when I coined the term "googlectual" -- as search gets incorporated into our thinking, the knowledge in the digital sphere becomes a part of what we know -- sort of. We lack good metaphors and ways for thinking about the relationship between knowledge-in-the-head and knowledge-in-the-world and the increasingly tight coupling between them.

Alright, I'll stop now.

Wednesday, June 13, 2007

Worst company names

Licketyship. A guy from there called my cell phone today for something, the connection wasn't so great, so I spent most of the call trying to figure out what sort of business model could involve coprophagy. Then there's the classic expertsexchange.com.

Sunday, June 03, 2007

The grain of sand at the core of the wikipedia pearl

Holy shit, Jimmy Wales (Wikipedia founder and guru) is an objectivist. That is just weird on multiple levels. For one thing, it's another example of a right-libertarian being in the vanguard of what amounts to a left-libertarian vision. When I was at MIT the objectivists used to quite literally worship the $ sign, now we have one whose slogan is "free knowledge for free minds".

For another, the philosophy of wikipedia seems determinedly anti-objectivist, at least superficially. They disdain expertise and seem to expect that truth will bubble up from a crowd-based distributed process. This egalitarian attitude seems very un-Randian to me, although who knows, maybe it is objectivist on a deeper level.

Given that Wikipedia seems to have become the new default authoritative source of knowledge, it's guiding philosophy and structures of governance become exceedingly interesting and important. The fact that it's run by someone under the influence of a nuttily simplistic ideology is a wee bit disturbing. On the other hand, Wales does not seem to have the virulently rabid form of the objectivist meme, and it certainly hasn't interfered with the success of his efforts.

Monday, May 28, 2007

Word and concept of the day: agalmics

Another person tries to make a compact characterization of the economics of open-source and other freely copyable goods: (via Notional Slurry)


agalmics (uh-GAL-miks), n. [Gr. "agalma", "a pleasing gift"]
The study and practice of the production and allocation of non-scarce goods.

...
agalmia, n.
The sum of the agalmic activity in a particular region or sphere. Analogous to an "economy" in economic theory.


My own particular interest is in how open source economics interfaces with the "normal" economics of scarcity -- ie, while it's wonderful that software can be given away for free, until potatoes can reproduce themselves as easily it will be problematic for people working on open source to feed themselves. Back in the embryonic days of the FSF I made this argument, but nobody paid me much attention, and as it turns out my objections, while valid, did not stop open source from taking over the world. And many people seem to be able to support themselves while working on free software, one way or another.

I still suspect there is something screwy about the economics, and I'm not the only one. It seems like programmers are collectively undermining their own value in the for-pay economy, to the delight of big service corporations like IBM.

what we're seeing playing out among coders is what I'll term the Programmer's Dilemma. Because skills in open source programming are increasingly necessary to enhance the potential career prospects of individual programmers, individual programmers have strong motivations to join in - and as more programmers join in, the incentive for each individual programmer to participate becomes ever stronger. At the same time, the total amount of money that goes to programmers falls as open source is adopted by more companies. Individual programmers, in other words, have selfish motives to engage in collectively destructive behavior.

Thursday, May 24, 2007

That syncing feeling

[profundities and delicious ironies have been scarce lately, so I'm lowering my posting threshold for awhile and will be sending out random bits of not-very-interesting tech geekery. This is by nature of an experiment.]

For various reasons, my life is scattered across several different machines and physical locations right now, and I'm looking for ways to keep things together. I set up an svn repository for most of my work files and code, but there's too much damn state information in tools.

One thing that promised to help was Google Browser Sync, which copies various items from one Firefox instance into another. At first I didn't think much of it, because it didn't do the one thing I wanted most (copying over extensions and their state). However, it does sync browser history and cookies, which is actually pretty useful. And I just found today that if you work with the history sidebar open, you can actually see the syncing happen, as elements from one computer pop on the other. That's pretty nifty.

Now if all the random config files of other programs would migrate themselves automagically from one place to another. It seems like half of the work in programming today is configuration, and the other half is glue.

Saturday, May 12, 2007

I'm naming my kid Trev.r

The other day I was joking with a pregnant colleague about the importance of picking a unique baby name that will permit easy Googling down the road. Turns out I was behind the curve by a few days since the WSJ ran an article on this very subject (via Rough Type).

There's something mildly disturbing about this new twist on technological systems reshaping human existence. It used to be enough for names to be reasonably unique within a family or village, but if we're all globally connected then we will need globally unique identifiers, or close to it.

Wednesday, April 18, 2007

This will go on your permanent record

Did you know that the Federal Government keeps databases on everybody's drug prescriptions? I didn't. The Virgina Tech shootings brought out this fact, according to Glenn Greenwald who quotes ABC news:
Some news accounts have suggested that Cho had a history of antidepressant use, but senior federal officials tell ABC News that they can find no record of such medication in the government's files. This does not completely rule out prescription drug use, including samples from a physician, drugs obtained through illegal Internet sources, or a gap in the federal database, but the sources say theirs is a reasonably complete search.
So, it turns out that despite the tradition of doctor-patient confidentiality, your entire drug history is recorded and available to any agent of government who is curious about you. And who knows who else can access this data?

You are entitled to see what data credit agencies have collected on you. There needs to be a similar law for government dossiers.

And apparently it's a good idea to avoid taking any prescription drug unless you want the world to know you're taking it.