The Master Switch: What will become of the internet?

David Leonhardt has a nice review of Tim Wu's new book "The Master Switch: The Rise and Fall of Information Empires", which made me want to run out and buy it.  Here's the main graf: 

AT&T is the star of Wu’s book, an intellectually ambitious history of modern communications. The organizing principle — only rarely overdrawn — is what Wu, a professor at Columbia Law School, calls “the cycle.” “History shows a typical progression of information technologies,” he writes, “from somebody’s hobby to somebody’s industry; from jury-rigged contraption to slick production marvel; from a freely accessible channel to one strictly controlled by a single corporation or cartel — from open to closed system.” Eventually, entrepreneurs or regulators smash apart the closed system, and the cycle begins anew.

The story covers the history of phones, radio, television, movies and, finally, the Internet. All of these businesses are susceptible to the cycle because all depend on networks, whether they’re composed of cables in the ground or movie theaters around the country. Once a company starts building such a network or gaining control over one, it begins slouching toward monopoly. If the government is not already deeply involved in the business by then (and it usually is), it soon will be.

What, then, of the internet?   Well, the answer is that it's up to us, as consumers and interested parties to make sure that the features of the internet that we love, its openness, ease of access and its culture of linking and transparency, survive the changes in its structure that are sure to come by.

Wu recently got into a tiff with other theorists over the meaning of the word "monopoly."  To know more, click here, here and here (and there are plenty of other links on the pages themselves).

Journalists and the web

One of my pet peeves with newspapers today is that even with all the ink (and tears!) that get shed about how the web has changed the dynamics of the newspaper industry -- and may end up transforming the publishing culture completely -- newspapers still don't use the web as it should be used.  The most important affordance of the web is linking -- and newspapers still don't link in their online articles.

So, I am always happy when an odd journalist does do that.  Here is A. O. Scott in his review of the Deathly Hallows (to be fair, this is not the first time Scott has linked):

The movie, in other words, belongs solidly to Mr. Radcliffe, Mr. Grint and Ms. Watson, who have grown into nimble actors, capable of nuances of feeling that would do their elders proud. One of the great pleasures of this penultimate “Potter” movie is the anticipation of stellar post-“Potter” careers for all three of them. 

Click on the link to see where it leads!  Heh.

The always-solid David Leonhardt links too.

Scott Rosenberg vs. Nicholas Carr

Scott Rosenberg has a nice post up on his blog explaining why the studies that Nicholas Carr cites to show that web links impede understanding do nothing of the sort.  This doesn't completely rebut what Carr is saying, of course, but it does tell us that it's really too soon to know whether links impede or enhance our understanding.

Here's the relevant graf:

“Hypertext” is the term invented by Ted Nelson in 1965 to describe text that, unlike traditional linear writing, spreads out in a network of nodes and links. Nelson’s idea hearkened back to Vannevar Bush’s celebrated “As We May Think,” paralleled Douglas Engelbart’s pioneering work on networked knowledge systems, and looked forward to today’s Web.

This original conception of hypertext fathered two lines of descent. One adopted hypertext as a practical tool for organizing and cross-associating information; the other embraced it as an experimental art form, which might transform the essentially linear nature of our reading into a branching game, puzzle or poem, in which the reader collaborates with the author. The pragmatists use links to try to enhance comprehension or add context, to say “here’s where I got this” or “here’s where you can learn more”; the hypertext artists deploy them as part of a larger experiment in expanding (or blowing up) the structure of traditional narrative.
These are fundamentally different endeavors. The pragmatic linkers have thrived in the Web era; the literary linkers have so far largely failed to reach anyone outside the academy. The Web has given us a hypertext world in which links providing useful pointers outnumber links with artistic intent a million to one. If we are going to study the impact of hypertext on our brains and our culture, surely we should look at the reality of the Web, not the dream of the hypertext artists and theorists.

The other big problem with Carr’s case against links lies in that ever-suspect phrase, “studies show.” Any time you hear those words your brain-alarm should sound: What studies? By whom? What do they show? What were they actually studying? How’d they design the study? Who paid for it?

To my surprise, as far as I can tell, not one of the many other writers who weighed in on delinkification earlier this year took the time to do so. I did, and here’s what I found.

You recall Carr’s statement that “people who read hypertext comprehend and learn less, studies show, than those who read the same material in printed form.” Yet the studies he cites show nothing of the sort. Carr’s critique of links employs a bait-and-switch dodge: He sets out to persuade us that Web links — practical, informational links — are brain-sucking attention scourges robbing us of the clarity of print. But he does so by citing a bunch of studies that actually examined the other kind of link, the “hypertext will change how we read” kind. Also, the studies almost completely exclude print.

If you’re still with me, come a little deeper into these linky weeds. In The Shallows, here is how Carr describes the study that is the linchpin of his argument:

In a 2001 study, two Canadian scholars asked seventy people to read “The Demon Lover,” a short story by the modernist writer Elizabeth Bowen. One group read the story in a traditional linear-text format; a second group read a version with links, as you’d find on a Web page. The hypertext readers took longer to read the story ,yet in subsequent interviews they also reported more confusion and uncertainty about what they had read. Three-quarters of them said that they had difficulty following the text, while only one in ten of the linear-text readers reported such problems. One hypertext reader complained, “The story was very jumpy…”

Sounds reasonable. Then you look at the study, and realize how misleadingly Carr has summarized it — and how little it actually proves.

The researchers Carr cites divided a group of readers into two groups. Both were provided with the text of Bowen’s story split into paragraph-sized chunks on a computer screen. (There’s no paper, no print, anywhere.) For the first group, each chunk concluded with a single link reading “next” that took them to the next paragraph. For the other group, the researchers took each of Bowen’s paragraphs and embedded three different links in each section — which seemed to branch in some meaningful way but actually all led the reader on to the same next paragraph. (The researchers didn’t provide readers with a “back” button, so they had no opportunity to explore the hypertext space — or discover that their links all pointed to the same destination.)
Here’s an illustration from the study:

Bowen’s story was written as reasonably traditional linear fiction, so the idea of rewriting it as literary hypertext is dubious to begin with. But that’s not what the researchers did. They didn’t turn the story into a genuine literary hypertext fiction, a maze of story chunks that demands you assemble your own meaning. Nor did they transform it into something resembling a piece of contemporary Web writing, with an occasional link thrown in to provide context or offer depth.
No, what the researchers did was to muck up a perfectly good story with meaningless links. Of course the readers of this version had a rougher time than the control group, who got to read a much more sensibly organized version. All this study proved was something we already knew: that badly executed hypertext can indeed ruin the process of reading. So, of course, can badly executed narrative structure, or grammar, or punctuation.

Rosenberg also has a nice follow-up post on what it is that links actually end up doing in articles on the web.  

Algorithmic culture and the bias of algorithms

Via Alan Jacobs, I came across a thought-provoking blog-post by Ted Striphas on "algorithmic culture."  The issue is the algorithm behind Amazon's "Popular Highlights" feature.  (In short, Amazon collects all the passages in its Kindle books that have been marked, collates this information and displays it on its website and/or on the Kindle.  So you can now see what other people have found interesting in a book and compare it with what you found interesting.)

Striphas brings up two problems, one minor, one major.  The minor one:

When Amazon uploads your passages and begins aggregating them with those of other readers, this sense of context is lost. What this means is that algorithmic culture, in its obsession with metrics and quantification, exists at least one level of abstraction beyond the acts of reading that first produced the data.

This is true but it could easily be remedied.  Kindle readers can also annotate passages in the text and if they feel like it, they could upload their annotations along with the passages they have marked.  That should supply the context of why the passages were highlighted. (Of course, this would bring up another thorny question: what algorithm to use to aggregate these annotations.) 

But he brings up another far more important point:

What I do fear, though, is the black box of algorithmic culture. We have virtually no idea of how Amazon’s Popular Highlights algorithm works, let alone who made it. All that information is proprietary, and given Amazon’s penchant for secrecy, the company is unlikely to open up about it anytime soon.

This is a very good point and it brings up what I often call the "bias" of algorithms.  Algorithms, after all, are made by people and they show all the biases that their designers put into them.  In fact, it's wrong to call them "biases" since these actually make the algorithm work!  Consider Google's search engine.  You type in a query and Google claims to return the links that you will find most "relevant."  But "relevant" here means something different from the way you use it in  your day-to-day life.  "Relevant" here means "relevant in the context of Google's algorithm" (a.k.a. PageRank). 

The problem is that this distinction is lost on people who just don't use Google all that much.  I spend a lot of time programming and Google is indispensable to me when I run into bugs.  So it is fair to say that I am something of an "expert" when it comes to using Google.  I understand that to use Google optimally, I need to use the right keywords, often the right combination of keywords along with the various operators that Google provides.  I am able to do this because:

1. I am in the computer science business, and I have some idea of how the PageRank algorithm works (although I suspect not all that much) and 
2. because I use Google a lot in my day-to-day life.  

I suspect that (1) isn't at all important but (2) is.  

But (2) also has a silver lining.  In his post, Striphas comments:

In the old paradigm of culture — you might call it “elite culture,” although I find the term “elite” to be so overused these days as to be almost meaningless — a small group of well-trained, trusted authorities determined not only what was worth reading, but also what within a given reading selection were the most important aspects to focus on. The basic principle is similar with algorithmic culture, which is also concerned with sorting, classifying, and hierarchizing cultural artifacts. [...] 

In the old cultural paradigm, you could question authorities about their reasons for selecting particular cultural artifacts as worthy, while dismissing or neglecting others. Not so with algorithmic culture, which wraps abstraction inside of secrecy and sells it back to you as, “the people have spoken.”

Well, yes and no.  There's a big difference between the black box of algorithms and the black box of elite preferences. Algorithms may be opaque but they are still rule-based.  You can still figure out how to use Google to your own advantage by playing with it.  For any query you give to it, Google will give the exact same response (well, for a certain period of time at least).    So you can play with it and find out what works for you and what doesn't.  The longer you play with it, the longer you use it, the more you become familiar with its features, the less opaque it seems.

Not so with what Striphas calls "elite culture," which, if anything, is far more opaque and far less amenable to this kind of trial-and-error practice.  (That's because the actions of experts aren't really rule-based.)

I am not sure where I am going with this and I am certainly not sure whether Amazon's Kindle aggregation mechanism will become as transparent as Google's search algorithm by trial-and-error but my point is that it's too soon to give up on algorithmic culture.


Postscript: My deeper worry is that when we actually reach the point when algorithms are used far more than they are now, the world will be divided into two types of people.  Those who can exploit the biases of the algorithm to make it work well for them (like I do PageRank).  And those who can't.  It's a scary thought although since I have no clue about how such a world will look like, this is still an empty worry.

How we read (articles and magazines) now

In this post, I want to compare the two modes of reading: how we read before the internet and how we read now. I will be limiting the analysis only to news articles and magazines since I believe, pace Nicholas Carr, that it isn't still clear how the internet has changed book-reading habits. But today, the internet IS the platform for delivering news and I believe it has changed our reading habits substantially. (It's also caused a big disruption in the publishing industry.)

Just to be clear, the analysis in this post is based on a sample size of 1: me! But I believe the salient points will hold true for many others.

How we read Pre-internet

Back in the days when there was no internet, we had two newspapers delivered to our home in the morning: a newspaper in English (we alternated between The Times of India or the Indian Express) and one in Marathi (Maharashtra Times or Loksatta). On Mondays we also took in the Economic Times and on weekends, my father would usually buy some more from the news-stand (The Sunday Observer, The Asian Age).

My father would usually read every newspaper from the first page to the last. My mother would usually only read the Marathi newspaper. My sister and I would usually at least skim through the English newspaper but concentrate on a few sections (usually comics, editorials, and sports).

Even in India, where newspapers are generally smaller, (i.e. less number of pages compared to, say, the tome that is the daily New York Times), a newspaper would try to cover everything it thought needed to be covered. So while the bulk of it would be devoted to national politics, there would be a smattering of world news, book reviews, entertainment and sports news, etc. No issue would be covered in too much depth (because of lack of space, which in turn corresponded to the high cost of newsprint) and the issues covered would be chosen keeping in mind the broad preference of a newspaper's readers.

So if you were interested European Union politics, you would be lucky to get an article every two weeks. But at the same time, the very fact that newspapers felt that they were the only way we got content, they did feel obligated to at least sample the whole spectrum of topics. E.g. even if the sports pages were dominated by cricket, soccer and tennis, at least once in a while, volleyball and football would be covered.

In the figure above (click to see full figure), I have chosen to represent how we read back then in terms of two orthogonal axes: the breadth of reading (X-axis) and the depth of reading (Y-axis). The breadth of reading corresponds to the topics that we could cover, the depth of reading corresponds to how much we could cover on a certain topic. Each line represents a topic; how long the line is corresponds to how deeply a topic is covered.

The characteristics of pre-internet reading were:

• The newspaper/magazine decided what topics it would cover (and it tried to take the interests of the majority of its readers into account). This meant, to some extent, that the "long tail" of topics was not covered.
• The limited space available for a newspaper meant that:
• The depth that it could go into for any topic that it covered was limited.
• However topics were sampled uniformly across the spectrum of topics. Which meant that even if you didn't care for national politics at all, the very act of glancing through the newspaper everyday forced you to have some idea of what was going on. Ditto for sports, or for entertainment.

Pre-internet reading thus tended to emphasize breadth over depth. This was its advantage as well as its disadvantage. Diligent readers were "well-rounded" but they came up against the hard limit of newsprint if they wanted to know more.

How we read now
Reading on the internet is different.

There's far far more material to read, and it's easy to get to, requiring nothing more than a click of the mouse.

There's also more control over the material meaning you can decide what you want to read and ignore the stuff that you don't care for. E.g. if you are interested in economic policy you can subscribe to the Business feed of the NYT and nothing else. If you're interested only in foreign policy and particularly in the U.S.-China relationship, you can use Yahoo Pipes and set up filters that will bring to you only those articles that mention the U.S. and China.

Finally, there's more depth. A newspaper, even an online newspaper, can only devote so much time and space to U.S.-China relations. But on the internet, this material can be augmented with the many blogs and wikis out there. There are blogs by anthropologists and economists, by foreign policy and international relations experts, and by political scientists, -- you name it! -- all of whom are interested in conveying their point of view to both lay and specialized audiences. In other words, you can choose to specialize in whatever you want and your specialization will be world-class.

In the figure above (click to see full figure), I've tried to represent the characteristics of online reading in terms of its breadth and depth. The characteristics of internet reading are

• We get to decide what topics we want to read about.
  
• There is unlimited "space;" it is possible to access any conceivable magazine or newspaper; in addition there are blogs, wikis and Twitter feeds that we can use as our filters to get to the interesting stuff that we care for.
  
• One can go as deep into a topic as one wants.
  
• But since our time is ultimately limited, emphasizing depth has the effect of sacrificing breadth. One could go as deep as one wants and read about international relations but the cost could well be no time to read any sports news. Whether this is a good or bad thing, I am not sure.

Reading on the internet tends to emphasize depth over breadth. This is its advantage as well as its disadvantage. It's possible to indulge your interests a lot, but it comes at the cost of being "well-rounded."