How I Harness My Twitter Stream

For me, microblogging just for the sake of it hasn’t been engaging enough to hold my interest, but what has kept me coming back to Twitter has been the growing network of people I follow.

I love to dip my Twitter cup into the stream of chatter from the people who build and use the products and services that I test and write about at eWEEK. It’s also interesting to hear what other media types are talking about, although too much of that can lead to head-bursting echo chamber periods where my peers make exactly the same comments about Steve Jobs for days on end.

But the main problem with a big and bountiful network of Twitter friends is that once your friends list grows beyond a fairly small number of people, it gets really tough to pay attention to what people are saying.

Things get even worse when you follow megaprolific Tweetsmiths who tend to flood your Twitter stream with their chatter, or if you follow news organisations that post links to all of their stories on Twitter. This weekend, I followed the Huffington Post for about 10 minutes before deciding that I couldn’t afford to pay attention to all of the site’s output–I’d never get to see Tweets from anyone else!

It’s too bad, because another really great thing about Twitter is the amount of standardisation the service imposes on content producers. Everyone gets 140 characters, and any images or videos or sound files or advertisements have to live behind an embedded link.

In contrast, my Google Reader is a mess of differently formatted entries, some containing only a headline, some with long, picture-laden, full-sized stories, some with way more advertisements than actual content, some with embedded videos or podcasts.

When I fire up Reader and set out to peruse my many RSS subscriptions, I’m constantly context-switching, stopping for a moment to figure out what I’m looking at, rather than mechanically scanning for what’s interesting.

Recently, while checking out the renovations that Google made to Reader in late 2008, I spied the heading “Stay connected to friends & family” within the application’s feed browsing interface. One of the drop-down options under the heading was Twitter–you can type in the name of a Twitter user and consume that user’s status updates through RSS. You can also subscribe to someone’s status update RSS feed right from his or her page using the familiar orange radar wave graphic in your browser’s address bar.

I figured that if I followed each of my Twitter friends through Google Reader, I could take advantage of Google’s “auto-sort” view mode, which prioritises feeds with fewer entries to prevent them from being drowned out by fatter feeds. Perfect for the Huffington Post scenario, right?

What’s more, the RSS route could allow me to organise my Twitter friends into groups, and even to unsubscribe from people without unfollowing them. It hasn’t been an issue for me, as far as I know, but apparently lots of people get hurt if they don’t get a follow, which has led some of Twitter’s cooler kids to wish for a “fake follow” feature.

Now, I wasn’t about to enter the names of each of my Twitter friends in Google Reader’s little drop-down input box, nor was I going to click the RSS button for each of my more than 300 friends. Twitter does offer an RSS feed that covers all of the people you follow, but one big friend feed wouldn’t work with Google’s auto-sort.

I hit the Web and found a Perl script called twitter-followers-to-opml, which talks to Twitter, fetches a list of the RSS status feeds of your followers, and spits the list out into OPML, which you can use to populate your feed reader.

I was looking for the feeds of my friends rather than my followers (although there’s significant overlap between the two lists), so I had to make a small change to the script, and swap out followers() for friends() near the top of the script. Also, since the Net::Twitter Perl library on which the script depends only fetches your last 100 friends, I had to figure out how to pass the page=2, page=3, etc., arguments to friends().

NOTE: At the time that I’m writing this post, the link to the script I’m talking about is dead. I’ll check back later and update the post. In any case, the script seems fairly straightforward, and probably wouldn’t be too tough for an actual hacker (in other words, not me) to throw together. Alternatively, Twitter could make life easier on everyone and add an OPML export option itself.

UPDATE: The script page is live again.

I imported all my Twitter friend feeds into Google Reader and tagged them all “twitter,” so I could read them at once. Before I left the office yesterday I marked all the items in my Twitter folder as read and came into the office today to find over 1,000 new, auto-sorted tweets waiting for me.

I like the way that Google Reader lets me scroll down through the tweets, marking them read as I go, and automatically refreshing the list. I scan my tweets, and middle-click the ones that pique my interest for follow-up.

So far, so good. I’m going to boost my Twitter friend network, refreshing my OPML file to pick up the new friends as I go, and we’ll see how well this Reader/Twitter combo works for keeping the flow useful.

If you are a Twitterin’ IT person, I would love to add you to my friends list, so send me an e-mail at jbrooks@eweek.com or ping me (where else) on Twitter, at jasonbrooks.