Harper Reed: “The power is not the mashup. It’s the data.”

Chicago Transit Authority | CTA Bus TrackerWhen most people go to the CTA Bus Tracker Web site and click on Estimated Arrival Times or the Bus Location Map, they see a way to find out when their next bus might get to the stop near them. Another nifty map mashup they can use to plot their trips daily.

When Harper Reed saw the same Web site last year, he saw every bus, everywhere, forevermore. And he wanted that data. “The power is not the mashup. It’s the data. The data is the answer,” says Harper.

So he set out to separate the data from the mashup. He used Firebug, the nifty Firefox add-on, to monitor that network traffic (the communication that goes between the browser window and the server) that was driving the data to the page.
This way, he found the URLs of the “endpoints” — a fancy word for URLs with special codes inside them — that control the entire Bus Tracker system.

Here’s a simple example:


Breaking it down, element by element, this seemingly complicated URL is really just one machine (your computer) making a request of another machine (the CTA’s Bus Tracker server). It’s saying:

  • http://ctabustracker.com/: Hey, Bus Tracker
  • bustime/: I wanted to ask you about Bus Times
  • eta/: Specifically, some Estimated Times of Arrival for a specific
    route going in a specific direction for a specific bus stop, right
    about now
  • eta.jsp?: So anyway, here’s what I wanted to ask you:
  • route=7: On Route 7, the Harrison Bus
  • &direction=West%20Bound: For buses heading away from the lake
  • &stop=5400%20W%20Harrison: At the bus stop located at 5400 West Harrison
  • &showAllBusses=on: Can you please let me know when the next bus (any bus on that route) will get there?

CTA Bus Tracker "API"
Bingo. And he did that again and again, making all sorts of requests of the Bus Tracker system– just as a user, like you and me– and found all of the possible data types that are stored in the database and delivered to your browser, like a river that you can never step into twice. No matter what you do with Bus Tracker, no matter when or how you hit
it, all it’s doing is answering requests that are formed like this. It’s the data. The data is the answer.

After he found these endpoints, he wrote down what each of them are, what they mean, and what information they give you. The result is the CTA Bus Tracker “API”. Think of it  s a specification written after the fact. Like if someone came to your house, looked at your toaster real closely, and wrote down how it makes bread crispy and hot. And
it’s not the Toaster Engineer doing the writing, but some other guy.

Last Fall Harper and I met with some of the people at the Chicago Transit Authority
who are responsible for Bus Tracker and all other communication tools
there. They are open to developers working with their data. The woman
who escorted us up from the lobby had on the exact same purple color as
Harper, so I took a picture.

Harper Reed at the Chicago Transit Authority lobbyWe also attended a Coffee with Ron Huberman that my brother, Kevin O’Neil of CTA Tattler, put together. Ron showed us a prototype of the new Web site they were working on (and launched recently). One part of the Web site he showed us — it is not yet launched — is Developer Tools.

Speaking of developers, another thing Harper did was to set up a system for making regular queries to Bus Tracker and caching (storing) the results in a separate, stand-alone database over on a server that goes by the name of http://chicago.transitapi.com. This allows other developers to have a reliable database to ask questions of (see above) without having to ask the CTA all the time, and perhaps tax their servers or otherwise bother them. Like having a record of every instance of the river, stored forever.

The Chicago Bus Tracker Widget uses the Chicago Transit API to do cool things with this data. See also Chicago’s CTA Bus Tracker for iPhone and iPod Touch. I’d like to see more apps that use some of the more obscure data in the API. A “which bus should I take” app would be cool — just click to show where I am on a map and all the nearby bus stops with arriving buses (hat tip: Paul Smith).

One thing I noticed in looking at the documentation is that there are a lot of “stubs” for data in there that are not populated with useful information. Some examples:

  • Under “getRoutePoints”, there is a field for “serviceBulletins”. It would be nice to see info from the relevant RSS feed popped right in there
  • There is a field for “shortName”, but it always seems to be
    populated with the same number as “id”. It would be good to see the
    vernacular name for the bus here — the “Western Express” rather than “49X”
  • There is a field for “Length”, but always seems to be empty. Is
    this the length of the actual bus stop, in feet? The length of the
    route, in miles? Either way, that would be telling you something
  • Take a look at a getBusesForRoute example for the #8 Halsted.
    Great info on the bus — where it is (longitude/ latitude), who’s
    driving it (operator), where it’s going to end up (finalStop). I’d like
    to also know the last time it was cleaned, last time it was serviced,
    whether the bus was ever a part of an accident or a crime scene. Why
  • Under getTimeAndTemp (currently the coolest way to get this info), it would be nice to also pull whether there is currently any precipitation

There are other things not in the API that would be useful to developers and bus-taking decision makers.

  • Is there a canopy at that bus stop?
  • How about a seat?
  • What kind of seat?
  • When was this bus stop last cleaned?
  • When was graffitti last removed here?

As of January 26, eleven routes will be added to Bus Tracker: No. 3 King Drive, X3 King Drive Express, 4 Cottage Grove, X4 Cottage Grove Express, 22 Clark, 36 Broadway, 49B North Western, 82 Kimball/Homan, 93 California/Dodge, 96 Lunt and 97 Skokie. Huberman has said that one of the criteria for adding buses was stabilization of operations (no bus bunching and good on-time performance), so that’s good news for riders on these routes.

All hail the CTA Bus Tracker, the CTA, the CTA Bus Tracker “API”, and Harper Reed.

CTA launches RSS feeds for every train and bus line

RSS Feed_1229466218100
Probably the best thing to come out of the launch of a new Web site for the Chicago Transit Authority is an RSS feed for every train line and bus line in the system. It's a veritable orgy of orange icons — you can subscribe to all sorts of breakdowns:

This is a really big step forward for the CTA. They tell me that they have also changed their operations considerably to support this service. They now have six full-time External Customer Communication Coordinators who work inside the CTA Control Center. Each of these statuses are updated manually by these people, who work alongside the Rail Controllers who are working to respond to any service outages or other issues in the system. These two functions — dealing with issues and communicating the current status of an issue to customers — used to be performed by the same person at the same time.

Now that they have the technical and operational infrastructure to support these feeds, the only thing I'd like to see is more. More information about each bus and train. Certainly Facebook and Twitter have made a certain subset of the population deeply accustomed to being updated about the most inane things. I'd certainly be open to being updated about every time a bus that I'm interested in gets into an accident, contains a passenger who is shot and killed, has one of its bus stops moved due to a scaffolding permit issued by the city, and so on. Hell, I'd even be interested in knowing that my bus got some gum cleaned off a seat.

Look for these feeds to be pumped into their appropriate CTA Tweet feeds soon.

New CTA Station Elevator Status Signage

I’ve been meaning to post this for a while. Here’s the new CTA signage system for Station Elevator Status. I noted a couple years ago that the previous system was a little goofy because the information people need (whether they’ll be able to get out of their destination station) is not available until you get down
to the station of origin– after the person has begun their journey.

This system is better — it definitely organizes it way better and makes it more scannable — but the fundamental issue remains. In fact, most of these boards are placed at the bottom of a set of stairs.

I called 888-YOUR CTA at 4:30PM on Saturday, July 19, 2008 and pushed "2". I got a detailed message that was recorded at 2PM, and they said the next update was at 5PM.

So obviously there is a person or set of people who pay attention to updating people. Why not tweet it and publish directly to the CTA Web site?