NPR now supports JSON Feed!

The NPR Tech Team achieved several technical accomplishments last month, most notably launching HTTPS support for npr.org. We didn’t make a public announcement at the time, but we also quietly launched something I’m personally very excited about: full JSON Feed support for NPR news feeds (example) and podcasts (example).

The official JSON Feed icon. (Source: JSONFeed.org)

Why JSON Feed?

When the JSON Feed project was announced in May 2017, I felt immediately that this is something NPR should be throwing its weight behind. As a media organization that produces both news articles (consumed by feed readers) and podcasts (consumed by Apple Podcasts and other podcast players), we have a heavy dependency on RSS. I often think that technology ages akin to the concept of “dog years” — that is, one calendar year for humans equals about 7 years for technology. By that logic, RSS 2.0 (the version of the spec most widely used today) is 105 years old.

I’m not going to speculate on all the reasons why the media and tech industries as a whole are still actively supporting a frozen spec from 15 years ago, but suffice it to say that personally, I think it’s a problem. One of my biggest issues with RSS is its usage of XML, which has already gone out of vogue for usage in web service APIs years ago in favor of JSON. Most programming courses don’t teach XML or ancillary techniques like XPath anymore; my partner just went through a web development bootcamp earlier this year and did not have a single assignment involving XML, though they used JSON regularly. The fact that the newest generation of developers is no longer learning this technology ought to be a clear sign that its star has faded.

Aside from its diminishing usage, another issue with XML is its brittleness, stemming from its draconian formatting and escaping rules. At NPR, we distribute podcasts and content from other sources, which we often get by scraping their RSS feeds; I can’t tell you how often we have issues with it because a feed author failed to properly encode an ampersand. The need for

Finally, the fact that the RSS specification is frozen is, in my opinion, problematic; I think we should always be striving to build and rebuild the web on top of living, actively-maintained technologies and standards that account for our needs today and two years from now, and are less concerned with what our needs might have been ten years ago. While I agree that too much flux would be difficult for e.g. feed reader and podcast player software developers to work with, there should always be room for change and evolution. Virtually all other open web technologies have standards bodies to whom requests for additions or changes to the specification can be submitted; such an avenue should always be available. And while, yes, XML does support namespaces and RSS can therefore technically be extended (and Apple Podcasts has done this with the custom namespace), this still ultimately feels like a poor man’s solution for an industry-wide problem. In particular, these namespaces imply vendor-specific solutions for common needs (such as the desire to distinguish between seasons of a podcast) rather than a standardized approach that all consumers can agree upon.

Despite all of this criticism of RSS, however, it isn’t going away anytime soon; even if it does, NPR will likely be one of the last holdouts supporting it, again because news feeds and podcasts are such a big part of how we distribute our content. But if a serious contender for a replacement technology were to emerge, we should simultaneously be supporting that replacement — especially if it doesn’t cost us anything. And that was at the core of why we decided to do this now, even though there are not that many feed readers and podcast players that support JSON Feed yet: because it doesn’t really cost us anything to add another feed type. The bandwidth ultimately is cheap, and the data is generated using existing APIs, so maintenance should be trivial.

The hope is that by NPR being among the first major publishers to support JSON Feed, other media companies will be encouraged to do the same, speeding adoption of the standard.

How we did it

I’m not going to go into the nitty-gritty implementation details because much of this involved deconstructing how we produce RSS feeds and wrestling with aging XML APIs under the hood — yes, there is some irony there. NPR is currently working on a new JSON-based hypermedia API that is intended to replace our XML-based Story API (which dates back to 2008); the plan is to eventually have our JSON Feed implementation (and all of the rest of our products) be driven by the new API, but it’s not yet ready for primetime, so building this on top of the XML API was the best we could do in the interim. Suffice it to say that that particular closet is full of skeletons, but since we’re planning on swapping it out for a whole new closet in the not-too-distant future, I’ll spare you the details.

In terms of getting the work on the roadmap, I’m happy to say that this was another example of a success story from NPR’s thrice-yearly Serendipity Days (our version of “hack week”, described in more detail by my colleague Jason Grosman here). Interestingly enough, JSON Feed was announced on the last day of our May iteration; when I saw the announcement, I thought to myself, “Dang it, that would’ve made for a great Serendipity Days project.” Thankfully, by the time our August iteration rolled around, I hadn’t thought of any other good projects to pursue instead, so I decided to implement JSON Feed. It was exactly the right size project, and I managed to get a fully-functional demo working just in time for presentation day.

There isn’t one formal process to get a Serendipity Days project into production (other than that it’s assumed that nothing is ready to ship immediately after SD is done), which is why it took until early November for the project to finally launch. In my case, it helped that the code was essentially already complete (other than some minor refactoring needed because towards the end I’d naïvely copy-pasted a few things that begged for a more reusable approach); it just needed to be reviewed by a few other developers to ensure I hadn’t done anything egregiously bad or questionable. I was also lucky enough to have a couple of product owners and managers champion this work because they agreed this is something NPR should support; hat-tip to Stacey Goers, Ha-Hoa Hamano, and Paul Miles. With their help, I managed to squeeze this into our November 7th release. And thus, NPR’s JSON Feeds quietly went live to the world!

[email protected]

See Full Bio