The Problem(s) With Javascript Feeds

Our previous article explained some of the many reasons to avoid Javascript datafeeds, including compatibility, reliance on the merchant’s site, code bloat, the tediousness of updates or modifications, and the invisibility of Javascript feeds to search engines. Have a peek at “Top 5 Reasons to Avoid Javascript Datafeeds” if you’d like a fuller explanation of each of these problems.

How to Make the Best of a Javascript Feed

Sometimes, unfortunately, you’re just plain stuck with Javascript feeds. Many merchants, aiming to provide a solution that will work out of the box even for inexperienced affiliates with little or no technical expertise, provide nothing but Javascript feeds. If you’re working with a merchant that doesn’t provide anything else, this article will help you figure out some of the steps you can take to overcome the inherent problems of datafeeds.

Cache the Feed Locally

One of the simplest steps you can take is simply to serve the datafeed and Javascript parsing code locally on your own server. This simple step provides two advantages over the ordinary Javascript feed:

1. Local caching reduces your reliance on the merchant’s site, since your pricing and product information will continue to be available even if the merchant site is down.
2. Local caching will help your individual web pages load more quickly.

The steps for caching your merchant’s datafeed information are very straightforward:

1. Schedule a simple script that will fetch the feed file from your merchant’s site at regular intervals and store it on your own server.
2. Download the merchant’s Javascript parsing code and store it on your own server. (This only needs to be done once.)
3. Edit the two lines of code in each of your product pages that would normally hit your merchant’s server for a fresh copy of the datafeed and the parsing code, and direct the request to your own server instead.

(A few Javascript datafeeds — the really sloppy ones — don’t include a separate parsing file, but instead build it into each individual web page. If that’s the case for your merchant’s system, you can also take the opportunity to strip out that extra code from each page and bundle it into a separate file, stored locally on your own server.)

Cache, Transform, and Feed to a Database

If you already serve pages using a database language such as PHP, a more complex approach to the problems of Javascript feeds is to grab the data file from your merchant’s site and then transform it, using a scriptable utility such as sed or awk, or direct coding in a language like PHP or Perl, into a format which your existing database system can use. In other words, if your pages are already served up using a database system such as PHP, why not leverage the capabilities built into that system to handle the datafeed?

While different merchants unfortunately employ different formats to deliver their datafeed files, increasingly merchants are standardizing on XML as the preferred file structure. In this case, libraries are widely available (and probably already installed on your server) to help with the transforming or parsing of XML files, and as a lowest common denominator, the job can always be done with the good old standby utilities like sed or awk. The aim here is to take the data portion of the merchant’s Javascript feed, transform it into whatever format your database system already handles, and feed it to the database.

Note that this approach is primarily suited to sites which are already running a databse system to serve pages, and which are looking to add product listings from a new merchant. This approach is unlikely to be very helpful if you actually have to convert an entire (X)HTML-based template site provided by the merchant into PHP just for the sake of overcoming the limitations of Javascript feeds.

Cache, Parse, and Load via SSI in Plain (X)HTML

The same cannot be said of our last suggestion, which happens to be our favourite technique here at Affiliate Insight: namely, rather than transforming the data file into a format which your site database can understand, we parse the feed into its constituent parts and load them via a server-side include (SSI) into plain old (X)HTML pages. This method overcomes virtually all of the disadvantages examined in our first article “Top 5 Reasons to Avoid Javascript Datafeeds”: it eliminates reliance on the merchant site, rids pages of Javascript bloat and incompatibilities, makes updates almost painless, and reveals the full product content of the datafeed to search engine spiders. And, it doesn’t require rebuilding/converting an entire site template in order to take advantage of the capabilities of PHP for delivering individual pages.

The steps involved in caching, parsing, and loading via SSI are:

1. Schedule a simple script that will fetch the feed file from your merchant’s site at regular intervals.
2. Schedule a second sript that will parse the feed data using PHP, Perl, or even sed or awk, saving the results into individual files containing pricing/product tables made of plain (X)HTML.
3. Edit out the tedious and repetitive Javascript calls that place product data into your pages, replacing all that gibberish with a single SSI line that grabs the pre-parsed, plain (X)HTML pricing and information tables.
4. Edit out the lines of your page code which load the Javascript feed and Javascript parsing files from the merchant’s site, and say goodbye to Javascript feeds forever!
5. If necessary, enable your server to parse plain HTML for server-side includes; this is just a matter of setting the file type .html or .htm to server-parsed, in the same way that .php files will probably already be set.

The end result will be vastly cleaner, simpler, and shorter code; pages that load in a fraction of the time; and actual product information that is available to search engine spiders.