POST, GET, and the iTMS
by Scot Hacker
Related link: http://kissthisguy.com/
After a (very) long period of hibernation, I recently resurrected an old site of mine, The Archive of Misheard Lyrics. But after populating the site with thousands of deep links into the iTunes music store, I've found them largely incompatible with the architecture of my site.
For years, I've wanted to stop serving so many random ads on the site and leverage its music-centric content to sell music. The iTunes affiliate program seemed like the perfect opportunity, so I signed up as an affiliate and began the process of populating my databases with special affiliate links to songs and artists.
The other big goal of the redesign was to leverage the collaborative power of the internet and let readers vote on the funniness (or lack thereof) of published lyrics and unpublished submissions. That system came together nicely, but that's where the iTMS connection started to turn sour.
The voting mechanism works through a simple form, submitted through a standard http POST action. PHP session tracking ensures that users can't vote on a given lyric more than once per session. But here's the rub: When you click a link (any link) into iTMS, it makes an http request. That link is processed by Apple, which sends back an http header telling the browser to launch an external application (iTunes). Because Apple is essentially resubmitting the URL of the linking page to itself, the browser thinks the page you just voted on is being re-POSTed. The browser then throws the standard "Are you sure you want to resubmit this form?" dialog, which is very confusing for the user.
To see this in action, visit the site and try some of the "Hear more from this artist" links near the tops of lyric pages. They should work fine. Then try voting on a lyric and then click the "Hear more" link at the top of the resulting page. Oops. Miserable user experience.
The alternative, of course, would be to rewrite the code around GET rather than POST, but that would be a misuse of GET, which is supposed to be used only for retrieving, not submitting information. In addition, using GET would result in all the form variables being sent in the URL string, which is messy. And hidden form variables would also become visible in the URL.
You can sort of see why Apple went this route -- it saves them from having to invent a new internet protocol (which would be a more elegant means of handing the hand-off problem) or from resorting to little downloadable "launch" files, which would litter users' desktops. But I have to wonder whether they considered the ramifications of their technique for sites that use POST for navigation.
Trying to get help -- or even feedback -- from the affiliates program has been a lesson in frustration. My emails to them have been totally ignored.
Unfortunately, the POST problem wasn't the end of my frustrations. It turns out that a lot of the links provided through the LinkShare back-end (Apple partners with LinkShare for the affiliates program; you have to use their back-end to generate your custom links) simply don't work. See the six (currently static) album covers in the left column of the site? Only two of them actually take you to that album in iTMS. The other four take you to the iTMS homepage. All six links were pasted directly out of the LinkShare link maker, and should work as-is. This problem is totally unrelated to the POST problem -- they're just dishing up broken links to affiliates, period.
My original plan for the album covers was to parse the LinkShare-provided iTunes RSS feeds, which are separated into loose genres, and associate them with the genre of the lyric currently being viewed. I envisioned this system where people would always see context-relevant, freshly delivered columns of iTMS-linked album covers. But as I started working with Apple's feeds, I found that they don't deliver RSS in such a way that you can simply use a parser such as Magpie to just grab album covers and links. The only way to get album covers from their RSS is as a series of table blobs, where you end up also displaying the prices, publisher, etc. To get what I want out of their feeds, I'm going to have to do a bunch of scraping and (fragile) post-processing. It would have been so easy for them to deliver nice, atomic XML elements that sites could display as they saw fit.
Finally, I was expecting to find some kind of ad rotation mechanism for affiliates. See those static Stevie Wonder banners at the top of all the lyrics pages? I should be able to drop in a block of code and have those rotated out automatically from iTMS. Instead, the only option is for me to return to LinkShare every few days and grab some new static code. Maybe I'll store a few dozen in a database and do my own randomizer for them. But why should this be so difficult?
An ideal way to run things would be for them to say, "Here's your partner ID. Now you can build this into a link that looks like this for a simple song link, or like this for an album cover, etc." That way I could simply grab URLs from iTunes itself and write code to integrate my partner ID into the links. But because of the way they format links, you're basically forced to scoop every link you want to use -- which comes complete with a hard-coded image link -- out of LinkShare.
I'm trying to sell music for Apple here. You'd think they'd welcome all the help they can get. This whole process has been incredibly frustrating. Maybe I've drunk too much of the Apple Kool-Aid, but I really expect better from them. At this point, I'm starting to consider alternatives to iTMS, such as Rhapsody. But I'm really not eager to face another database population job. If I don't hear from something useful from someone at Apple soon, it's back to the drawing board.
Is there something I'm missing? Or shouldn't this all be a whole lot easier?
In the HTTP spec...
It uses the phrase "SHOULD NOT" which in W3C-language (to the best of my translation abilities) means that they'd much rather you didn't, but understand you may need to, do something a certain way.
From the spec (ftp://ftp.isi.edu/in-notes/rfc2616.txt):
9.1.1 Safe Methods
Right - nothing technically stops me from re-doing the site with GET rather than POST, but I would hate to because :
In the HTTP spec…
No, SHOULD means DO THIS ONLY IF YOU HAVE FULLY UNDERSTOOD THE CONSEQUENCES AND ARE PREPARED TO FACE INTEROPERABILITY PROBLEMS.
FWIW, I went ahead and implemented a hack to work around the POST problem. Now, when a vote has occurred it processes the vote on the POST, then uses a PHP header Location: directive to pass the user to the next page. The result is that the user receives the next page as if they had clicked there from a normal link, so links to iTMS on the resulting page don't throw a "Re-POST?" dialog. Not quite as clean, but it works.
why dodn't you just get the xml datafeed from itunes? it's about 2gb, and has everything you need so you don't have to use the linkmaker..it took me about 2 months to get approved