Extending Yahoo! Pipes with Google App Engine

In my Uploads and Favorites pipe I wanted to turn the timestamps into so called "humanized dates", like these six examples:

23 seconds ago
1 minute ago
4 hours ago
2 days ago
7 months ago
1 year ago

This could not be done by using only Yahoo! Pipes.

Google App Engine to the rescue

Yahoo! Pipes can be extended using a web service, which accepts the feed items as json, and outputs them (after changes) as json. I used this blog article Using Google App Engine to Extend Yahoo! Pipes as a starting point. Django framework has date humanizing functions coded in Python, and I borrowed those functions from there. Then I created a new Google App Engine application "humandate", and made files app.yaml and humandate.py.

app.yaml

Data provided by Pastebin.com - Download Raw - See Original

humandate.py

Data provided by Pastebin.com - Download Raw - See Original

I put these two files in a directory called "humandate" under my Google App Engine directory, with favicon.ico (the GAE default icon), and the development environment is ready. I now can deploy the application using "./appcfg.py update humandate/" or test it locally using "./dev_appserver.py humandate/", then the app runs at http://localhost:8080.

Testing the date humanizer

I made also an url based interface to "humandate" in addition to Yahoo! Pipes json interface. Url based interface works by giving a unix timestamp as "unixtime" parameter, like this. This made testing the date humanizer easier.

Using the service in Yahoo! Pipes

This service expects items to have y:published.utime field present. Service reads it from every item, turns it into a humanized date, and makes a new field "when" containing the humanized date. Then the service returns the items back to pipes.

See also my previous article about the four youtube pipes I've made (one of them uses this humandate, and one uses it indirectly). And have fun with Google App Engine and Yahoo! Pipes.