Add Yahoo! Pipes to your toolbox!

TodayI added a feed to the Profoss website. The problem is that the Profoss website contains news in different sections: one for each event, and a general news section about the Profoss itself. And I didn't want to publish multiple feeds: each time a new event is announced, people would have to subscribe to an additional feed; and each time an event is over, people would have to unsubscribe as no more news would be published.

I could have developed this on the server, but it was much easier to generate a feed for each of the news sections, and combine them with Yahoo! Pipes. I had never used Yahoo! Pipes before, but this didn't prevent me of getting a working (and definitive) solution in less than 10 minutes! I created 3 feed reader elements (one for the Asterisk event, one for the virtualisation event, and one for the general news) I then added an union node that combines all input in one output, and connected the feed readers' outputs to the input of this node. As I wanted the items to be ordered by publishing date, I added a sorting node, and specified the publishing date as sorting criteria. After connecting the output of the sorting node to the pipe's output, I was done. Running the piped provides a feed combining all items of the individual feeds ordered by publishig date.

Using the url of the rss output of the pipe with FeedBurner gives the final feed to subscribe to.


This is a very simple example of Yahoo! Pipes use, but the fact that it took me only 10 minutes to get the definitive solution working is a good indication it is very intuitive and effective. It is well thought out, eg you can see the output generated by each node of the graph, which helps you debugging your pipes. There's also good documentation, and the great variety of nodes available (there's a location extractor node for example!) should help you manipulate feeds in a lot of situations.

Using it made me think of CloverETL, also a great tool, but for data(base) migration and manipulation.