Building a Sweet RSS Feed in Yahoo! Pipes

I’ve been working on and off on a series of posts on web 2.0 tools that sales people can use to be better prepared for sales calls and to outperform the competition. You can find the beginning of the series here.One of the posts that I’ve really been looking forward to writing is one on Yahoo! Pipes. Pipes, while it appears complicated, is actually a fairly simple data aggregator. Do you have a penchant for building a gizmo that scours criagslist for something and sends you a report with a link and price? Pipes can do that for you. Want to combine Last.fm with YouTube? No problem (that one is really cool). Pipes is really super fantastic good for making any kind of mashup you can envision.I actually discovered Pipes when I was trying to develop a golf / Flickr mashup. Tangent: It would have combined Flickr with golf course reviews in local. If anyone wants to build it, please do so, I couldn’t make it work. What I could do, though, was develop simple, combined RSS feeds. Essentially taking feeds from a number of different sites and combining them into one data feed that I could subscribe to. In other words, I could take all of the official Yahoo! RSS feeds and, rather than subscribing to 20 different feeds, I simply subscribe to one.Using this in a sales environment is really quite simple. I’ll walk you through the first one and you can take it from there. I’ll show you how to make a basic pipe that aggregates a number of sales blogs together. The really nice part about Pipes is that the code is open. I’ll give you a link to my pipe and you can modify it as you see fit.The first step is to go to Yahoo! Pipes and login.Now, you can do one of two things. What I would recommend is that you browse around the pipes and find a couple that you like. From there, just steal their code and modify as indicated above. If you are more of the macho, I want to build this on my own type of person, press the ‘Create a Pipe’ button.You’ll be presented with a screen that looks like a piece of graph paper with the words ‘drag modules here’ in the center. On the left hand side, you’ll see said modules starting with Sources, User Inputs, Operators, Url, etc. These will be your building blocks for the application you are building.To build an RSS aggregator, first, it is important to think about which feeds you want to subscribe to. In my case, I’m going to grab the following sales & marketing blog feeds:

Sweet, you’re with me right? Pipes is open to the front page looking like graph paper and you have the blogs you want. Here is the easy part.From the left hand side of the screen, grab the module called ‘Fetch Feed’ and drag it onto the graph paper. You should have a cool little box that looks like this:

Media_httpwwwscottsch_jpcjl

Grab a few of them and put them on your graph paper.Next, fill in the boxes with the site feed address. This is the one that ends in RSS, atom or has Feedburner or something like that in it. For example, the feed address for my site is: http://feeds.feedburner.com/scottschnaars/kVtx. Just right click (if you are right handed) on the little orange box and select ‘Copy Link Location’. Hopefully you have something that looks a little bit like this:

Media_httpwwwscottsch_txozi

(Click for Bigger Image)You don’t need to be nearly so anal about how clean it is. We are pretty smart at Yahoo and can figure it out if you have modules all over the place.You have to hook all of these feeds together. Grab a module under ‘Operators’ called ‘Union’. This will take all of your feeds and stuff them into one pipe. The ‘Union’ module will support up to 5 feeds and you can unionize unions of unions if you want. Very Big Love.Next, under operators, grab a module called ‘Sort’, drag it to your graph paper and from the drop down menu, pick ‘Itemby.pubDate’ and leave the default as ascending.Now the fun part, connect your modules using Pipes. Simply grab the little ball underneath your ‘Fetch Feed’ module and drag it to your ‘Union’ module. When you are finished dragging feeds to unions, drag your union to your sort. You probably notices that you also have a ‘Pipe Output’ module that was on the page when you dropped your first module there. You want to hook everything to this. In the end, you should have some type of Dr. Octopus schematic that looks more or less like this:

Media_httpwwwscottsch_qhfja

If you did it right, data should be coming out in the gray part at the bottom of the screen. Not just any data, but the data that you want in your application. Congratulations, you’re a Yahoo! Pipes developer. Share your apps with your friends and colleagues.Like I said above, my recommendation would be to find an application that you like and tailor it to your needs. You can access this application and copy the code by clicking here.If you create an application or an aggregator using Pipes, please share them with me.

Building a Sweet RSS Feed in Yahoo! Pipes

I’ve been working on and off on a series of posts on web 2.0 tools that sales people can use to be better prepared for sales calls and to outperform the competition. You can find the beginning of the series here.

One of the posts that I’ve really been looking forward to writing is one on Yahoo! Pipes. Pipes, while it appears complicated, is actually a fairly simple data aggregator. Do you have a penchant for building a gizmo that scours criagslist for something and sends you a report with a link and price? Pipes can do that for you. Want to combine Last.fm with YouTube? No problem (that one is really cool). Pipes is really super fantastic good for making any kind of mashup you can envision.

I actually discovered Pipes when I was trying to develop a golf / Flickr mashup. Tangent: It would have combined Flickr with golf course reviews in local. If anyone wants to build it, please do so, I couldn’t make it work. What I could do, though, was develop simple, combined RSS feeds. Essentially taking feeds from a number of different sites and combining them into one data feed that I could subscribe to. In other words, I could take all of the official Yahoo! RSS feeds and, rather than subscribing to 20 different feeds, I simply subscribe to one.

Using this in a sales environment is really quite simple. I’ll walk you through the first one and you can take it from there. I’ll show you how to make a basic pipe that aggregates a number of sales blogs together. The really nice part about Pipes is that the code is open. I’ll give you a link to my pipe and you can modify it as you see fit.

The first step is to go to Yahoo! Pipes and login.

Now, you can do one of two things. What I would recommend is that you browse around the pipes and find a couple that you like. From there, just steal their code and modify as indicated above. If you are more of the macho, I want to build this on my own type of person, press the ‘Create a Pipe’ button.

You’ll be presented with a screen that looks like a piece of graph paper with the words ‘drag modules here’ in the center. On the left hand side, you’ll see said modules starting with Sources, User Inputs, Operators, Url, etc. These will be your building blocks for the application you are building.

To build an RSS aggregator, first, it is important to think about which feeds you want to subscribe to. In my case, I’m going to grab the following sales & marketing blog feeds:

Sweet, you’re with me right? Pipes is open to the front page looking like graph paper and you have the blogs you want. Here is the easy part.

From the left hand side of the screen, grab the module called ‘Fetch Feed’ and drag it onto the graph paper. You should have a cool little box that looks like this:
Pipes - Fetch Feed
Grab a few of them and put them on your graph paper.

Next, fill in the boxes with the site feed address. This is the one that ends in RSS, atom or has Feedburner or something like that in it. For example, the feed address for my site is: http://feeds.feedburner.com/scottschnaars/kVtx. Just right click (if you are right handed) on the little orange box and select ‘Copy Link Location’. Hopefully you have something that looks a little bit like this:
Pipes Editing
(Click for Bigger Image)

You don’t need to be nearly so anal about how clean it is. We are pretty smart at Yahoo and can figure it out if you have modules all over the place.

You have to hook all of these feeds together. Grab a module under ‘Operators’ called ‘Union’. This will take all of your feeds and stuff them into one pipe. The ‘Union’ module will support up to 5 feeds and you can unionize unions of unions if you want. Very Big Love.

Next, under operators, grab a module called ‘Sort’, drag it to your graph paper and from the drop down menu, pick ‘Itemby.pubDate’ and leave the default as ascending.

Now the fun part, connect your modules using Pipes. Simply grab the little ball underneath your ‘Fetch Feed’ module and drag it to your ‘Union’ module. When you are finished dragging feeds to unions, drag your union to your sort. You probably notices that you also have a ‘Pipe Output’ module that was on the page when you dropped your first module there. You want to hook everything to this. In the end, you should have some type of Dr. Octopus schematic that looks more or less like this:
Pipes Editing 3

If you did it right, data should be coming out in the gray part at the bottom of the screen. Not just any data, but the data that you want in your application. Congratulations, you’re a Yahoo! Pipes developer. Share your apps with your friends and colleagues.

Like I said above, my recommendation would be to find an application that you like and tailor it to your needs. You can access this application and copy the code by clicking here.

If you create an application or an aggregator using Pipes, please share them with me.