Newsmastering Tips and Hacks – Part I – Keyword RSS GeneratorsJune 13, 2007 at 3:53 am | Posted in Unscientific Research | 1 Comment
Tags: newsmastering, rss, tools
This article is a first one of series of the articles we contemplated for a while to share our Newsmastering experiences/lessons learned and, hopefully, trigger a discussion amongst a small but growing community of Newsmasters.
Here are some of our tidbits on this very broad subject – Newsmastering Hacks – How to generate RSS feeds from MSN.com and others at currently defunct Radar Forums, however, they are not structured and were written mostly in a response to challenges we encountered with some sources of RSS feeds.
The Newsmastering can be defined as the ability to define, locate, select, aggregate, filter, process and publish Web-based information based on a specific subject. The RSS Newsmastering utilizes the Really Simple Syndication (RSS) feeds, which include brief summaries of web and news articles, blog and user group postings, images, videos and audios, messages board/forum threads, job postings etc.
These pieces can be composed into specific information channels, which have a variety of names: Feed Digests, Lenses, FeedBots, and Bees etc. We call them News Radars.
News Radars applications are multiple and just to name few:
1) Web Buzz and News Monitoring
2) Competitive Intelligence and Market Research
3) Conversational Marketing
4) Web Site Dynamic Content (Widgets and Mini-Sites)
5) Web Knowledge Bases
6) Job Search Engines
7) Internet TV Channels
8) Search Engine Optimization (SEO)
The information aggregation, filtering, processing and publishing requires the Newsmastering tools, and these processes can be mostly automated after they set up by Newsmasters. However, definition, location and selection of the information for the News Radars are one area where most of the human skill is required.
According to The Birth of the NewsMaster, newsmastering is
“the ability to concert, orchestrate, edit, and refine quality search formulas that tap into the whole RSS universe and beyond, and that filter out relevant content based on selected keywords, sources, type of content, ranking and many other possible criteria”. Very well said!
It seems that just this ability to concert, orchestrate, edit and refine Web search query and define/locate/select the relevant content for the Web Channels will be a set of skills that sets apart a great Newsmaster from the rest of the pack, and which will be sought after in years to come when a need for processing overabundant “raw” information from the Web is better understood.
Where can you find descent RSS feeds for your Channel? In a nutshell, today there are only three (3) ways to do it:
1) Generate feeds using search queries against multiple web, blog, news, user group, message boards and multimedia search engines (e.g., MSN/Live.com, Technorati, Google News, Google Groups and Video, Boardtracker, Blogdigger) and social networking, bookmaking and publishing sites (del.icio.us, Digg, YouTube, Flickr etc.)
2) Locate “native” RSS feeds available at sites or blogs of your interest
3) Scrape those Web sites or blogs, which do not have the RSS feeds
We will cover all the above ways in our series; however, we would like to start from the keyword-based RSS feed generation, which, at this moment, is a prevalent mode of the web content acquisition despite the growth in a number of “native” RSS feeds.
We will also skip for now the Channel’s subject definition (or refining), which is an initial step in creating professional constantly updated Web channel or “News Radar”. This step is so important, albeit being maybe a far fetching, that it requires a separate article and, therefore, we consciously decided not to follow the Newsmastering “lifecycle” and start from the generating the content, and then move to the subject’s definition/refining.
Most leading search engines provide the RSS output in one form or another although some prominent Web sites have somewhat limited RSS output capabilities (Amazon, del.icio.us, MySpace, Flickr).
However, Google, Yahoo, Live and Ask web search do not provide RSS feeds. There are some hacks and tools, which provide the feeds through the search engine’s APIs. For instance, Ben Hammersley describes how you can generate the feeds for Google searches – Ben Hammersley – Google to RSS.
The oldest keyword-based RSS feed generator is Kebberfegg released by the end of 2005 by a Web Search and SEO pundit Tara Calishain. Kebberfegg stands for (Keyword Based RSS Feed Generator).
Kebberfegg is a tool to help you generate large sets of keyword-based RSS feeds at one time. It gives you one place to set up as many as 64 (?) keyword-based RSS feeds at a time, in yummy HTML or OPML flavors.
Keyword-based feeds are great because they can save you a lot of time by automatically updating search results and sending them to your RSS feed reader. But it can take a lot of time to set up all the keyword-based feeds you might want to use across several different resources.
Enter your query in the box below. Underneath that you’ll have the option to choose for what categories you would like to generate feeds — you may wish to search only blogs, for example, or only news sites. You may choose multiple categories — use your CTRL key to select more than one category.
Beneath that you’ll have the choice to generate an OPML file containing your newly-generated feeds, or an HTML list with a link to the main site, a plain RSS link, and a direct link to add the feed to My Yahoo.
Using the tool is quite simple:
- First, enter your keyword strategy into the search box.
- For this demo, we will use the query “Granada real estate.”
- Then, we select a category or categories of feed sources I would be interested in receiving.
- We decided to select the following categories: Multimedia and News and News Search Engines.
- You can choose the output as a plain RSS link on an HTML or OPML, that you can import into a feed reader
- OK, we chose the HTML option and now have a page of direct RSS links (preconfigured for my search query, ready to be added to my aggregator.
- You’ll see an XML button or buttons to add the feed directly to My Yahoo, Bloglines, MultiRSS or get the feed via E-Mail
If we choose an OPML option, we get an OPML file through the site’s URL:
However, the file can’t be recognized by either FeedDemon or our engine:
Overall, the tool is quite reliable, and has not changed much since we discovered it last year; it seems that Tara just added several sources since then.
Keotag (just recently out of beta state) describes itself as “tag search multiple engines, tag generator and social bookmark links generator”. The tool is a child of Netwizz.net – a Web and Blog Agency from New Caledonia.
The tool has attractive and slick user interface:
Go to Keotag and enter a query; simpler ones are better because you are searching tag sites. Keotag refreshes with a Technorati graph showing you how active that query has been (sometimes the graph is blank) and a set of icons. If you mouseover the icons you’ll get a popup showing you with what search engines they’re affiliated. Many of the search engines will be familiar to you — Technorati, Yahoo, Feedster, Google blog search, etc.
Click on an icon and a preview window will open showing you a summary of the content from that site. For example, click on the Technorati icon and you’ll get a list of Technorati’s tag search result for your query. (Note that the tag search is NOT the blog search. The tag search is in my experience more limited in its results.) You won’t get any additional information that you might get from the search result at the site, like index date, source, or page size. It’s just like getting a headline-only RSS feed. The results from a site also include an XML link to get an RSS feed for that specific query from the site.
Despite Tara’s recommendations, we tried a complicated search query “Granada real estate” and was lucky with Google Blog Search, IceRocket, Live Search (just added lately), Blogdigger, Blogpulse, Yahoo etc.
We are not sure that the above sites are all using tags, but it doesn’t really matter as long one can generate a descent quality feed.
The search output is in RSS and OPML format:
Clicking on the RSS icon brought up a regular RSS feed with URL address which can be copied/pasted into a feed reader of your choice, while clicking on the OPML sign generates an OPML file, which can be saved on your hard drive:
The OPML file is working very well with FeedDemon and other RSS Readers.
Generally, the tool is fun to use but it only provides feeds from 18 sources – search and blog engines, and social sites. It doesn’t have flexibility of Kebberfegg in a sense that one cannot choose the source type (e.g., multimedia or blogs).
TagJag (former Gada.be) is our “weapon of choice” even though we never seen the tool so volatile in terms of quality and usability.
Gada.be launched in October 2005 as an RSS search engine – it brings together results from Feedster, Technorati, Sphere, Tailrank, YouTube and hundreds of other services. It’ also easily accessible from mobile devices. Gada.be’s biggest drawbacks were always the name (great for mobiles, but not great for computer users) and the incomprehensible design – with these issues fixed, it could be a great service.
TagJag is unique in a sense of generating and discovering RSS feeds, which could not be generated and/or discovered otherwise.
The feeds are categorized quite well by their type – for instance, Blogs, Discovery, Jobs, Multimedia, News, etc.
You type in a specific keyword/key phrase, choose the feeds category and press “Find It” button. The feeds icons appear, however, cannot be previewed without pressing the icon (this feature was available before – see our TagJag New Interface posting:
The OPML file feature, which did not work until lately, produces a nice file, which is nicely rendered by FeedDemon RSS reader:
It seems like Chris Pirillo – a guy behind TagJag is perpetually looking for new ways to improve the tool that makes it flaky sometimes.
Regardless, we are heavy-duty users of TagJag and, in our opinion; this tool is a best one amongst this kind that is reflected by its Alexa rankings vs. Keotag vs. Kebberfegg:
Now enter a newcomer Radar Wizard. It is our tool ((RadarFarms.com) and, by definition, we cannot be objective. It is still has modest capabilities comparing to the aforementioned heavyweights (we only generate seven feeds); however, it allows creating, loading and hosting News Radars (Feed Digests) at our site.
All you need to do is specify a keyword or key phrase, which represents your Radars’ subject, choose Feed Type (News, Blogs etc.) or All, generate and preview the feeds, select or unselect them, and then publish your Radar.
We are working on increasing a number of feeds, which can be generated for a specified keywords/key phrase and other unique features, which will be delivered shortly.
Read more about Radar Wizard at: http://www.radarfarms.com/rw_intro.php