Categories
Feedhaus

Welcome to Feedhaus

feedhaus_public_alphaWelcome to the Feedhaus blog!

Several of us at bdg are working hard to bring you a next-generation, Web 2.0 news site that will change the way the world views news by always keeping you in the know about what’s hot now. As we’re preparing for launch, which is scheduled for the end of Q3 2007, we thought we’d give you a little taste of what’s to come.

First, some background:

Feedhaus is a concept that I dreamed up in the middle of 2006. It spawned from my desire to have a place where I could go and find out what’s going on right now so that I could “scoop” my friends and coworkers with breaking news before they found out about it. My options right now are limited. There’s Google News, which is pretty good for mainstream headlines. There’s Digg, which is good for niche news and speciality/weird items. There’s a few creative takes on news aggregation, like Marumushi’s News Map and Original Signal. There’s also a slew of feed aggregators; however, all news aggregators focus on the individual (like Google Reader) and not the community (like del.icio.us).

What if you could combine the convenience and power of news aggregation with a user-driven folksonomy to classify the news?

Then, unlike Digg and del.icio.us — which are solely based on user input to classify and popularize information — the relevance of user-classified news would change based on real-world events, not on Diggs or other end-user actions. And what if you could see the lifecycle of news stories waxing and waning in popularity and relevance in real-time, without ever hitting the refresh button? Enter Feedhaus. . . .

Recent changes in the way content gets delivered on the web, along with some slick technologies (Rome, Comet and Lucene to name three of them) and some creative coding by bdg-ers Chris Bucchere and Andrew Bays, make all of this possible — even, dare I say, easy. Nearly every news site, blog and most Web 2.0 sites (including all the sites referenced in this post), expose their content through structured data feeds using RSS/RDF or Atom. Feedhaus allows users to classify feeds from any source and of any format with tags, much like del.icio.us or Flickr.

But, unlike those sites, which allow users to tag static content, when you tag a feed on Feedhaus, it’s as though you’re tagging a living news source that’s constantly growing and changing.

Imagine a tag cloud where the tags actually grow and shrink based on real-world events, all powered by background agents that are constantly checking feeds for newly added content. Then, when you click on a tag, a tag-specific page appears, showing a realtime-updated list of articles aggregated from all the feeds associated with that tag along with a Flickr photo badge and a YouTube video stream with images and video, respectively, matching that tag. Now, you’re beginning to understand Feedhaus.

Here at bdg we have a lot of other ideas about features for Feedhaus and we’re struggling to cut out all the fat and launch just “the right” number of features to give me — and all our users — exactly what they need: a single place to find out what’s hot now.

If you’re interested in participating in our private alpha, please e-mail us. (We won’t use your e-mail address for anything other than to notify you about the beta and make other Feedhaus-related announcements.)

Categories
Featured Posts Software Development

Say hello world to comet

A couple of weekends ago I inflicted upon myself a quest to discover what all the buzz was about regarding Comet. What I discovered is that there is quite a bit of code out there to help you get started but the documentation around that code, and about Comet in general, is severely lacking. All I really wanted to find was a Comet-based Hello World, which as any developer knows, is the cornerstone of any programming language or methodology.

Since I couldn’t find one on Google, I ascertained that no Hello World exists for Comet and therefore I took it upon myself to write one.

For those of you who are new to Comet, the first thing you should do is read Alex Russell’s seminal blog post on the topic. At its core, Comet is really just a message bus for browser clients. In one browser, you can subscribe to a message and in another you can publish a message. When a message gets published, every browser that’s subscribed (almost) instantaneously receives it.

What? I thought clients (browsers) had to initiate communication per the HTTP spec. How does this work?

Under the covers, Comet implementations use a little-known feature of some web server implementations called continuations (or hanging gets). I won’t go into details here, but at a high level, a continuation initiates from the browser (as all HTTP requests must do) and then, when it’s received by the server, the thread handling it basically goes to sleep until it gets a message or times out. When it times out, it wakes up and sends a response back to the browser asking for a new request. When the thread on the server receives a message, it wakes up and sends the message payload sent back to the browser (which also implies that it’s time to send a new request). Via this mechanism, HTTP is more or less “inverted” so that the server is essentially messaging the client instead of vice-versa.

A few questions immediately pop into mind, so let’s just deal with them right now:

Why is this better than Ajax alone?

It boils down to latency and users’ tolerance for it. In the worst case, traditional web applications force entire page refreshes. Ajax applications are a little better, because they can refresh smaller parts of a page in response to users’ actions, but the upshot is that the users are still waiting for responses, right? A Comet-driven application has essentially removed the user from the picture. Instead of the user asking for fresh data, the server just sends it along as soon as it changes, given the application more of a “realtime” feel and removing virtually all perceived latency.

So are we back to client server again?

Sort of. Comet gives you the benefit of server-to-client messaging without the deployment issues associated with fat clients.

Can’t applets do this?

Of course they can. But who wants to download an applet when some lightweight Javascript will do the trick?

Why the name Comet?

Well, clearly it’s a pun on Ajax. But it’s not the only name for this sort of technology. There’s something out there called pushlets which claims to do the same thing as Comet, but which didn’t seem to catch on, I guess.

Back to the whole point of this post: my hello world. I pieced this example together using dojo.io.cometd and a recent version of Tomcat that into which I dropped the relevant parts of Jetty to provide support for continuations.

It’s finally time to say “hello world” to my hello world.

First off, download one of the more recent dojo builds that contains support for dojo.io.cometd. Drop dojo.js on your Java-based web/application server. (I used Tomcat, but you can use JBoss, Jetty, Weblogic, Websphere or any other web server with support for servlets.) Add this page in the root of your application:

<script src="js/dojo.js" type="text/javascript"></script>
<script type="text/javascript">//<![CDATA[
  dojo.require("dojo.io.cometd");
  cometd.init({}, "cometd");
  cometd.subscribe("/hello/world", false, "publishHandler");
  publishHandler = function(msg) { alert(msg.data.test); }
// ]]></script>
<input type="button" value="Click Me!" />

Without a cometd-enabled web server behind it, the above page does absolutely nothing.

So, to make this work, I needed to find a Java-based web/application server with support for continuations. I’m sure there are many ways to skin this cat, but I picked Jetty. You can get Jetty source and binaries if you’d like to follow along. Since all of our customers who embrace open source are lightyears more comfortable with Tomcat than they are with any other open source web/application server (ahem . . . Jetty), I decided to embed Jetty in Tomcat rather than run everything on Jetty alone. It’s all just Java, right?

Here I ran into a few snags. The maven build file for Jetty didn’t work for me, so I dropped everything in org.mortbay.cometd and org.mortbay.cometd.filter into my Eclipse project and just integrated it with the ant build.xml I was already using to build my web application. Here’s the relevant portion of my build.xml:

<javac srcdir="${srcdir}" destdir="${classdir}" debug="true" debuglevel="lines,vars,source">
<classpath>
<pathelement location="${jetty.home}/lib/jetty-util-6.0.1.jar"/>
<pathelement location="${jetty.home}/lib/servlet-api-2.5-6.0.1.jar"/>
</classpath>
</javac>

Once Jetty was essentially hacked into Tomcat, the rest was smooth sailing. I just wrote a JSP that dropped a “goodbye world” message onto the same old queue that I used in the last example, but I did so using server-side code. Here’s the JSP:

<%@page import="org.mortbay.cometd.*"%>
<%@page import="java.util.*"%>
<%
Bayeux b = (Bayeux)getServletContext().getAttribute(CometdServlet.ORG_MORTBAY_BAYEUX);
Channel c = b.getChannel("/hello/world");
Map message = new HashMap();
message.put("test", "goodbye world");
c.publish(message, b.newClient());
%>

This page does not produce any output of its own; rather, it just drops the “goodbye world” message on the queue. When you hit this page in a browser, any other browser listening to the /hello/world queue will get the message. The above JSP, along with the dojo page you created in the first step, should be enough to wire together two different flavors of Comet messaging: browser to server to browser and just plain old server to browser.

I’m curious 1) if this was helpful and 2) if you’d like to share what you’re doing with Comet with me (and please don’t say cleaning your kitchen).