Say hello world to comet

A couple of weekends ago I inflicted upon myself a quest to discover what all the buzz was about regarding Comet. What I discovered is that there is quite a bit of code out there to help you get started but the documentation around that code, and about Comet in general, is severely lacking. All I really wanted to find was a Comet-based Hello World, which as any developer knows, is the cornerstone of any programming language or methodology.

Since I couldn’t find one on Google, I ascertained that no Hello World exists for Comet and therefore I took it upon myself to write one.

For those of you who are new to Comet, the first thing you should do is read Alex Russell’s seminal blog post on the topic. At its core, Comet is really just a message bus for browser clients. In one browser, you can subscribe to a message and in another you can publish a message. When a message gets published, every browser that’s subscribed (almost) instantaneously receives it.

What? I thought clients (browsers) had to initiate communication per the HTTP spec. How does this work?

Under the covers, Comet implementations use a little-known feature of some web server implementations called continuations (or hanging gets). I won’t go into details here, but at a high level, a continuation initiates from the browser (as all HTTP requests must do) and then, when it’s received by the server, the thread handling it basically goes to sleep until it gets a message or times out. When it times out, it wakes up and sends a response back to the browser asking for a new request. When the thread on the server receives a message, it wakes up and sends the message payload sent back to the browser (which also implies that it’s time to send a new request). Via this mechanism, HTTP is more or less “inverted” so that the server is essentially messaging the client instead of vice-versa.

A few questions immediately pop into mind, so let’s just deal with them right now:

Why is this better than Ajax alone?

It boils down to latency and users’ tolerance for it. In the worst case, traditional web applications force entire page refreshes. Ajax applications are a little better, because they can refresh smaller parts of a page in response to users’ actions, but the upshot is that the users are still waiting for responses, right? A Comet-driven application has essentially removed the user from the picture. Instead of the user asking for fresh data, the server just sends it along as soon as it changes, given the application more of a “realtime” feel and removing virtually all perceived latency.

So are we back to client server again?

Sort of. Comet gives you the benefit of server-to-client messaging without the deployment issues associated with fat clients.

Can’t applets do this?

Of course they can. But who wants to download an applet when some lightweight Javascript will do the trick?

Why the name Comet?

Well, clearly it’s a pun on Ajax. But it’s not the only name for this sort of technology. There’s something out there called pushlets which claims to do the same thing as Comet, but which didn’t seem to catch on, I guess.

Back to the whole point of this post: my hello world. I pieced this example together using dojo.io.cometd and a recent version of Tomcat that into which I dropped the relevant parts of Jetty to provide support for continuations.

It’s finally time to say “hello world” to my hello world.

First off, download one of the more recent dojo builds that contains support for dojo.io.cometd. Drop dojo.js on your Java-based web/application server. (I used Tomcat, but you can use JBoss, Jetty, Weblogic, Websphere or any other web server with support for servlets.) Add this page in the root of your application:

<script src="js/dojo.js" type="text/javascript"></script>
<script type="text/javascript">//<![CDATA[
  dojo.require("dojo.io.cometd");
  cometd.init({}, "cometd");
  cometd.subscribe("/hello/world", false, "publishHandler");
  publishHandler = function(msg) { alert(msg.data.test); }
// ]]></script>
<input type="button" value="Click Me!" />

Without a cometd-enabled web server behind it, the above page does absolutely nothing.

So, to make this work, I needed to find a Java-based web/application server with support for continuations. I’m sure there are many ways to skin this cat, but I picked Jetty. You can get Jetty source and binaries if you’d like to follow along. Since all of our customers who embrace open source are lightyears more comfortable with Tomcat than they are with any other open source web/application server (ahem . . . Jetty), I decided to embed Jetty in Tomcat rather than run everything on Jetty alone. It’s all just Java, right?

Here I ran into a few snags. The maven build file for Jetty didn’t work for me, so I dropped everything in org.mortbay.cometd and org.mortbay.cometd.filter into my Eclipse project and just integrated it with the ant build.xml I was already using to build my web application. Here’s the relevant portion of my build.xml:

<javac srcdir="${srcdir}" destdir="${classdir}" debug="true" debuglevel="lines,vars,source">
<classpath>
<pathelement location="${jetty.home}/lib/jetty-util-6.0.1.jar"/>
<pathelement location="${jetty.home}/lib/servlet-api-2.5-6.0.1.jar"/>
</classpath>
</javac>

Once Jetty was essentially hacked into Tomcat, the rest was smooth sailing. I just wrote a JSP that dropped a “goodbye world” message onto the same old queue that I used in the last example, but I did so using server-side code. Here’s the JSP:

<%@page import="org.mortbay.cometd.*"%>
<%@page import="java.util.*"%>
<%
Bayeux b = (Bayeux)getServletContext().getAttribute(CometdServlet.ORG_MORTBAY_BAYEUX);
Channel c = b.getChannel("/hello/world");
Map message = new HashMap();
message.put("test", "goodbye world");
c.publish(message, b.newClient());
%>

This page does not produce any output of its own; rather, it just drops the “goodbye world” message on the queue. When you hit this page in a browser, any other browser listening to the /hello/world queue will get the message. The above JSP, along with the dojo page you created in the first step, should be enough to wire together two different flavors of Comet messaging: browser to server to browser and just plain old server to browser.

I’m curious 1) if this was helpful and 2) if you’d like to share what you’re doing with Comet with me (and please don’t say cleaning your kitchen).

26 Replies to “Say hello world to comet”

  1. Hey Chris,

    I hope you read this as I really need help.

    I have been trying to hack jetty into tomcat but to no avail.
    I guess lots of packages have changed and things have been updates since 2008.

    Would you please help me figure out / or a simple update to the tutorial on how to integrate jetty in tomcat (step by step) using eclipse would be really nice.

    thanks a ton.

    (I have taken this up for my major project in my university and would be glad if you could help me do this using Eclipse)

  2. Who knows where to download XRumer 5.0 Palladium?
    Help, please. All recommend this program to effectively advertise on the Internet, this is the best program!

  3. Hey Andrew,

    I checked your comments

    1-My dojo.js includes dojo.io.cometd
    2- I load the dojo page first, then load the jsp page to send the message (as you have instructed). Nothing is received on dojo side
    3-WEB-INF/lib includes jetty and jetty-util jars, WEB-INF/classes includes org.mortbay.cometd package and its files

    I have no clue further. Maybe you can zip/war your helloworld example and send it to me?

  4. Engin,

    As I was further considering your problem, something crossed my mind. If you’re trying to Chris’ server-to-client cometd example, you shouldn’t be clicking the button. The dojo client in that case is waiting for a cometd message from the server-side jsp. So, you would load the dojo page first, then load the jsp page to send the message.

    I’m not sure about this, but I think this could cause a problem. If you load the client-side dojo page first, it tries to subscribe to the “/hello/world” channel. If this also creates the channel if it does not exist, then there should be no problem when the jsp page loads. That is, the jsp will use getChannel(“/hello/world”) and in fact acquire a non-null object. But what if the client subscribing to the channel did not create it?

    If the client did not create it, then the jsp will not find the channel, and will create it itself. It will then send the message over the channel. But since the client did not create the channel, it therefore subscribed to nothing. So, no matter how often you reload the jsp to resend the message, the client never receives it. Refreshing the dojo client, however, should cause it to properly subscribe to the now-operational channel.

    I have never tried sending a message from the client to the server. If you are hoping that by pressing the button on the dojo client that the jsp page will receive the message, you will have to change Chris’ code above. The reason for this is that the jsp page does not subscribe to the channel, it only publishes messages over it. Change the code to subscribe to the channel as well. This should allow you to click the client-side button and have the data reach the server-side jsp.

  5. Engin,

    I would start by making sure that your dojo client-side setup is working properly. When I first tried, I was getting javascript errors left and right – but I was looking at server-side stuff in an attempt to solve it. I should have been looking at my client-side configuration. There are many different dojo builds, with essential code often being found in different locations. Thus, make sure dojo can find the files and libraries it is looking to utilize.

    Regards,
    Andrew

  6. I am getting the Bayeux object in the JSP file and creating a new channel if none is available:

    Channel c = b.getChannel(“/hello/world”);
    if(c==null)
    c = b.newChannel(“/hello/world”);

    However, if I open the page with dojo and click the button, nothing happens.

    I have the following in my lib directory:

    cometd-6.1.1.jar
    jetty-6.1.1.jar
    jetty-util-6.1.1.jar

    I have BayeuxStartupListener.class under classes directory.

    My web.xml is the same as Andrew Bays (see previous comments).

    any ideas?

  7. Hi Chris,

    I have been extending your code so that I can capture messages sent down channels on the server side. I also found that when I dropped my webapp into Jetty, it just worked… it is surprisingly similar to Tomcat… (logging is a bit more tricky- you need to ‘configure’ it which is boring). One problem I have met is with multiple clients… If I open up the hello world example in multiple browsers, it still works, however, the Post only returns when the “meta/reconnect” channel goes down, and then comes up again (so sometimes it can take up to 45 seconds). Did your implementation do the same thing??? I thought it might be a Tomcat problem (since it is lacking the asynchronous features), but the same happens in Jetty 😐

    Will

  8. I have configured my servlet to prime the Channel if it does not already exist. If you try to do a getChannel() and it hasn’t been created, you run into a problem. You could just tell the servlet to call newChannel() in this case, which is what I do. It seems that the dojo.io.cometd library is a bit more dynamic in that if you try to subscribe or publish to a channel that has not been created (initialized), it will automatically create it for you.

  9. Here is my servlet configuration in web.xml – I hope this is helpful in regards to the question about web.xml’s contents:


    cometd
    org.mortbay.cometd.CometdServlet
    filters /WEB-INF/filters.json

    timeout 15000

    multi-timeout 1500

    1

  10. Hello,

    The cometd client-side Javascript is of course found in Dojo. Dojo builds are available here: [url]http://download.dojotoolkit.org/release-0.4.1/[/url]

    To acquire the server-side cometd library, please visit [url]http://dist.codehaus.org/jetty/[/url]. If you decide not to use Jetty as your server, you will need to take the org.mortbay.cometd library from th extras directory and include it in the source for your project.

    Regards,
    Andrew

  11. Since this is the only resource I found that combines tomcat with comet, I thought I’d add a few comments that may help others… The javascript line cometd.init({}, “cometd”); should be redirected (using the web.xml file) to refer to org.mortbay.cometd.CometdServlet .

    If you wanted to use a good old fashioned servlet, then you need a doGet *and* doPost method (each method doing exactly the same as the JSP-got stuck on that for a while!)…

    You also need to load up the web page and run the javascript by pressing the button (which initialises the Channel of communication) before you load the jsp/servlet.

    Will

  12. I really can’t get your example to work… I have tried everything. What did you put in your web.xml file??? Surely you would have had to initialise the CometdServlet? Currently, I get an error on the line

    Bayeux b = (Bayeux)getServletContext().getAttribute(CometdServlet.ORG_MORTBAY_BAYEUX);

    As the servlet cannot find the attribute with that name. The attributes that are available are as follows:

    org.apache.catalina.jsp_classpath
    javax.servlet.context.tempdir
    org.apache.catalina.resources
    org.apache.catalina.WELCOME_FILES

    Any ideas/further clarification of your implementation would be much appreciated…

    Alex

  13. Several people have reported problems getting this example to work, so I’ve taken it upon myself to revise it appropriately. I think the main issue was that you need BOTH a client and server in order to get Comet-style messaging working. The revised blog post walks you through the whole setup before you attempt to send any test messages using either browser to server to browser messaging or server to browser messaging.

    Please, if you are so inclined, run these instructions and let me know if there are still any leftover kinks.

    Thanks!

  14. Thanks for your comment, Jan. The maven build that failed was actually for the cometd source that I got from http://svn.xantus.org/shortbus/trunk/cometd-java.

    Truth be told, it was just easier for me to build the code myself using the ant build.xml that I had already created for my project than to diagnose the problems with the maven build.

    But, if you want to know, here’s the exact error I got:

    [INFO] Trace org.apache.maven.lifecycle.LifecycleExecutionException: Cannot execute mojo: resources. It requires a project with an existing pom.xml, but the build is not using one.

    It was just easier, for the purposes of this hello world, to compile Jetty’s servlet-based cometd implementation into Tomcat than to figure out how to build and run Jetty. However, given what you said about performance and scale-ability, I realize that we should do some serious performance testing on Jetty to see how continuations can help. Yes, I knew about continuations and the performance implications of not having them, but for the prototype — as you correctly surmised — I didn’t care.

    On a related note, some people have asked me offline what on earth we’re doing with comet and Plumtree. The answer is: you’ll know when we launch it. 😉

  15. Chris,

    I’d be interested to know what aspect of the Jetty build did not work for you.

    Also, bear in mind that Jetty has the Continuations
    mechanism, which means that an Ajax application will scale very well under load in Jetty. The link above illustrates the load pattern imposed by Ajax apps with some pretty interesting figures. Of course, for a “Hello World” type demo, load is pretty irrelevant 🙂

  16. Something I neglected to mention — I got the latest version of the cometd source from:

    svn://svn.xantus.org/shortbus/trunk

    I didn’t modify the source at all — I just dropped it into my Eclipse project and compiled it against the Jetty 6.0.1 jars. Right now my build puts the cometd classes in WEB-INF/classes and the jars in WEB-INF/lib, just like any Java web application. It shouldn’t make any difference if you war it or ear it up and drop it into jboss or any Java application server — there’s no reason AFAIK that your web server has to be Jetty.

  17. sorry to bother you again.

    But….

    With the jetty Code I’m looking at doesn’t have a ‘newClient’ method.

    What version were you using?

    I’m using jetty-6.0.1.zip.

    Jeff Porter

  18. Great article Chris.

    Just working my through the demo stuff now.

    and …

    1) YEP, VERY!

    2) Will do. Just need clear it with my boss.

    Jeff Porter

Leave a Reply