Letter of Commendation from NAFSA

We received a letter of commendation from one of our newer customers, The National Arts Festival of South Africa. Tony agreed to allow us to post it here on our blog.

NAFSA_LetterheadChris Bucchere
Social Collective
13800 Coppermine Road
Herndon, VA 20171
USA

7 May 2009

Dear Chris,

Just a note of sincere thanks to you and your team at Social Collective for the highly professional way in which you’ve approached the implementation of your tool on our website.

Undeterred by the fact that his client is several thousand miles and timezones away, Mike has done a great and very patient job of attending to the implementation and the multitude of requests for tweaks and changes from us.

The end result is great and in the couple of days the site has been live the reaction has been overwhelmingly positive.

You’ve got a great product and a great team – we’re very glad to have encountered both!

Thank you.

Regards,

Tony Lankester
CEO: National Arts Festival, South Africa

SXSW Launches my.SXSW Powered by The Social Collective

sxsw2009Austin, Texas – February 2nd, 2009 – Today South By Southwest (SXSW), announced the launch of my.SXSW, the official social networking and scheduling tool for the 2009 conferences and festivals. Using my.SXSW, attendees can access the site to interact with one another, build their personalized conference schedules, join exclusive groups and form lasting relationships with other attendees. The social networking software was developed by SXSW and BDG, an industry-leading provider of a white-label conference social networking solution called The Social Collective. my.SXSW, powered by The Social Collective, represents a groundbreaking new way for SXSW attendees to enhance their offline conference experience through socializing online.

“We’ve always wanted to provide a space online for registrants to network before, during and after the ten days in March that they spend in Austin,” said Scott Wilcox, CTO of SXSW, Inc. “This year we teamed up with BDG to implement The Social Collective as my.SXSW, our new online registrant directory and social network. In addition to our collaboration with an outside team, we have tried to use the feedback we’ve received over the years to provide social networking opportunities for SXSW registrants. BDG and SXSW put a lot of thought into my.SXSW, resulting in a user experience we know will be a hit with our community.”

my.SXSW offers several new benefits to conference registrants, many of which are being introduced for the first time at SXSW 2009:

—Personalized and shareable event schedules
—Badge photo and avatar uploading and editing
—Rich media event pages with MP3 downloads, photos, band and speaker bios, etc.
—Seamless integration with Twitter, Flickr and Facebook
—Interest groups with integrated messaging
—Event/Schedule change notifications
—Social Networking (public profile pages, follow people and be followed)
—Private and public messaging on events, in groups and everywhere else
—Intuitive mobile device experience (to be launched just prior to the event)

For filmmakers looking for distribution, bands looking for producers, small businesses looking to develop their brands or digital creatives looking for new collaborators, my.SXSW provides a unique opportunity for people from all corners of the globe to get together and communicate before, after and during the event.

“We’re excited to see The Social Collective in use at SXSW,” said Chris Bucchere, CEO of BDG. “SXSW has come to be known as a trendsetting conference, not just for music and film, but for technology as well. The market for growing communities around conferences and events is just beginning to develop and of course SXSW is setting a great example of how to enhance people’s experience at their event using the right combination of web, e-mail, SMS and mobile tools.”

The Social Collective’s rich registration and schedule APIs enable seamless integration with the SXSW registrant and schedule data stores. As a result, there is no additional sign up or site registration needed. my.SXSW is open only to Film, Interactive, Music, Gold and Platinum SXSW registrants. Sign-in instructions will be sent via e-mail shortly after payment is confirmed and registration is processed by SXSW.

About SXSW

MUSIC AND MEDIA CONFERENCE features a legendary festival showcasing more than 1,800 musical acts of all genres from around the globe on over eighty stages in downtown Austin. By day, the Austin Convention Center comes alive with conference registrants doing business in the Trade Show and partaking of a full agenda of informative, provocative panel discussions featuring hundreds of speakers of international stature. In its 23rd year, SXSW remains an essential event on the music industry calendar. SXSW Music takes place March 18-22, 2009.

The SXSW FILM CONFERENCE AND FESTIVAL explores all aspects of the art and business of independent filmmaking. The Conference hosts a five-day adventure in the latest filmmaking trends and new technology, featuring distinguished speakers and mentors. The internationally-acclaimed, nine-day Festival boasts some of the most wideranging programming of any US event of its kind, from provocative documentaries to subversive Hollywood comedies, with a special focus on emerging talents. SXSW Film takes place March 13-21, 2009

The SXSW INTERACTIVE FESTIVAL celebrates the creativity and passion behind the coolest new media technologies. In addition to panel sessions that cover everything from web design to bootstrapping to social networks, attendees make new business connections at the three-day Trade Show & Exhibition. The newest element of the event is ScreenBurn at SXSW, which adds specific gaming industry programming as well as a three-day Arcade to the mix. SXSW Interactive takes place March 13-17, 2009

Find out more at http://www.sxsw.com

About The Social Collective

The Social Collective provides a fun and interactive means for conference attendees to meet and network with one another in a safe and secure environment before, during and after any conference or event. It improves attendees’ conference experiences and gives conference organizers more happy and loyal customers. The white-label product/service offering includes branding of the conference social network, integration with existing registration and e-commerce systems, data migration, site archival and Twitter, Flickr and Facebook integration.

Find out more at http://thesocialcollective.com

For more information, contact Elizabeth Derczo, SXSW, at [email protected], 512/467-7979 ext. 209 or Jennifer Dunleavey at The Social Collective, [email protected], 703/234-7910

Top Ten Tips for Writing Plumtree Crawlers that Actually Work

Just in time for Halloween, I’ve decided to publish my Top Ten Tips for Writing Plumtree Crawlers that Actually Work. This post may scare you a little bit, but hey, that’s the spirit of Halloween, right?

[Editor’s note: yes, we’re still calling it Plumtree. Why? I did a Google search today and 771,000 hits came up for “plumtree” as opposed to around 300,000 for “aqualogic” and just over 400,000 for “webcenter.” Ignoring the obvious — that a short, simple name always wins over a technically convoluted one — it just helps clarify what we’re talking about. For example, if we say “WebCenter,” no one knows whether we’re talking about Oracle’s drag-n-drop environment for creating JSR-168 portlets (WebCenter Suite) or Plumtree’s Foundation/Portal (WebCenter Interaction). So, frankly, you can call it whatever you want, but we’re still gonna call it Plumtree so that people will know WTF we’re talking about.]

So, you want to write a Plumtree Crawler Web Service (CWS), eh?

Here are ten tips that I learned the hard way (i.e. by NOT doing them):

1. Don’t actually build a crawler
2. If you must, at least RTFM
3. “Hierarchyze” your content
4. Test first
5. When testing, use the Service Station 2.0 (if you can get it)
6. Code for thread safety
7. Write DRY code (or else)
8. Don’t confuse ChildDocument with Document
9. Use the named methods on DocumentMetaData
10. RTFM (again)

Before I get into the gory details, let me give you some background. First off, what’s a CWS anyway? It’s the code behind what Oracle now calls Content Services, which spider through various types of content (for lack of a better term) and import pointers to those bits of content into the Knowledge Directory. This ability to spider content and normalize its metadata is one of the most underrated features in Plumtree. (FYI, it was also the first feature we built and arguably, the best.)

Each bit of spidered content is called a Document or a Card or a Link depending on whether you’re looking at the product, the API or the documentation, respectively. It’s important to realize that CWSs don’t actually move content into Plumtree; rather, they store only pointers/links and metadata and they help the Plumtree search engine (known under the covers as Ripfire Ignite) build its index of searchable fulltext and properties.

Today, Plumtree ships with one OOTB CWS that knows how to crawl/spider web pages. Not surprisingly, it’s known as the Web Crawler. Don’t let the name mislead you: the web crawler can actually crawl almost anything, as I explain in my first tip, which is:

Don’t actually build a crawler.

But I’m getting ahead of myself.

So, back to the background on crawlers. Oracle ships five of ’em, AFAIK: one for Windows files, one for Lotus Notes databases, one for Exchange Public Folders, one for Documentum and one for Sharepoint. Their names give you blatantly obvious hints at what they do, so I won’t get into it. Along with the OOTB crawlers, Oracle also exposes a nice, clean API for writing crawlers in Java or .NET. (If you really want to push the envelope, you can try writing a crawler in PHP, Python, Ruby, Perl, C++ or whatever, but it’s hard enough to write one in Java or .NET, so I wouldn’t go there. If you do, though, make sure that your language has a really good SOAP stack.)

So, after reading this, you still want to write a crawler, yes?

Let’s get into my Top Ten Tips:

1. Don’t actually build a crawler

Yes, you really don’t want to go here. Building crawlers is not that hard, as there’s a clean, well documented API. However, getting them work is a whole other story.

Most applications these days have a web UI. So, take advantage of it. Point the OOTB web crawler at the web UI and see what it does. Some web UIs will work well, other won’t (particularly if they use frames or lots of javascript.)

Let’s assume for a moment that this technique doesn’t work. Or perhaps you’re dealing with some awful client-server or greenscreen POS that doesn’t have a web UI. Either way, you may still be able to use the web crawler.

How? Well, the web crawler can crawl almost anything using something that we used to call the Woogle Web. Think of it this way. Say you want to crawl a database. Perhaps that database contains bug reports. Rather than waste your time trying to write a database crawler, just write an index.jsp (or .php, .aspx, .html.erb, .you-name-it) page that does something like select id from bugs and dumps out a list of all the bugs in the database. Then, hyperlink each one to a detail page (that’s essentially a select * from bugs where id = ? query). Your index page can be sinfully ugly. However, put some effort into your detail pages, making them look pretty AND using meta tags to map each field in the database to its value.

Then, simply point the OOTB web crawler at your index page, tell it not to import your index page, map your properties to the appropriate meta tags, crawl at a depth of one level, and get yourself a nice cup of coffee. By the time you get back, the OOTB web crawler will have created documents/links/cards for every bug with links to your pretty detail pages and every bit of text will be full-text searchable. So will every property, meaning that you can run an advanced search on bug id, component, assignee, severity, etc.

At Plumtree, we used to call this a Woogle Web. It may sound ridiculous, but

a Woogle Web is a great way to crawl virtually anything without lifting a finger.

However, a Woogle Web won’t work for everything. If you’re dealing with a product where you can’t even get your head around the database schema AND you have a published Java, .NET or Web Service API, then you might want to think about writing a custom crawler.

2. If you must, at least RTFM

If you’re anything like me, reading the manual is what you do after you’ve tried everything else and nothing has worked out. In the case of Plumtree crawlers, I recommend swallowing your pride (at least momentarily) and reading all of the documentation, including their “tips” (which are totally different from and not nearly as entertaining as my tips, but equally valuable).

Once you’re done reading all the documentation, you might also want to consult Tip #10.

3. “Hierarchyze” your content

Um, yeah, I know “hierarchyze” isn’t a word. But since crawlers only know how to crawl hierarchies, if your data aren’t hierarchical, you darn well better figure out how to represent them hierarchically before you start writing code. Even if you don’t think this step is necessary, just do it because I said so for now. You’ll thank me later.

4. Test first

Don’t even try to write your crawler and then run it and expect it to work. Ha! Instead, write unit tests for every modular bit of code that you throw down. To every extent that’s it’s possible, write these tests first. It’ll save your butt later.

5. When testing, use the Service Station 2.0 (if you can find it)

When you do finally get around to integration testing your crawler, it’ll save you a lot of time if you use the Service Station 2.0. However, it may take you a long time to get it, so start the process early.

Unlike every product Oracle distributes, Service Station is 1) free and 2) not available for download. Yes, you read that correctly.

To get it, you need to contact support. I called them and told them I needed it and after two weeks I got nothing but a confused voicemail back saying that support doesn’t fulfill product orders. Um, yeah. So I called back and literally begged to talk to someone who actually knew what this thing was. Then I painstakingly explained why I couldn’t get it from edelivery (because it’s not there) nor from commerce.bea.com (because it’s permanently redirecting to oracle.com) nor from one.bea.com (because there it says to contact support). So, after my desperate pleas and 15 calendar days of waiting, I got an e-mail with an FTP link to download the Service Station 2.0.

After installing this little gem, my life got a lot easier. Now instead of testing by launching a Plumtree job to kick off the crawler (and then watching it crash and burn), I could use the Service Station to synchronously invoke each method on the crawler and log the results.

Another handy testing tool is the PocketSOAP TCPTrace utility. (It’s also very handy for writing Plumtree portlets.) You can set it up between the Service Station and your CWS and watch the SOAP calls go back and forth in clear text. Very nice.

6. Code for thread safety

So, as the documentation says (and as I completely ignored), crawlers are multithreaded. The portal will launch several threads against your code and, unless you code for thread safety, these threads will proceed to eat your lunch.

Coding for threadsafety means not only that you need to synchronize access to any class-level variables, but also that you must use only threadsafe objects (e.g. in Java, use ArrayList instead of Vector).

7. Write DRY code (or else)

Even though you’re probably writing your CWS in Java or .NET, stick to the ol’ Ruby on Rails adage:

Don’t Repeat Yourself.

Say for example, that you need to build a big ol’ Map of all the Document objects in order to retrieve a document and send its metadata back to the mothership (Plumtree). It’s really important that you don’t build that map every time IDocumentProvider.attachToDocument is called. If you do, your crawler is going to run for a very very very long time. Crawlers don’t have to be super fast, but they shouldn’t be dogshit slow either.

As a better choice, build the Map the first time attachToDocument is called and store it as a class-level variable. Then, with each successive call to attachToDocument, check for the existence of the Map and, if it’s already built, don’t build it again. And don’t forget to synchronize not only the building of the Map, but also the access to the variable that checks whether the Map exists or not. Like I said, this isn’t a walk in the park. (See Tip #1.)

8. Don’t confuse ChildDocument with Document

IContainer has a getChildDocuments method. This, contrary to how it looks on the surface, does not return IDocument objects. Instead, it expects you to return an array of ChildDocument objects. These, I repeat, are not IDocument objects. Instead, they’re like little containers of information about child documents that Plumtree uses so that it knows when to call IDocumentProvider.attachToDocument. It is that call (attachToDocument) and not getChildDocuments that actually returns an IDocument object, where all the heavy lifting of document retrieval actually gets done.

You may not understand this tip right now, but if drop by and read it again after you’ve tried to code against the API for a few hours, and it should make more sense.

9. Use the named methods on DocumentMetaData

This one really burned me badly. I saw that DocumentMetaData had a “put” method. So, naturally, I called it several times to fill it up with the default metadata. Then I wasted the next two hours trying to figure out why Plumtree kept saying that I was missing obvious default metadata properties (like FileName). The solution? Call the methods that actually have names like setFileName, setIndexingURL, etc. — don’t use put for standard metadata. Instead, only use it for custom metadata.

10. RTFM (again)

I can’t stress the importance of reading the documentation enough.

If you think you understand what to do, read it again anyway. I guarantee that you’ll set yourself up for success if you really read and thoroughly digest the documentation before you lift a finger and start writing your test cases (which of course you’re going to write before you write your code, right?).

* * *

As always, if you get stuck, feel free to reach out to the Plumtree experts at bdg. We’re here to help. But don’t be surprised if the first thing we do is try to dissuade you from writing a crawler.

Have a safe and happy Halloween and don’t let Plumtree Crawlers scare you more than they should.

Boo!

Nobody’s Gonna Read This (and Why That Makes Me Happy)

Boy do I love the fact that no one reads this blog. And to the few people who are exceptions to that general rule — thank you for being so supportive!

I just hit two or three web pages in a row (TechCrunch, Digg and the Meebo blog) wherein each post I read had 80+ comments that reminded me why I rarely ever actually read comments.

Haters, trolls, flamers, spammers — whatever you want to call them, the internet is ridden with people who are filled with spite and rage. The funny thing is that in no other forum (except for perhaps while driving) are people this cruel to one another. It’s just not socially acceptable.

I realize that e-hate isn’t a new problem: in fact, it dates back to the early days of UseNet, Netiquette and the ol’ “do we allow AOLer’s on the internet” debate. While doing some fact-checking on wikipedia, I was really amused to read about Godwin’s Law, which sums up what I’m talking about better than I ever could: “As a Usenet discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches one.”

We all know the Kathy Sierra story. I’m glad she had a thick enough skin to re-emerge in the blogging world and on Twitter because the world is a better place with her contributions than it is without them.

We all remember The Great Sarah Lacy Twitter Massacre of SXSW 2008. I recently met Sarah at a tech event in DC and, believe it or not, she doesn’t have horns, literally or figuratively.

Jason Calacanis recently “retired” from blogging. When I read his post, I immediately thought that it was just a PR stunt, but I’m beginning to realize that I can sympathize with his viewpoint. I really don’t want to ever be an A-list blogger or “internet famous” because it’s just like painting a big target on your own ass.

I love my family and close friends, I love the physical neighborhood in which I live and I love the virtual networks that have developed around my career and my passions for the past 15 years or so that I’ve been using the internet.

But honestly, a big part of me doesn’t want anyone else to read this. Not because I don’t take criticism well. (I don’t, but then again nobody does.) I just wish some of the same general rules that apply to social interactions — at say, a cocktail party, a baseball game or at the supermarket — would apply to the internet.

Comments welcome. Just be nice, ok?

Write an ALUI IDS in Under 15 Lines Using Ruby on Rails

Not only is it possible to write an ALUI Identity Service in Ruby on Rails, it’s remarkably easy. I was able to do the entire authentication part in fewer than 15 lines of code! However, I ran into problems on the synchronization side and ended up writing that part in Java. Read on for all the gory details.

As part of building the suite of social applications for BEA Participate 2008, we’re designing a social application framework in Ruby on Rails and integrating it with ALI 6.5. Not being a big fan of LDAP, I decided to put the users of the social application framework in the database (which is MySQL). Now, when we integrate with ALI, we need to sync this user repository (just as many enterprises do with Active Directory or LDAP).

So I set out to build an IDS to pull in users, groups and memberships in Ruby on Rails.

It’s pretty obvious that Ruby on Rails favors REST over SOAP for their web service support. However, they still support SOAP for interoperability and it mostly works. I did have to make one patch to Ruby’s core XML processing libraries to get things humming along. I haven’t submitted the patch back to Ruby yet, but at some point I will. Basically, the problem was that the parser didn’t recognize the UTF-8 encoding if it was enclosed in quotes (“UTF-8”). This patch suggestion guided me in the right direction, but I ended up doing something a little different because the suggested patch didn’t work.

I changed line 27 of lib/ruby/1.8/rexml/encoding.rb as follows:

 enc = enc.nil? ? nil : enc.upcase.gsub('"','') #that's a double quote inside single quotes

Now that Ruby’s XML parser recognized UTF-8 as a valid format, it decided that it didn’t support UTF-8! To work around this, I installed iconv, which is available for Windows and *nix and works seamlessly with Ruby. In fact, after installation, all the XML parsing issues went bye-bye.

Now, on to the IDS code. From your rails project, type:

ruby script/generate web_service Authenticate

This creates app/apis/authenticate_api.rb. In that file, place the following lines of code:

class AuthenticateApi < ActionWebService::API::Base
 api_method :Authenticate, :expects => [{:Username =>
:string}, {:Password =>
:string}, {:NameValuePairs =>
[:string]}], :returns =>
[:string]
end

All you’re doing here is extending ActionWebService and declaring the input/output params for your web service. Now type the following command:

ruby script/generate controller Authenticate

This creates the controller, where, if you stick with direct dispatching (which I recommend), you’ll be doing all the heavy lifting. (And there isn’t much.) This file should contain the following:

class AuthenticateController < ApplicationController
 web_service_dispatching_mode :direct
 wsdl_service_name 'Authenticate'
 web_service_scaffold :invoke

 def Authenticate(username, password, nameValuePairs)
   if User.authenticate(username, password)
     return ""
   else
     raise "-102" #generic username/password failure code
   end
 end
end

Replace User.authenticate with whatever mechanism you’re using to authenticate your users. (I’m using the login_generator gem.) That’s all there is to it! Just point your AWS to http://localhost:3000/authenticate/api and you’re off to the races.

Now, if you want to do some functional testing (independently of the portal), rails sets up a nice web service scaffold UI to let you invoke your web service and examine the result. Just visit http://localhost:3000/authenticate/invoke to see all of that tasty goodness.

There you have it — a Ruby on Rails-based IDS for ALUI in fewer than 15 lines of code!

The synchronization side of the IDS was almost just as simple to write, but after countless hours of debugging, I gave up on it and re-wrote it in Java using the supported ALUI IDK. Although I never could quite put my finger on it, it seemed the problem had something to do with some subtleties about how BEA’s XML parser was handing UTF-8 newlines. I’ll post the code here just in case anyone has an interest in trying to get it to work. Caveat: this code is untested and currently it fails on the call to GetGroups because of the aforementioned problems.

In app/apis/synchronize_api.rb:

class SynchronizeApi < ActionWebService::API::Base
 api_method :Initialize, :expects =>
[{:NameValuePairs =>
[:string]}], :returns =>
[:integer]
 api_method :GetGroups, :returns =>
[[:string]]
 api_method :GetUsers, :returns =>
[[:string]]
 api_method :GetMembers, :expects =>
[{:GroupID => :string}], :returns =>
[[:string]]
 api_method :Shutdown
end

In app/controllers/synchronize_controller.rb:

class SynchronizeController < ApplicationController
  web_service_dispatching_mode :direct
  wsdl_service_name 'Synchronize'
  web_service_scaffold :invoke

  def Initialize(nameValuePairs)
    session['initialized'] = true
    return 2
  end

  def GetGroups()
    if session['initialized']
      session['initialized'] = false
      groups = Group.find_all
      
      groupNames = Array.new
      for group in groups
        groupNames << "<SecureObject Name=\"#{group.name}\" AuthName=\"#{group.name}\" UniqueName=\"#{group.id}\"/>" 
      end 
      return groupNames
    else
      return nil
    end
  end
  
  def GetUsers()
    if session['initialized']
      session['initialized'] = false
      users = User.find_all
      
      userNames = Array.new
      for user in users
        userNames << "<SecureObject Name=\"#{user.login}\" AuthName=\"#{user.login}\" UniqueName=\"#{user.id}\"/>" 
      end
      
      return userNames
    else
      return nil
    end
  end

  def Shutdown()
    return nil
  end
end

Comments

Comments are listed in date ascending order (oldest first)

  • Nice post, Chris. This is the first time I’ve seen this done!

    Posted by: dmeyer on January 20, 2008 at 4:16 PM

  • Thank you, David.I just noticed that part of my sync code was chomped off in the blog post because WordPress was assuming that was actually an opening HTML/XML tag. I made the correction so the above code now accurately reflects what I was testing.

    Posted by: bucchere on January 21, 2008 at 1:16 PM

The Enterprise Relevance of Web 2.0

Register Now!

The concepts behind Web 2.0, social networks, and collaboration are now poised to transform your enterprise, providing solutions such as collaborative mashups, expertise discovery and social search to enhance your existing portal.

According to Gartner, Web 2.0 will have a major impact on a broad range of traditional enterprises. Gartner states that “positive business model change will result in unexpected ways, and enterprises must prepare for this transition.”

Register to attend this exciting seminar on Wednesday, November 14th, 6:00 pm and hear how BEA’s three new products will “two-dot-oh” your company’s Web along with other topics that include:

  • How Web 2.0 can bring true value to your business
  • How to differentiate between Web 2.0 and Enterprise 2.0
  • How to implement new Web 2.0 concepts like blogging, wikis, tagging and social networking into your business and allow IT governance and control
  • How to enhance your existing portal infrastructure

Enjoy free hors d’oeuvres and an open bar along with presentations that define Web 2.0 and show how BEA’s new social computing products Pages, Ensemble and Pathways can deliver true business value from Web 2.0 and bdg’s newest products that bridge the gap between Web 2.0 and the enterprise.

Attendance is limited, so please take a moment to register now. I look forward to meeting you at the event.

Date: Wednesday, November 14th
Time: 6:00 p.m. – 8:00 p.m.
Location: Marriott Tyson’s Corner

8028 Leesburg Pike
Vienna, VA, 22182
(703) 734-3200

Directions

Register Now!

Comments

Comments are listed in date ascending order (oldest first)

  • I’m sorry I missed this! If you have a notification list for events like these please include me, I’d love to hear about future events you guys sponsor. [email protected] Thanks!

    Posted by: geoffgarcia on January 17, 2008 at 1:33 PM

  • Hi Geoff! The event was down here in Tyson’s Corner, VA, so we focused on local attendees. I’ll make sure to include you next time, even though if my memory serves me correctly, you’re up in NY.

    Posted by: bucchere on January 17, 2008 at 6:47 PM

  • Oh, I almost forgot. If I can find the time, I’ll put together a video podcast of the event. I have the footage; I just haven’t had the time to do the editing. 🙁

    Posted by: bucchere on January 17, 2008 at 6:48 PM

Predictions: Will Oracle Acquire BEA?

There’s been a lot of speculation in response to some press releases from Oracle that an all-cash buyout of BEA may be immanent. More than two years ago, I made an entry on my company’s blog that said, effectively, that by acquiring Plumtree, BEA painted a target on itself to be acquired by Oracle. Here’s the snippet from my other blog dated August 28, 2005:

Will this deal make BEA even more of an acquisition target for Oracle?

Everyone I know — myself included — had a feeling that Plumtree would be acquired some day. But the major questions were 1) when and 2) by whom? Quite some time ago and long before Plumtree had its Java strategy fleshed out, there were rumors of a Microsoft takeover. Then Siebel. Then Peoplesoft. But BEA? I never would have guessed.

I personally thought Oracle would be the suitor, especially after they acquired Oblix, PeopleSoft and J.D. Edwards. After extending its tentacles into almost every enterprise software market (and proving tremendously incapable of producing any decent software applications other than a database), Oracle snapped up ERP, HR and SSO/Identity Management in the blink of an eye. It seemed reasonable to me that a good portal product that could integrate with all those applications would be a clear next target. Oracle’s portal certainly doesn’t cut the mustard. In fact, they often offer it up for free only to be beaten out by Plumtree, which is, ahem, a far cry from free.

Now the next pressing question: is Oracle even more likely to acquire Plumtree now that they’re a part of BEA? Now they’d get an excellent application server and a cross-platform, industry-leading portal. You know it crossed Larry Ellison’s mind when he heard the news. Food for thought.

I also said that BEA would keep the name Plumtree and lo-and-behold, they changed it to AquaLogic. So I wasn’t 100% right, but at least I can say that I called this one.

Comments

Comments are listed in date ascending order (oldest first)

  • Someone just walked into my office and said, “Hey, since BEA already has a dual portal strategy (ALI and WLP), what will happen if they get acquired by Oracle, which already has their own portal product?”

    Two years ago, I predicted a merging of WLP and ALI, with the result being much like ALI with the great developer tools you get from WLP and workshop tacked on to it. Obviously that’s not exactly how things played out.

    So my prediction this time is that all three portals will “seamlessly” co-exist under one roof, giving consumers plenty of ways to portalize all under the Oracle name. We’ll call it the Portal Trifecta — w00t!

    Posted by: bucchere on October 12, 2007 at 10:40 AM

  • Oracle is going to support SqlServer 2000 & 2005 for Aqualogic? And support .NET? Interesting if they would sell the Aqualogic piece of to to Microsoft. Give MOS a better external portal….?

    Posted by: vivekvp on October 12, 2007 at 11:37 AM

  • Great question, Vivek. I was surprised to see BEA pledge support for ALUI on .NET and SQL Server. I’ll be even more surprised to see that happen over at Oracle. Remember though, Oracle runs on Windows!

    Posted by: bucchere on October 12, 2007 at 12:08 PM

  • Chris, don’t you mean 4 portal products; ALUI, WLP, Oracle Portal, and WebCenter? The merger makes a lot of sense from my view point, but in all seriousness the one area which will need a lot of help is Portal. IBM has only one WebSphere Portal code base.

    Posted by: Dr. BEA Good on October 16, 2007 at 9:33 PM

  • It’s hard to image that a company maintains three or four full-featured portal products, even a giant like IBM, Oracle or MS.

    Posted by: caiwenliang on October 17, 2007 at 5:16 AM

  • Four portals? Yikes! I just don’t want confused consumers to go off and buy Sharepoint or WebSphere portal when I think ALUI and WLP are superior products.

    Posted by: bucchere on October 18, 2007 at 2:11 PM

Say hello world to comet

A couple of weekends ago I inflicted upon myself a quest to discover what all the buzz was about regarding Comet. What I discovered is that there is quite a bit of code out there to help you get started but the documentation around that code, and about Comet in general, is severely lacking. All I really wanted to find was a Comet-based Hello World, which as any developer knows, is the cornerstone of any programming language or methodology.

Since I couldn’t find one on Google, I ascertained that no Hello World exists for Comet and therefore I took it upon myself to write one.

For those of you who are new to Comet, the first thing you should do is read Alex Russell’s seminal blog post on the topic. At its core, Comet is really just a message bus for browser clients. In one browser, you can subscribe to a message and in another you can publish a message. When a message gets published, every browser that’s subscribed (almost) instantaneously receives it.

What? I thought clients (browsers) had to initiate communication per the HTTP spec. How does this work?

Under the covers, Comet implementations use a little-known feature of some web server implementations called continuations (or hanging gets). I won’t go into details here, but at a high level, a continuation initiates from the browser (as all HTTP requests must do) and then, when it’s received by the server, the thread handling it basically goes to sleep until it gets a message or times out. When it times out, it wakes up and sends a response back to the browser asking for a new request. When the thread on the server receives a message, it wakes up and sends the message payload sent back to the browser (which also implies that it’s time to send a new request). Via this mechanism, HTTP is more or less “inverted” so that the server is essentially messaging the client instead of vice-versa.

A few questions immediately pop into mind, so let’s just deal with them right now:

Why is this better than Ajax alone?

It boils down to latency and users’ tolerance for it. In the worst case, traditional web applications force entire page refreshes. Ajax applications are a little better, because they can refresh smaller parts of a page in response to users’ actions, but the upshot is that the users are still waiting for responses, right? A Comet-driven application has essentially removed the user from the picture. Instead of the user asking for fresh data, the server just sends it along as soon as it changes, given the application more of a “realtime” feel and removing virtually all perceived latency.

So are we back to client server again?

Sort of. Comet gives you the benefit of server-to-client messaging without the deployment issues associated with fat clients.

Can’t applets do this?

Of course they can. But who wants to download an applet when some lightweight Javascript will do the trick?

Why the name Comet?

Well, clearly it’s a pun on Ajax. But it’s not the only name for this sort of technology. There’s something out there called pushlets which claims to do the same thing as Comet, but which didn’t seem to catch on, I guess.

Back to the whole point of this post: my hello world. I pieced this example together using dojo.io.cometd and a recent version of Tomcat that into which I dropped the relevant parts of Jetty to provide support for continuations.

It’s finally time to say “hello world” to my hello world.

First off, download one of the more recent dojo builds that contains support for dojo.io.cometd. Drop dojo.js on your Java-based web/application server. (I used Tomcat, but you can use JBoss, Jetty, Weblogic, Websphere or any other web server with support for servlets.) Add this page in the root of your application:

<script src="js/dojo.js" type="text/javascript"></script>
<script type="text/javascript">//<![CDATA[
  dojo.require("dojo.io.cometd");
  cometd.init({}, "cometd");
  cometd.subscribe("/hello/world", false, "publishHandler");
  publishHandler = function(msg) { alert(msg.data.test); }
// ]]></script>
<input type="button" value="Click Me!" />

Without a cometd-enabled web server behind it, the above page does absolutely nothing.

So, to make this work, I needed to find a Java-based web/application server with support for continuations. I’m sure there are many ways to skin this cat, but I picked Jetty. You can get Jetty source and binaries if you’d like to follow along. Since all of our customers who embrace open source are lightyears more comfortable with Tomcat than they are with any other open source web/application server (ahem . . . Jetty), I decided to embed Jetty in Tomcat rather than run everything on Jetty alone. It’s all just Java, right?

Here I ran into a few snags. The maven build file for Jetty didn’t work for me, so I dropped everything in org.mortbay.cometd and org.mortbay.cometd.filter into my Eclipse project and just integrated it with the ant build.xml I was already using to build my web application. Here’s the relevant portion of my build.xml:

<javac srcdir="${srcdir}" destdir="${classdir}" debug="true" debuglevel="lines,vars,source">
<classpath>
<pathelement location="${jetty.home}/lib/jetty-util-6.0.1.jar"/>
<pathelement location="${jetty.home}/lib/servlet-api-2.5-6.0.1.jar"/>
</classpath>
</javac>

Once Jetty was essentially hacked into Tomcat, the rest was smooth sailing. I just wrote a JSP that dropped a “goodbye world” message onto the same old queue that I used in the last example, but I did so using server-side code. Here’s the JSP:

<%@page import="org.mortbay.cometd.*"%>
<%@page import="java.util.*"%>
<%
Bayeux b = (Bayeux)getServletContext().getAttribute(CometdServlet.ORG_MORTBAY_BAYEUX);
Channel c = b.getChannel("/hello/world");
Map message = new HashMap();
message.put("test", "goodbye world");
c.publish(message, b.newClient());
%>

This page does not produce any output of its own; rather, it just drops the “goodbye world” message on the queue. When you hit this page in a browser, any other browser listening to the /hello/world queue will get the message. The above JSP, along with the dojo page you created in the first step, should be enough to wire together two different flavors of Comet messaging: browser to server to browser and just plain old server to browser.

I’m curious 1) if this was helpful and 2) if you’d like to share what you’re doing with Comet with me (and please don’t say cleaning your kitchen).