I’m a fan of learning lots of languages, and there are plenty of interesting ones out there to learn. I was recently watching some of the Clojure videos, and Rich Hickey had a nice quote about concurrent programming:

Mutable objects are the new spaghetti code.

Having written more than my fair share of multi-threaded applications, I completely agree. Shared nothing concurrency is the way to go.

My tool of choice is currently Erlang, as you can probably tell if you’ve been reading this blog recently. We can also see Python approaching this model with their multiprocessing module. The Twisted mailing list often has threads on message passing concurrency in Python as well. Even Guido thinks this is the way to go:

My expectation is that something not entirely unlike the Actor paradigm, but implemented on top of the multiprocessing package, will eventually become the recommended best practice for reaping the benefits of multiple cores.

Of course, the wonderful thing about Erlang is that it takes the multiprocessing module style concept to the next level with extremely lightweight, cheap processes. You can create a tens of thousands of Erlang processing in a few milliseconds. It also already contains distribution primitives so that processes can be created on any machine; this is something that Clojure and Python still lack. The price you pay for distribution is in slightly more complex failure handling, but Erlang’s OTP behaviors make these pretty easy to deal with.

I imagine that we will see many languages start picking up these pieces and playing around with them as more and more people realize that shared state and mutable objects are a difficult path forward.

Have you notice the drums roll and trumpeting around the latest Sametime interoperability announcement? I always find it entertaining to witness how incumbents can turn a failure to respond to their customers wishes into a world saving progress without even a blush!

At the end of the summer of 2004, I had the opportunity to see IBM in action at a workshop held by the Financial Service Instant Messaging Association on the subject of IM interoperability. FIMA had been worried very early on by the disparity of instant messaging protocols, both in the public and the enterprise IM space. So they set to give their least common denominator functional definition of interoperability, which when you think of it is not asking for the moon…
At the workshop, vendor presented their views, and it all went smoothly until the chair of the Jabber Software Foundation asked IBM why they wanted to support the still unfinished SIMPLE protocol as a federation protocol rather than XMPP which was already stable and widely used alongside Sametime in the financial industry.
Anyone either business savvy or a little familiar with Sametime's technical architecture would have though along the same lines. From an architecture standpoint, the multiplexers, community hubs and community server applications of sametime, with their permanent TCP links, can be functionally mapped one to one with XMPP client connectors, session managers and service components. From a business perspective, federating the existing large financial industry's Sametime islands with the expanding XMPP footprint would have help IBM keep an edge against the nascent Microsoft LCS expansion.
Instead, we were granted a flame from the IBM representative in favor of SIMPLE which was more typical of a Gorilla's behavior than a business representative:

If challenged by a younger or even by an outsider male, a silverback [Silverbacks are the strong, dominant troop leaders] will scream, beat his chest, break branches, bare his teeth, then charge forward.

In short, SIMPLE was the way to go, and everyone not endorsing it was stupid and doomed to fail. Well, two years after, IBM announced interoperability with XMPP, which instantaneously guaranties secure and controlled communication with any XMPP server, including those at Google. They also announced interoperability with AOL, but funnily enough they do not specify what protocol they use. You maybe interested to know that AOL offers both an LCS flavor of SIMPLE and XMPP to enterprises for federation with AIM. As there is no mention of interoperability with LCS in the Sametime announcement, it would be interesting to have a clearer insight on what they IBM is using for interconnecting with AOL. As to Yahoo!, well it is always a little difficult to have anything but a fuzzy statement from these guys. Anyway, the announcement said federation between Sametime and Yahoo messenger is on the "road map".

Ken Camp came up with a nice and illustrative contraction to apply to people unable to spot the appropriate answer to end-users' aspirations, cluetards… In this case, IBM is only different from the average by the size, just a large cluetard. If I were a financial Sametime customer, I would claim compensation for having been treated this way and left longing for so many years.
More importantly, this episode illustrates once again how large incumbent IT vendors are blind to real users' needs, when they are not in line with their core business. IBM business is in selling servers, not in providing communication between peoples. I find it amazing that a pure technical choice made by some ex-Ubique geek ended up depriving an important business community of an easy and beneficial way to conduct their business with small firms or other institutions.

Beyond this, Giacomo is in my opinion asking the right question: should we forget SIMPLE? In a perfect world, I would certainly answer positively. In the real world, it will take some time in view of the interests at stake. Having been involved in designing applications federating both protocols, I certainly have a unique perspective on the strength and weaknesses of each of them. From a technical standpoint, my only observation is that SIMPLE simply lacks the simplicity needed for a wide adoption.

• Except the Microsoft LCS there is no SIMPLE based instant messaging and presence (IMP) application in service outside the telecom industry. Anybody having looked closely at the actual protocol used by LCS will agree that it's actual compliance with SIMPLE stops short after the first SERVICE packet. Microsoft has leaned the hard way that SIMPLE was not ready for IMP, and has been obliged to supplement the lack of the protocol with LCS own extensions. Such things as multi client instances, message sessions, contact lists management, preferences, multi user conferencing are notably absent from the specification and have partially been implemented by Microsoft using non-standard extensions.
• From a user agent perspective, the closest to a "standard" SIMPLE client implementation is the Counterpath' eyebeam softphone and it XLite derivative. Their development team deserves the greatest respect for having incessantly been trying to accommodate the ever changing scope of XCAP, and the various contact lists management options that must be part of a "workable" SIMPLE solution. But while doing this, they had no time to automate the status change to "on the phone" when a SIP call was taking place…
• From a standard stand point, those who have red the 2500 plus pages from the 3GPP describing the use of SIMPLE for presence are well aware that, for example, list services, XCAP and a stable definition of rich PIDF are on the critical path for delivering the service as expected by the IMS architecture. XCAP is probably the most blatant example of wheel re-invention in this industry, where the oversized ego of the author is holding back an entire industry. More importantly, if one look at the SIMPLE RFCs and drafts authors, one can quickly infer that the standardization process is done by a vendor led consortium. As I have said before, almost every vendor led consortium has failed to impose any long lived interoperability standard on the Internet. I am afraid SIMPLE is another one of them, and it is time to realize it…
• Finally, there is the complexity of SIMPLE. It makes it difficult to grasp by the average application developer outside those versed in "voice over…". When adding together the rather rigid framework provided by SIMPLE and the vast differences in interpretations by various implementers, it makes this protocol look "elitist" when compared to XMPP. As I mentioned above, having designed and implemented converged systems supporting those two protocols simultaneously, I can vouch that any application programmer can grasp how XMPP works; the same is not true from SIMPLE.

As Ken put it, simplicity and elegance alway make winners and losers. Simplicity is unfortunately not the first quality of SIMPLE. As a result, it has not seen mainstream adoption by web application developers which are instead turning to XMPP. That said, SIMPLE will not go away easily in view of the heavy investments made by many cluetards. Unfortunately this state of affairs will help maintain the silo divide between the telecom and the internet worlds.

The term "elegance" finds its origin in the Latin "eligere" which also gave the derived term "election". In fashion as in real life, elegance implies making a choice. In the Sametime case, it was between remaining Incredibly Blind and Mediocre, or working at solving real end-users pain points. You can guess what choice was made…

Technorati Tags: XMPP, Jabber, SIP, Interoperability, Antecipate

This is the most common definition given of a gateway in the telecommunication context:

A gateway is node that translates between two otherwise incompatible networks or network segments. Gateways perform code and protocol conversion to facilitate traffic between different architectures.

On the ground of this definition, the application that IBM has built on its Websphere J2EE server for Sametime is effectively a gateway. For that exact same reason, I believe that a very substantial leap of faith is required to envisage that it could lead to a multi-protocol presence server.
Fom the information available, the gateway is an application build on the JSR-000116 SIP Servlet API. It is in its way following a similar architecture of what IBM tout as the Websphere Presence Server. The SIP Servlet API is an attempt at leveraging developers' knowledge of HTTP web containers and extending it to SIP. On paper this is a sensible idea. In practice it has a few drawbacks.

• First, it is entirely based on the SIP protocol, which means that applications built on top of this technology will be SIP applications. As such, the SIP servlets are designed to process SIP requests and return SIP responses.
  That somewhat restrict the scope, as each network protocol has very specific characteristics which are rarely though of as "universal". Moreover, not every presence protocol uses a request/response paradigm in the way SIP does. XMPP for example uses simple notification packets to signal modification in presence sates, without requiring these packets to be answered.  In that respect, it becomes more difficult to mold XMPP into a servlet paradigm in the way it was possible with SIP. Consequently, the advantages expected from a common programming paradigm become somewhat blurred.
• Second, it relies on a J2EE server to run. One can argue that it is a proven robust architecture. I would simply say that J2EE servers were not invented to provide solutions to real-time communications problems, but rather to bring solution to transactional application problems in the enterprise. The table herewith summarizes the differences between the two contexts.

All in all, if the IBM gateway had been implemented using a more appropriate application server such as a JSR 22 JAIN SLEE API implementation, I would have been tempted to take the leap of faith. Unfortunately, this gateway architecture prohibits any use beyond bridging between SIP and other protocols. To be fair, the same inability applies to the equally shortsighted approach taken by Jabber Inc. A solution that only sees the world through the eyes of a particular protocol will always leave us far from what is necessary to build a multi-protocol presence server. For the simple reason presence encompasses more than just protocols…

Technorati Tags: XMPP, SIP, Presence, Antecipate

Our tests reports page has been changed and is now available at the new address. You can still access test reports for old version at the old tests reports address.

The reason for this change is that we run much more tests now than we used to do before and the page needed some redesign to make it easier to access all reports. We not only increase number of individual test cases but also we run increase number of test scenarios.

We have plans for the future to add more operating systems, more databases and more complex scenarios to the test suite to make sure we release new versions with as little bugs as possible.

If you have any suggestions to the test report page or any ideas for test scenarios use cases I would be happy to hear them.

read more

Google this week announced that it was going to launch free video chat with its Gmail messaging service.

Essentially it is a web browser plug-in that will allow Gmail users to conduct video chat sessions with other users. Rajen Singh, senior product manager for Google's enterprise group, stated that this is a "significant” move and is something that Google can see bringing people closer together particularly in a business context.

This is further evidence that video IM will be a useful tool for the business. For instance it can be used as a way to cut travel costs for face-to-face meetings and in the current economic climate, this will be a very attractive proposition for many businesses.

Brian Kraemer at CRN praised the application for its ease of use, "Users can pop the window out of the Gmail interface, just like a Gchat instant message window. Users are then given the option to go full screen with the chat, for an up close and personal experience with your chat partner.”

This further proves that video chat is the next stage of evolution for instant messaging and XMPP (on which Google’s messaging services are based). XMPP server developers like ProcessOne are already developing video and voice chat application for businesses based on XMPP and we see considerable growth in this market in the future.

You can get more details on the articles from CRN - Gmail add is video and voice chat, and Information Week - Google's Gmail gets free video chat.

In a relatively slow week for XMPP-related news:

Deadline

Web Worker Daily, amongst others, reported on the launch of ‘Deadline‘ a web-based reminder service with a plain english interface which uses email and XMPP IM to send reminders of appointments and tasks.

FriendFeed

FriendFeed co-founder Bret Taylor joined the Gilmore Gang podcast to talk about their release of XMPP realtime services. The podcast is worth a listen, with the XMPP specific discussions starting at 15:30 into the show.

IBM Blue Spruce

IBM gave ReadWriteWeb a first look at Blue Spruce, a complete browser-based application development platform that the company has been working on for the last 5 months and plans to launch around 2010. Blue Spruce combines a Co-Web Server and a Client Toolkit, which together provide a development platform that enables developers to push their applications to the browser. Server and Client communicate using XMPP. Slightly more detail can be found in an article on CMS Wire.

It appears as though something broke with Google Talk’s servers recently. Reports are coming in of sudden TLS negotiation failures. Fabio claims that older versions of OpenSSL are at fault, but Ralph suspects that something changed at Google.

Specifically this means Twisted Python and Wokkel code won’t work with Google Talk (hosted or otherwise). I’m sure other clients are affected as well, but I haven’t seen many other reports of this. There are work arounds, but they are not very good ones, and certainly will not be accepted solutions upstream. The main one is to switch to SSL v2 or v3, although Fabio reports that using the latest releases of OpenSSL instead of the distribution supplied ones also appears to work.

Both Ralph and I have been attempting to reach our contacts on the Google Talk team. Please leave a comment if you have any more information.

We have just launched the first version of tweet.IM, a Twitter - instant messaging gateway service, based on the XMPP protocol.

ProcessOne has just launched a gateway service between XMPP and Twitter. The service is available from tweet.IM. It allows you to use Twitter directly from your favorite instant messaging client. It supports getting friends timeline and direct messages, but also posting public Tweet or direct messages.

You can use it from any server, as long as your server accepts server-to-server connections to tweet.im / twitter.tweet.im

Direct messages are received from a separate contact in a separate chat window / tab. Replying to this contact sends a direct message.

A forum is open and waiting for your comments and suggestions: tweet.IM forum.

Please, note that GTalk is currently behaving strangely, not storing / accepting the contact subscription acknowledgement. There is a workaround for that problem: after having filled the form on tweet.IM and accepted the subscription request on Gtalk, simply ask to add the contact twitter.tweet.im in your roster. It should do the trick. It used to work normally this afternoon, so I guess this is a temporary workaround.

Pedro was trying to convince me this week that ejabberd needs an HTTP backend. This would allow anyone to substitute a backend of their choice very easily, without having to code an ejabberd module in Erlang. It would be easy enough to make an HTTP server wrapper on top of any kind of database you wanted.

I wasn’t entirely convinced at first, but as we talked about it more, we decided a CouchDB backend would be really awesome. This could actually serve two purposes. The first is two enable CouchDB as a replacement backend for Erlang’s Mnesia or traditional RDBMSes. The second is that since the CouchDB API is so simple, it could be implemented by anything that speaks HTTP, assuming you didn’t need any view logic.

I’m not sure that it’s easy to do without the view logic without going through all the queries and trying it, but even if that’s the case, a CouchDB backend would be a big win. For starters, dealing with Mnesia is sort of a black art, and CouchDB is as simple as can be.

Someone has already created mod_couch which enables ejabberd to use CouchDB via the ecouch library. Now we just need the rest of it.

To celebrate that ejabberd turns 6 years old, I've prepared a video that shows the history or ejabberd trunk SVN during those years: authors, acknowledgments, type of files, dates and releases. The video was built with code_swarm.

Download ejabberd-6-years-code.avi (12.5 MB) from:

• RapidShare, click in "Free user" button, wait 50 seconds, then click in "Download" big button.
• Jabber.ru direct download

Notes:

read more

I had the pleasure of hanging out with Pedro Melo for a few days in Lisbon, and we had a lot of fun throwing around crazy XMPP related ideas. Of course, such crazy ideas are not new for Pedro; just take a look at Rasputine.

Rasputine is a gateway from XMPP to telnet which Pedro wrote for playing on a Lambda Moo called Moosaico, but it can be used for far more. For example, since routers usually support telnet administration, you could add them as friends and manipulate them via XMPP. Just imagine the crazy things you could do if you could add commands to Rasputine to have those friends join a MUC. Synchronized routing changes!

Pedro has also written about Rasputine a couple of times. Go check it out, and keep and eye out in case any of our joint plans come to life.

No matter how many automatic tests you run only real users can check the software in real use. Well I am not saying automatic tests are for nothing... Just, they can not be a real user replacement.

This is just a bug fix, minor Tigase server release. Everybody who is affected by any of mentioned below problems is encouraged to update.

read more

We are not moving project tracker or Tigase server development to launchpad.net however.

The purpose of the project created over there is to host prebuilt Ubuntu packages in a PPA (personal package archive).

A few weeks ago I announced that Gentoo ebuilds are available. I have also received information about RPM packages being maintained by someone. That would be the third packaging system for the Tigase server I know of. This makes live easier for all those who want to install Tigase server on different Linux distributions.

While I can not maintain packages for all systems myself I am always happy to help and assist those who decide to take this responsibility. If anybody is interested in helping with those please contact me. The Gentoo Linux ebuilds will be hosted in the main Tigase server SVN repository. Ubuntu packages on launchpad.net, I am not sure where RPM packages will be available but there is always a lot of space in our SVN.

read more

A new release of Openfire with critical bug fixes has been released. It is highly recommended for existing installation to upgrade to this new release. The complete set of fixes can be found here. The SIP plugin should be updated too.

 

Download Openfire from: http://www.igniterealtime.org/downloads/index.jsp

Download Connection Manager from: http://www.igniterealtime.org/projects/openfire/connection_manager.jsp

 

Enjoy!

 

Openfire Team

One of the most exciting projects around is CouchDB. We’ve been playing with it for months, and we even used it in Speeqe for group chat logging. It is built like the Web; it scales like the Web; and it’s as cool as the Web.

CouchDB is schema-less, eventually consistent, and speaks HTTP and JSON natively. One of the coolest pieces is the view layer, which uses MapReduce(http://en.wikipedia.org/wiki/Mapreduce) with map and reduce functions provided by JavaScript.

It replicates well in any topology you desire. Finally, we have master-master replication that works well. It can do this because it doesn’t assume away the problem of data conflicts. Instead, CouchDB provides tools to deal with conflicts.

Still not convinced? CouchDB is a peer-to-peer to application server out of the box, thanks to its well done Web interface. It also has the coolest logo of any project I’ve seen in quite some time.

So relax, and check out CouchDB.

I tend to see the same set of things done over and over again in clients as they go from "I only care about XMPP chatting" to "users keep asking us for better transport support". Many clients look for bare JIDs in your roster and treat them in a special manner, which is fine except in some cases I have bare JIDs in my roster and the service it's attached to is gone, so the client gets a little confused. Clients like Psi will dutifully look for jabber:iq:register and offer that as an option on services. Many of them you have to go disco browsing to find the transports available in the first place. Nothing wrong with any of that. Some clients may choose to go ahead and look for things marked jabber:iq:gateway and add them to some special list. That's not really what this is about though.

A number of clients will add in basically "hacks" for individual chat services. Psi has "service icon sets", and if it's a new service that isn't in Psi's list, it comes up with default icons. Spark has a bizarre mechanism in which it explicitly adds support for each individual transport via what I call little java "nuggets". But a common theme I see here is individually handling each service. Some new service comes along, now all of the clients have to be updated if they want to have cool icons. Personally I think that's crap. ;) I'm a big fan of dynamic lists of things. In other words a client shouldn't have to -know- about any of the specific transports. It should be able to simply pull a list of services on the server in question, if it wants gateway ones look for jabber:iq:gateway, and --- and here's the missing part --- be able to retrieve whatever it needs to provide a nice experience to the end user.

So what all tends to be custom added? Generally it boils down to icons. Icons icons icons. Icons for presence, Icons for representing the transport, etc etc. I don't think I've ever seen anything beyond icons that couldn't be determined from disco queries. So why couldn't service icons be provided by the service itself? I could see something along the lines of using pubsub where the transport publishes it's icon sets, one for service icons (maybe a set of different standardized sizes) and one for presence representation icons. That way a client could just come in and go "gimme the icons I should use" and move on. I admit my knowledge of pubsub is -still- limited, but my understanding is that there are version numbers associated with the published data, and the client could cache what it got and simply check for "is there a newer set?".

The primary caveat here that I can think of is, that pulls the 'control' of the icons away from the client and into the transport, and there's always a chance that the client doesn't "like" the icons the transport is sharing. Of course, for the most part the clients are aiming to use icons similar to those of the service the transport is providing, so they kind of already have to deal with the possibility that for example AIM's icon might not look great in their client. So the question is ... would client authors use this or just end up scoffing at icons coming from the transports themselves?

I suppose if the clients really felt like they needed to have different icons for some services, they could always override what they're getting from the transport and do their own anyway. It just always feels like a hackjob to me to see things like "if this JID matches with aim.* then use the AIM set". Especially those instances make me cringe because JIDs can be anything for those transports, where the transports -will- be providing a gateway identity that tells you flat out what service it is, assuming the transport authors bothered to follow spec. ;)

The last release of the Tigase server included 2 our components: PubSub and MUC.

Unfortunately both components were not working with Apache Derby database. Due to a small bug. On top of that there was name change for the main class of the MUC component in the mean time I was not aware of. This causes configuration to be incorrectly generated by the installer.

We are going to release a big fix version today or at latest tomorrow for the Tigase server. In mean time for all those keen to get both components working ASAP I am giving both files to download below this article. If you want to install them, remove tigase-pubsub.jar and tigase-muc.jar from your libs/ directory and put there files downloaded from here.

For the MUC component you have to also edit init.properties file and replace MUCService with MUCComponent. Don't forget to remove the existing XML config file after before restarting the server if you edited the init.properties.

We are sorry for all the inconvenience that could caused.

read more

Yesterday I arrived in Lisbon, Portugal, and today I gave a presentation on XMPP at SAPO Codebits. There were quite a few people in attendance, and we explored XMPP APIs for a fictional server monitoring application.

I also saw a presentation on Dojo by Peter Svensson and Jan Lehnardt’s presentation on CouchDB. CouchDB is one of my favorite up and coming projects, and it has definitely made databases cool again. Jan implied that some big companies are deploying CouchDB now for applications that will be announced shortly.

I didn’t see the presentation on server side JavaScript, but everyone was talking about it afterward. I think a server side toolkit would be great for getting people started in Web development. They would only have to learn one programming language, not two or more. They have some real hurdles to overcome first, most of them related to the lack of any kind of library support for things like disk access and sockets. Between those ideas and Peter’s talk about moving processing to the client, the client-side templating and rendering frameworks that are popping up are starting to sound pretty appealing. We might as well be making use of all the computers accessing our services.

I had a great time today, and the SAPO folks are putting on a great conference. It’s certainly clear that Portugal has a lot of talented developers.

Version 0.1 of XEP-0255: Location Query has been released.

This specification defines an XMPP protocol extension for querying a compliant server for information about the geographical or physical location of an entity."

The changelog is:

Initial published version. (psa)

Version 0.1 of XEP-0254: PubSub Queueing has been released.

This specification defines an extension to XMPP publish-subscribe for queueing information at a node.

The changelog is:

Initial published version. (psa)