Super Complex, Pan Dimensional Distributed Computing – It’s Not Just For Breakfast Anymore

As you may know, the OSGi Enterprise Expert Group is currently in the process of defining the interaction with “external” systems and formalizing the idea of what it means to have a “distributed” registry in OSGi. Having been through what seems like the same process now for the third time (EJB, SCA and now this effort), the overriding issue seems to be that distributed systems “are not like local systems”. One would have thought that this nugget of wisdom would be deeply lodged in the DNA of anyone who’s done anything with distributed systems for more than – say – a bit of example code found on The Server Side, but sometimes we do have to bring what should be obvious facts to the fore and illuminate them with Klieg lights so that the current, next and last generation who hasn’t figured this out yet will.
Amongst the things that are often brought up in these discussions are things like asynchronous messaging, marker interfaces, and – of course – the holiest of holies: METADATA. Now, naturally, I do agree with all of this. Asynchronous message patterns are darn important in not just distributed computing, but in “local” computing as well – so important to me are these patterns that I actually wrote a framework called Anubis for doing this and it’s something that literally provides a great deal of the foundation of the work that I do on a daily basis. Ditto for METADATA and all that jazz…
But what I believe should be obvious to people and for some reason seems to be not so obvious, is that all these things have absolutely nothing to do with OSGi. None. Natta. Zip. Zero. And what’s really odd to me is that when I say things like that, people believe that I’m saying “things like asynchronous messaging, marker interfaces, METADATA, etc, are not important”. Naturally, because of this misunderstanding, I spend 99% of my time trying to point out the fact that – yes – I actually do believe these things are of crucial importance and that – yes – these issues are necessary to resolve and come up with good solutions.
The point that never seems to get across is that I believe these issues are orthogonal to OSGi.

Continue reading

Distributing OSGi – Differences From R-OSGi

A couple of people have asked about the differences between the implementation I have for a distributed OSGi framework and the framework described by Jan Rellermeyer (see my previous entry on the subject). In a nutshell, the primary difference is that the way services are advertised and discovered is completely hidden in my implementation. The implications, though, that this small difference has on usability and integration with existing OSGi frameworks is rather profound.

Continue reading

Distributing OSGi

At this year’s EclipseCon, I went to a talk by Jan Rellermeyer regarding his R-OSGi system. His stuff is open source, which you can find here. The slides for his talk are available here. It’s extremely interesting and I would suggest anyone who’s interested in the distributed OSGi stream to both download and play with the r-OSGi system and to read the paper and presentation. In particular, I think that his discussion of the alternatives vis a vis Jini, UPnP and SLP is quite illuminating.
After I went to Jan’s talk I went back to my cave and started to play around with my own version of the system. I had been playing with SLP (Service Location Protocol) quite a bit in other distributed systems and it was quite obvious after Jan’s talk that SLP and OSGi are a good fit. However, I was rather troubled by the degree that SLP was exposed to the end programmer and started on my own implementation.
What I ended up with is a system which has, I believe, a minimal surface exposed to the end programmers of an OSGi system. From the service interaction perspective of the programmer, the only mechanism which is added to standard OSGi is the way you indicate which services you would like to import into your local OSGi process registry. There is no exposure of SLP or any other system for discovering services. Rather, the system simply makes use of the already existing mechanisms in OSGi for discovering services – i.e. the service registry, filters, ServiceReferences and ServiceTrackers n’ such.

Continue reading

EclipseCon: Day Three

Keynote was again fantastic. Hugh Thompson talking about security and such. Fuzz testing. Negative requirements (e.g. users should only be able to login with u/p: positive requirement. No one can access the database except through logging in: negative requirement). Chatted with the Paramesus guys as well as the gentleman from Knopflerfish. Went to the talk on “OSGi: the good, the bad and the ugly” by BEA guys. Man, they’ve done a lot of work and I’m pretty jealous. They don’t seem to think that OSGi needs “distribution services” either. Good to see. Also, they hate – hate, I say – the whole evil of 277. Good. May 277 rot on the vine.
It was interesting to hear them talk about their experiences developing with OSGi. It’s quite the common refrain: “I thought I wrote modular code until I started working in OSGi”. And it’s going to become even more common to hear this. Sadly, it’s going to be one of the barriers to entry into OSGi: i.e. it’s never a good business strategy (as I found out in the past) to force developers to understand their code. I’m looking into tools like Lattix, which seems quite promising in being able to help you understand and figure out your modularity and layering in your architecture. Something very, very, important in OSGi and if it takes off like we all think, something that a lot more people will be paying attention too.
Which is a really good thing, when you think about it. I really despair at times for all the talk by people who constantly complain about having to do actual work to make use of something. I mean, we need to keep raising the bar – it’s the only way we’re going to make things better. Dumb people produce dumb code and dumb code is dangerous, insecure and something that should be eliminated. It won’t be eliminated by better IDEs, team development environments, SCRUM or the religion of Agile. It simply comes down to better people producing better code and leaning from their mistakes in a virtuous cycle. </rant>

EclipseCon: Day Two

R0ml was here – excellent key note. Economics and responsibility. Warrants vs. ? Can’t remember. “Of course Eclipse is the worst programming IDE on the market – IT’S FREE”. And anything which is worse than eclipse isn’t going to be used, much less have someone charge money for. This is what “free” does – it sets the lower level of expectations. Literate programming. Reminds me of the old days when I worshiped Knuth and his idea of Literate Programming. Only thing was, Knuth’s idea of an IDE was raw TEX – insanity. Anyways, very cool stuff from ROml. Hard to keep track of and I was too busy listening to take notes. If you ever get a chance to hear him talk, I would suggest dropping everything and doing so. Well worth the dollar.
Ajax panel – what is different about Ajax that makes it impossible to build upon the past? I actually got a small quote in ARN on this (hey, we take what publicity we can)

An audience member emphasized that rather than devising new technologies, it may be a good idea to leverage what has already been done. “They should be standing on the shoulders of giants,” said Hal Hildebrand, an architect in Oracle’s Java platform group, after the session. AJAX is a new interface technology and lots of work already has been done in this area, he said.

Just let me expand a bit on this. X has been around for, what? Over twenty years? What’s so different about AJAX? It’s just a remote windowing system. It’s like a frickin’ VT100, for Bob’s sake. So you do it with XML or JASON. Oooooooohhhhh. I’m so impressed.
A button is still a button and a list is still a list. My god. All the Javascript is still written in the same namespace! No wonder you can’t get your act together out there and you have name collisions all over the place. Geebus, we learned about namespaces and scopes back in kindergarten. But I guess Web 2.0 broke all the rules and those bad boys are making it up as it goes along. Heck I heard Carl Hewitt (who was there, of course) has a class system for Javascript and a JIT. Man, that sounds an awful lot like Java to me. Congratulations, y’all. You just reinvented 1997.
Okay. Deep breaths.
OSGi panel – was it good for you? So I was on this panel. We played to a packed moderately occupied theater and it went over pretty well. It’s pretty clear that OSGi’s star is on the rise and the major players in the market – including House Harkonnen – are clearly buying into it. All in all, it was pretty good for me and hell, OSGi even kissed me in the morning and made me breakfast.
R-OSGi. This talk by Jan Rellermey of ETH was worth the entire cost of the conference for me (especially as House Harkonnen is paying for it). Basically, he turned OSGi into a distributed system in a quite elegant way. His talk started out with some background as to what choices are available for modeling OSGi as a distributed system. The first and obvious choice is JINI. Now, JINI is one of those technologies which seems to generate a lot of religious fervor. Sometimes I feel like the people who are JINI fans are kind of like old style communists who keep on telling people that communism was never really tried – JINI is the next distributed system and always will be.
Anyhow, companies like Parmeus have taken the JINI route in creating a distributed OSGi system and have run with it. I have to say, though, I agree with Jan’s take on JINI and why it’s not a great fit for OSGi. But I’ll leave that to Jan. Let’s just say that I think JINI has way too much baggage and is – quite frankly – a poor fit.
Next Jan talked about Universal Plug n’ Play, which is another candidate model he considered for distributing OSGi. Again, the model doesn’t quite fit with the way OSGi services are used and the way they are filtered and discovered. Yes, OSGi has a UPnP service definition and Jan pointed us to Domoware’s UPnP implementation which he highly recommends.
In the end, Jan decided to base his work around SLP – the Service Location Protocol. And I must say that I smacked my forehead in a D’oh! moment. I had played around with SLP quite a bit when I was fooling around with Smartfrog – it’s quite a sweet protocol. As Jan Points out, the mechanism SLP uses for attributes and filtering is the same LDAP query format that OSGi uses for the service filters. So, it’s a 100% match with the way OSGi does things.
Jan’s framework has the usual cool stuff such as auto generated proxies using CGLIB and does some pretty neat stuff like shipping virtual bundles across the network for the clients. Really, you have to get the code and start playing with it to see how cool his stuff is. Having been playing around with it for a while, I do have some critiques and changes (of course) that I’m making. But that’s the subject for another post ;)
Enterprise OSGi – Siemens, SCA, distributed registries and such. It was interesting to see their presentation and the problems they’re trying to solve. After the R-OSGi talk, I was pretty well primed and have a zillion potential solutions for them. Going to be much more interesting in the OSGi Enterprise Expert Group.
Jazz – Envy’s electric boogaloo. I must say that the thing that we love to do is create tools. tools for creating tools which create tools which allow us to create tools for managing the process to collaboratively develop tools for creating tools. Ye gods. They have their own source code control. Yea, that’ll go over real well at companies :)

EclipseCon: Day One

Scammed a “committer” shirt from the conference. Really, it was all innocent. When I registered, the nice woman who was taking my registration asked if I was a “commuter”. Well, at least, that’s what I thought she said. “Yes,” I helpfully replied – hell, I had just spent an hour driving from my secluded fortress of silence to Santa Clara. “Go to that table after you’re done here and we have a nice shirt for you.”
So, I head over to the table and she takes my badge and consults a list. “Gee,” I think to myself, “amazing technology that they have which knows – in advance – whether I was commuting to EclipseCon would take information sharing between the organizers of the convention and surrounding hotels on a scale that boggles the imagination.” But perhaps the DHS has tendrils that are far more pervasive than even my fevered, paranoid mind could imagine.
“I’m sorry, but you’re not on the list.” What? I’m not on the list? “Of course, I’m on the list”. Confused, the nice lady asked, “So you have been okay’d by the person on the other side?” “Of course.” I replied with confidence. So she wrote down my name on the list – so lord knows someone is going to get a laugh from that one – and she handed me my quite nice button shirt with the “Eclipse Committer” embroidered badge.
Psyche! Here I thought I was going to get a “commuter” shirt and I end up with a open source deputy’s badge. I rule the town of RockRidge.
Key Note by Scott Adams: All I can say is that it is pretty surreal to sit in a room with thousands of geeks reading Dilbert cartoons. Granted, it’s funny and he’s got a lot of interesting stories behind the cartoons. I don’t, however, for a moment believe that he was passed up for promotions because of “diversity” requirements. But then, any one who loves to dig him self deeper with this I.D. crap can’t be expected to do anything less.
Met one of my very old friends Hendrik, and Dave L. was there as well. Kind of like the old OOPSLAs, as I was promised by Herr Milinkovitch.
Went to a talk by the guy who heads up Mylar Tasks: very cool stuff. I’m now starting to use it in my work and it really is an amazing piece of work. Again, another reason why IDEA is losing out to Eclipse. Really, there’s no reason why the Mylar guys shouldn’t be doing this for IntelliJ – so something has got to be going on. The stuff that the Mylar guys are planning seems quite aggressive and a bit overreaching – everyone wants to rule the world, I suppose – but no doubt will still be worth using.
Google summer of code – ho hum. “what did you learn?” bizarre. Sometimes I think that Google really is nothing but a glorified IT department filled with Dilberts.
Short talks
RMI in OSGi – seems like a lot of the problem could be solved by the Spring context class loader rather than the wild, wild west of the service registry only/bundle nightmare. Context is everything.
OSGi and the JBoss Microcontainer – love the Russians. Microcontainer is a fine grained state machine for managing dependencies. Scoped metadata. Aspectized deployment ??? Integration points: Metadata. Seems like their metadata model is much slicker than the Spring namespace extension model, but that’s something easily fixed. Controller – dependency state machine – seems like a stunning degree of overkill and over modeling. Lifecycle transitions modeled with state machine. Overall, seems like a seriously severe case of the second system effect. Lot’s of wrapping. Man, this is going to suck. Seems to me that I remember a time when JBoss was telling us that the old JMX microcontainer was the greatest thing since sliced bread. I guess this is even better than that ;)