Okay, I have to confess that I didn’t think I was taking Bob to the woodshed in my response to his response to my interminable screed. But he’s a good sport (haven’t spotted any wolf packs nipping at my heels, yet) and follows up the initial serve with some good points worth thinking about.
One of things that Bob throws out in his post is a rather interesting product called Java Rebel that bears watching. It seems like very interesting technology and I’m quite curious as to what it looks like under the covers.
But going to the meta point that Dr. Pasker seems to be pointing at, I just happened to read a rather interesting article who’s topic seems quite germane to the discussion at hand: Multiprocess versus Multithreaded…or why Java infects Unix with the Windows mindset. I highly recommend anyone interested in this kind of stuff read that post as I think it does speak to the heart of the issue.
One of the eternispecs has been the Java Isolate spec – JSR 121. It’s always garnered a lot of interest and at one time, some sound and fury was directed towards it. But it’s something that has yet to find its way into the outside world.
Between the two topics above – Java as a Windows mindset and Isolates – I think that one can get a pretty clear picture of what’s going on and where the essential issues lay. The problem is simply that we’ve effectively eliminated the process out of the Java model without providing the required support within the model for the work that we’d like to do which is desperately seeking a process model.
Just found the Java Native Access library for dynamic access to native libraries without JNI. Sweet. Had been using the JNI wrapper, which provided similar functionality, (but cost money and had licensing encrustations) previously. Glad to see this available as it makes it far easier for me to start integrating with C based physics engines (ODE) and 3D gaming engines (OGRE) in the 3rd Space project.
Bryan Atsatt, who is House Harkonnen’s representative on the JSR 277 expert group, has been doing a lot of work trying to bring the two warring parties together and not simply salvage the relationship but to turn the momentum around and spark a new found friendship between the two.
His first post on his new blog is dedicated to this premise, JSR 277 Could Be Great for OSGi
The initial spec actually has two separate parts: an api/framework, and an implementation with a new distribution format. Unfortunately, these are presented in a way that seriously blurs the distinction. Worse, the new distribution format (“.jam” files) often takes center stage.
The emPHAsis is on the wrong syLLAble.
The api/framework layer must be the primary focus of the JSR 277 specification, providing a coherent compile/runtime model that enables multiple implementations. Specific implementations, while required to surface framework/api issues, should be documented in appendices or even separate specs.
If the EDR spec had been written from this perspective, we probably would have avoided most of the current animosity. We can and should fix this in the next draft release. Implementations can then be seen on equal footing. More importantly, they can be left to compete on their own merits.
Not everyone wants or needs OSGi, and the new .jam implementation may be right for them.
But we know that there will be LOTS of bundles around when SE 7 ships, vastly outnumbering .jam files, so we need an OSGi implementation. ASAP. Built by OSGi insiders. Without it, we cannot have confidence that the api/framework abstractions are right or complete. With it, we not only gain spec validation but have a ready-made solution for using bundles on SE 7.
Give it a read.
So Bob Pasker has a typically thoughtful response to my post regarding the changes I’ve made to the infrastructure of the Prime Mover simulation framework. The regaling of my adventures reminds him of his conclusion that Java is a lousy language for building infrastructure.
It’s something that I never really thought about, personally. It’s not that I haven’t cursed Java for some reason or another on a continual basis since I started programming in it. But that’s always the way it is when you’re pushing the limits of what a language is capable of and what a system is designed to do. You love to hate it and the very fact that it fights against you as you bend, fold and mutilate it, in the process of violating the warranty is just our way of saying that we really love it – really, we do. Sure, it’s not a healthy relationship, but then again, specializing in these kinds of things has never really been healthy.
Now, I must say that, in defense of proposition that Java as a fantastic language for building infrastructure, that the “pain and suffering” I experienced (not really, but I know it looks that way from the outside) in the refactoring of the guts of the Prime Mover infrastructure really didn’t have anything to do with the shortcomings of Java as an infrastructure programming language per se. Indeed, the primary library that I’ve made use of in this project, the ASM byte code engineering framework, has proven to be a superlative tool for doing precisely the kind of stuff that Bob reminisces about WRT the days of the wild, wild west – back when we had to wrangle byte codes with our bare hands and force them to our will using nothing more than our teeth and a screwdriver.
Rather, all the issues I documented wrt Prime Mover have been problems of my own making – in the end, I’m always the worst enemy I have. Far outstripping any puny terror and punishment that Guy Steel or the hordes of Orc programmers over at JavaSoft can dish out.
Well, that was fun. I just spent a decent chunk of my weekend swapping out the fundamental mechanism in the Prime Mover
event driven simulation framework. Between taking care of the little one and the massive allergy attack I was suffering (thanks, Mom Nature!), I think I should get an award or something. Wait! Here’s a gold star I can put on my laptop case.
In any event, the changes to the underlying framework are actually quite cool. What had happened was that Prime Mover was finally getting used in anger – well, gently played with, really – and this exposed some serious short comings in the data flow analysis I was using to perform the byte code rewriting which made the magic happen under the hood. Over a few beers at Bucks on Friday afternoon, we mulled over a few different strategies for fixing the issue – none of which were particularly appetizing.
So, let me step back a bit and lay out what happened to the framework and why.
(Part I – Preliminaries
Part III – The First Cut is the Deepest)
From my perspective, one of the major pitfalls in any project which starts out to produce a management infrastructure is that the project almost immediately starts focussing on the API layer rather than the defining the large scale system behavior. In many ways, this is completely understandable, given that the API has the most immediate impact on the first users of the system – i.e. those hapless fools that form the brigade of developers who have to integrate their systems into your management infrastructure. Given that in most large organizations, APIs become the mechanism that groups use to mediate their interaction – not just at the Java level, but in a visceral sense that governs the actual political interaction between the groups. Somewhat because APIs are something concrete and form a nucleus around which people can argue concretely about. But mostly it’s because most people are rather ignorant of how systems actually interact, but one thing they do know is that there are APIs and consequently these concrete manifestations of handles that can be universally understood become the battleground upon which system integration takes place. Or, put another way, APIs are the lowest common denominator that even managers can understand, consequently they are the only focus of pretty much every large scale project.
But the problem with this focus is that an API doesn’t define a system, rather it’s the other way around. The way I think about it is that the APIs of a system are like the inner core of a sphere. Defining the surface of the system – i.e. what the system “looks” like – will provide enormous leverage on the internals of that system. And this leverage will simply force things into place – meaning that the reason the API exists is because it is literally the inevitable result of the forces that keep the system together.
But I digress.
Update: Part II – the long kiss goodnight, Part III – The First Cut is the Deepest
C’mon in! The water’s wonderful!
This is the first in a series of posts documenting the research I’ve been doing into a different way of thinking about system management infrastructure. For quite some time, I’ve been obsessed with the idea of how to simply and effectively manage large scale systems. Throughout this obsession, I’ve travelled down various roads and found myself in several box canyons along the way. I’ve tried out a lot of different strategies and have finally settled into something which provides the kind of framework I’ve been looking for which I haven’t found replicated anywhere.
Note that I’m certainly not making the claim that it is “Teh Best” management infrastructure. Rather, what I’m making the claim is that it’s the most interesting management architecture to me. As anyone who knows me can testify, I have rather peculiar tastes and I am a strange bird at times. So fair warning, eh?
In any event, what I plan to do is to provide a fairly deep dive into the architecture that I’ve come up with. In the standard tradition of all literature scientific and technical, it will be presented in precisely the opposite order in which I actually came up with things – i.e. from the top down, in a semi coherent form that makes sense. Lord knows that actual discoveries and explorations are more a matter of luck in which you discover something and then spend an inordinate amount of time tracking down why the heck you managed to stumble upon it and where it fits in the larger picture of things that you’re trying to map out. I’ve always found this cognitive dissonance amusing, myself, and hope you won’t mind to much when I veer off into seemingly irrelevant paths rather than sticking to the point at hand.
If you’re one of those people who can’t wait until the end of the story to find out what’s going on, by all means download the PDF of my talk on the subject at last year’s Spring Experience entitled Digging the Trenches on the Ninth Level. If you’re not familiar with Dante’s Divine Comedy, then you won’t get the joke. But suffice it to say that I’m a big believer in the principle that every time you solve a problem, you discover ten more problems that you didn’t know you had.
A perfect example of this sometimes perverse law is something as simple as email. Email solved a lot of problems that a modern economy and social population have, but in doing so it created a lot more. Without email, we would never have been subjected to the sublime beauty of penile extension spam nor would your grandmother be subjected to the horror of id phishing which you discover has snagged her bank account and drained all her life’s savings leaving you with a predicament that makes you wonder what all this progress was supposed to do in the first place.
Likewise, I firmly believe that in solving the problems I believe have been addressed by OSGi, Spring DM and management architectures like mine, we’ve inadvertently unleashed new levels of horror that will ensure future generations will curse our names as they suffer from the fall out and live the unspeakable abominations unleashed from these “solutions” and witness them unfold in ways that we couldn’t possibly imagine.
So, with that cheery panorama as the back drop, I’ll end this introductory post and start working on the next post, which provides high level overview and ten dollar tour of the sewers that I’ve been digging for your benefit on the ninth level of hell.
Remember. I dig because I care. After all, you do want that frozen crap to be routed somewhere and dealt with, don’t you?
My talk at last year’s Spring Experience talk on the next generation of application server architecture is available here.
The talk is about OSGi and how the next generation of application server platforms will simply do away with the cumbersome and rather dated component models that we all know and hate in favor of the vastly superior OSGi platform. Or that’s the theory at least. Only time will tell if I’m correct or just another mad hatter sniffing too much mercury outgassing from the various toys littering his office.
In addition, I also lay out the management architecture I’ve been experimenting with for the past year. Obviously, it uses OSGi as its base, but OSGi – by itself – isn’t sufficient to provide the kind of management infrastructure you need to manage large numbers of processes. I call this management architecture – for lack of a better name – Event Driven Autonomic Management. I’ll be kicking off a series of posts going into far more detail on this architecture as a means of documenting the research I’ve been doing.
Think of it as therapy, as talking about it on this blog – posting, so to speak, to the wind about concepts and issues that no one else seems to find terribly interesting or useful. You can tell I’m a great hit at parties, can’t you?