XTP, JDBC and NIO: It’s a Dirty Job, But Someone’s Got To Do It

xtp-new-and-improved.jpgSo, I was thinking more about the topics in the last post and was wondering about how to go about the transformation of the JDBC implementation and what kind of cooperation would be required with the various driver implementations to pull it off. Originally, my thought would be to make the non-blocking transform at the boundary of the JDBC API, itself. The idea being that there would have to perhaps unpleasant details I’d have to deal with at the driver implementation level and I’d likely need the cooperation of the vendor in some fashion to carry it out.
Boy was I wrong. After playing with a few examples, I found that I can have this framework operate at the java.net.Socket layer and by doing so, I can completely bypass any cooperation requirement with the JDBC implementation, itself, and simply coopt any JDBC implementation as part of the continuation chain – i.e. the JDBC driver is treated just like the application code would be. This means that the transformation is actually even more powerful than I originally suspected and means I can transform any existing blocking code – within reason, of course – into a completely threadless, non-blocking form.
The upshot is that I can now put the X in XTP without even making anyone write a single line of code.

My leg is wet. He’s telling me that it’s raining, but he’s peeing on it.

Wow. I walked into the ballroom for the keynote and found that they had squeezed the room in half from the configuration it was in yesterday. We discussed this amongst ourselves and wondered about it out loud. Sitting down, we realized that perhaps this was intentional, to give the illusion that the room was packed.
Should have been a warning sign I paid attention to.
Sam Ramji got on the stage and did the prerequisite jokes about MicroSoft and Opensource. So it was funny in that uncomfortable way you have when your hated boss tells a decent joke or three in a row. But then Sam settled down into what was clearly a PR powerpoint campaign, letting us all know that M$ was, in fact, a pioneer in open source development and how rather than being something they were against, it was something they invented.
Anyways, it was probably the most boring keynote I’ve ever been to in my entire life and believe me, I’ve sat through some doozies of KeyNotestm at HarkonnenWorld over the years, not to mention the lower strata of Smalltalk conferences in Times Square that I have participated in.
I can’t claim that I was actually surprised by this, but I was disappointed… Contrasted with yesterday’s keynote by FSJ, it was a particularly bad choice for today.

Fake Steve Jobs

Count me as one of those who left the fan base of FSJ after he was revealed. But it was really only for a few months. Well, weeks really. Okay, 6 days. Almost a week. Basically, I couldn’t get away from the god damned blog because everyone kept sending me links and like a heroin junkie, I kept following them.
In any event, FSJ did the opening key note for this year’s EclipseCon. Pretty funny guy even in person – something that I was pretty worried about. Funny in writing is a different skill than funny in real time, in front of thousands. He was excellent.
Sadly, he shied away from the whole “Freetards” schtick that is actually a cornerstone of his blog entries. The Larry as Zod doppleganger bit was priceless. We like to call him the “Baron” here at House Harkonnen, but Zod is a pretty good stand in.

Mass Production Is Not Mass Design

communist-party.pngEric Newcomer has an entry up asking Have We Got it all Backwards with Software Assembly? Basically, Eric comes to the conclusion that

The easy part should be developing the reusable services. The hard part should be their composition and assembly.

Myself, having lived through several waves of “the software component market is going to really take off this time” (Yourdon’s Decline and Fall of the American Programmer comes quickly to mind), I have a rather similar view of this whole debate.
I think it’s a big mistake to think that the analogy to mass production is the reusable software component. Quite frankly, the analogy to mass production is the disk copy – and with the internet, it’s the download. From my point of view, there simply is no issue with mass production in the software business. We can create far more copies of anything than anyone could possibly make use of.
Production isn’t the issue. Design is.

Continue reading

Blade Running

Remember the scene in Bladerunner where Harrison Ford’s character used his multi-function TV thingy to create a three dimensional view of a two dimensional photograph so that he could go poking around in the scene in search of clues? At the time, I was pretty skeptical because I didn’t think there could be enough three dimensional information embedded in a two dimensional photograph to do what he was doing in the film.
Turns out I was wrong. Some very smart people at Stanford have put together a sweet algorithm that takes regular two dimensional photos and turns them into three dimensional scenes. They have a web site that allows you to upload your own photographs so you can try their warez out for yourself.

100% Pure Java OS Process Control

One of the things that frustrates me to no end is the lack of real process management facilities in Java. Sure, it’s a hard thing to do cross platform but for the love of “Bob” why on earth can’t you at least provide adequate models for the stuff that has existed since – I don’t know – THE BEGINNING OF TIME?
Okay, okay. Slack.
One of the fallouts of the lack of apparent support for the management of underlying operating system processes is that you now get C based management forced down your throat. This inherently sucks because now you have platform based dependencies and – even worse – you invariably have the people doing this who are, shall we say, non-OO experts (that is, C++ people who understand Java syntax only because it’s so “C-like”). Having railed against this issue numerous times and gotten nowhere, I finally have had the need to actually produce a 100% pure Java OS process management framework for the project I’m currently working on.
One of the things that never ceases to amaze me about the architecture and design of new systems is the process of shedding preconceptions that happens once you finally get sucked into the system after batting it about from a distance while you’re sizing it up. It’s something I dearly love about this profession in that the process is a transformation of your mind set as well as an act of creation.

Continue reading

OSGi Tutorials

Neil has put up a summary of the parts of his excellent OSGi tutorials at the EclipseZone.

Very nice stuff if you’re interested in learning about OSGi and are mystified by the spec.
Of course, the Spring/OSGi framework makes all of this a piece of cake – ;)

Spring/OSGi integration reaches first milestone

So, one of the projects I’ve been working on has been the integration of Spring and OSGi. This has been the first real open source project I’ve been involved in and it’s been a very good experience. The team is small and everyone is smarter than I am, so that’s always a pleasant experience.
The project reached its first milestone release yesterday. I have to say that this is one of the finest pieces of code I’ve ever worked on. The system we released, while incomplete, is quite solid in what it does implement. Most of the features are there, though, and we’re using the Spring/OSGi integration internally on our own projects here at House Harkonnen.
Costin, the young buck from I21 who’s been doing a lot of excellent work, has a blog entry on the Spring/OSGi integration over at the I21 blog, providing an excellent overview of what we’ve accomplished.
One of the features I’ve been making a lot of use of is the whole JUnit integration. Early on, it became quite clear that it would be a monumental task to mock up an entire OSGi environment. Worse, because we wanted to support three OSGi containers (Equinox, Knopflerfish and Felix), it also became crystal clear that it would be a galactic size task to replicate all the quirks of these containers in a mock environment. So Costin really grabbed the bull by the horns and created one honey of a integrated testing framework for in situ OSGi unit testing. Really, you have to play with it to really understand how very cool it is.
Basically, the issue with any in-container testing is getting your unit test to run inside the container. This requires setting up and starting the container, of course, deploying your test code (in the OSGi container case, the bundles you need for your test scenario) and then there’s the JUnit test itself. Now you have to remotely trigger the running of the test and somehow get the results back from the test run.
Costin’s framework does all this with a very slick framework which sets up and runs the OSGi container in the same process, wrapping your JUnit test class within an automagically generated OSGi bundle. The end result is that you now have in-container testing which can be run as any JUnit test is run – from Ant or from Maven, or from within your favorite IDE. As I said, this is very cool and you have to actually play with it to believe it. There’s nothing even approaching it for any other container.
So, what’s most excellent about this is that I can now structure my build to test the actual services and frameworks with simple mocks, outside of the OSGi environment, due to all the features we all know and love from Spring – i.e. we use Spring to create and configure the services that the bundle contains. Then I can have another Maven module which takes the bundle – which at this point is known to be pretty well functioning – and perform further testing in the OSGi container. As there are things that are simply impractical to mock outside of the OSGi container, such as class loading and dynamic services and events, this results in a system which has been battle hardened right in your build. By the time you actually deploy the code in anger for functional testing, the system has a far greater probability of correct operation. As I said: quite cool.
Anyways, give Costin’s blog entry a read. You can download the Spring/OSGi M1 release, and you should visit the official Spring/OSGi integration project site as well.

Patent office to seek wisdom of crowds

Finally, a common sense idea that may finally have an impact on the rampant abuse of patents we’ve seen in the last two decades:
Open Call From the Patent Office

The Patent and Trademark Office is starting a pilot project that will not only post patent applications on the Web and invite comments but also use a community rating system designed to push the most respected comments to the top of the file, for serious consideration by the agency’s examiners. A first for the federal government, the system resembles the one used by Wikipedia, the popular user-created online encyclopedia.
“For the first time in history, it allows the patent-office examiners to open up their cubicles and get access to a whole world of technical experts,” said David J. Kappos, vice president and assistant general counsel at IBM.
It’s quite a switch. For generations, the agency responsible for awarding patents, one of the cornerstones of innovation, has kept its distance from the very technological advances it has made possible. The project, scheduled to begin in the spring, evolved out of a meeting between IBM, the top recipient of U.S. patents for 14 years in a row, and New York Law School Professor Beth Noveck. Noveck called the initiative “revolutionary” and said it will bring about “the first major change to our patent examination system since the 19th century.”

Well keep our eye on this. Quite frankly I think that a system that hasn’t had a major change since the 19th century is going to find things a might bit interesting here in the 21st and I predict their foray into the wild, wild west internet will result in some nasty bumps.
Here’s hoping they respond with grace.

Rule Development and Knowledge “Engineering”

So, as I mentioned in previous posts, one of the things I’m currently working with is rule systems, in particular, Jess. Regardless of how lame Equinox may or may not be, one of the advantages it has is that the developer of Jess has gone and done the work involved to integrate Jess into Eclipse. This integration is manifest in a syntax sensitive, highlighting editor, a debugger and an inspector for the rete network. Quite cool stuff in its own right.
However, when you’re developing rule systems, the first thing you have to do is start developing an ontology. Unless I’m missing something really basic (known to happen), developing an ontology is pretty close to what architects and designers (even lowly programmers) do in Object Oriented technology all the time. So it’s a whole system that anyone who’s programmed in OO for a number of years will find quite comfortable.
One of the primary differences between an ontology and an OO system in – say – Java is that classes in Java are primarily a means of sharing code. I can’t tell you the number of times I’ve rolled my eyes over programmers and <shudder> architects discussing – in all seriousness – what an object “truly was”. I mean, my god. It’s FRICKIN’ CODE. It’s not a journey into the platonic realm of pure form. Geebus.
With the ontology, things are even more murky. As will be constantly pointed out to you, there is no “correct” ontology for a given domain. In fact, translation between ontologies is one of the big reasons for OWL and other things that use RDF like a dark blood ritual.
In any event, for developing your ontology, there doesn’t seem to be anything better than protégé. It’s a Java based application for developing and manipulating ontologies. Quite nice and definitely recommended.
What’s even better is the integration of Jess into protégé through a plugin called JessTab. With JessTab, you now really have almost the equivalent of a programming IDE for knowledge “engineering”.
Quite cool.