Negotiating With a Protocol

This, imho, is pretty much the portrait of a corporate “solution” to the “problem” of P2P
Comcast and BitTorrent: Why You Can’t Negotiate with a Protocol

The problem is that you can’t negotiate with a protocol, for the same reason that you can’t negotiate with (say) the English language. You can use the language to negotiate with someone, but you can’t have a negotiation where the other party is the language. You can negotiate with the Queen of England, or English Department at Princeton, or the people who publish the most popular dictionary. But the language itself just isn’t the kind of entity that can make an agreement or have an intention.
This property of protocols — that you can’t get a meeting with them, convince them to change their behavior, or make a deal with them — seems especially challenging to some Washington policymakers. If, as they do, you live in a world driven by meetings and deal-making, a world where problem-solving means convincing someone to change something, then it’s natural to think that every protocol, and every piece of technology, must be owned and managed by some entity.

I think we’re at the threshold of one of those eras where the old guard simply doesn’t understand the rules any more and is reflexively dealing with the new reality using all the power of its impotent tools. Sure, a frustrated Lizard – a HUGE frustrated lizard, mind you – is pretty terrifying to be on the wrong end of. However, the poor system simply can’t comprehend what’s happening to it. All it can do is lash out, find someone to take to lunch and negotiate a contract with.
Sad, really.

threeheads.jpg

“I Just Want To Interoperate”

So the other foot drops in the Microsoft “Open Source” debacle. When I was at EclipseCon, we had Sam Ramji of Microsoft regale us about how MS wasn’t just going to play in Open Source like everyone else, they had in fact invented Open Source. It was a talk so boring, that I left half way through it.
In any event, today we find out how open the source is.
Royalties are the admission price Microsoft tells freetards

Microsoft wants to license Windows patents to open source companies in the same way it’s licensed patents to companies like Motorola in the past. “Because cathedrals can do agreements with each other its possible to sit down with the companies we have and say: ‘Let’s see what we can work out that works for you and our business’.”
Smith was borrowing the phrase “cathedrals” from Eric Raymond’s book The Cathedral and the Bazaar, which talks about the open source, or bazaar, method of development versus the traditional vendor approach. “We’d be prepared to sit down with any entity that can deal with the issue in real terms,” Smith said.
It’s a vital emphasis, and one that could harm technology and business innovation in open source. Many open source products and businesses today have begun life with individuals or groups of individuals working on projects, unencumbered by worries about the ability of their project to support royalty payments to a patent owner down the line.
“We are much better connected with the open source community today, we love open source software running on Windows and we are working to interoperate with it,” Smith said. ” But I can’t give you an answer saying: ‘Here’s the blank check,’ he told OSBC.
Having made Microsoft’s position clear, Smith called for a willingness in the open source community to compromise in negotiations and solve problems. In translation, that appeared to mean: stop requesting publication of all Microsoft patents under a royalty free license. According to Smith a solution can be reached to “normalize the IP relations” to “reach almost all spectrums”

cat-and-mouse.jpg-1.jpg

A Couple Good Books

Despite living in the online digital age where PDF rules, and despite loving my Kindle, I find that pretty much nothing beats a book – still. And although my vast library still has books largely unread, I can’t help but buy more. Myself, I find that the number of books I have unread and thus undiscovered in my library a more pleasant metric than the books I’ve already read. Sure, I treasure the books I’ve already tasted, but the ones I have yet to discover are the ones that give me pleasure…
Anyways, I got two very good books – as yet unread, naturally – from Amazon today and I can’t wait to start applying and diving into both of them.
The prize of the two is Foundations Of Multidimensional and Metric Data Structures. It’s a 1000 page, oversized book on just what the title proclaims: a comprehensive work on multidimensional and metric data structures. Amazing stuff and quite dense – both literally and figuratively. The paper is that fantastic, acid free glossy textbook paper that can be used as a bullet proof fabric substitute in a pinch.
Anyways, it’s a 2006 copyright and it’s absolutely stunning in its depth. As the foreword reads, this book is encyclopedic, organizing a bewildering array of spatial and multi-dimensional indexing methods into a coherent field. Very cool stuff, to say the least.
The second book promises to be a very good read as well: 3D Game Engine Architecture – Engineering Real-Time Applications with Wild Magic. Although the book is about a specific engine, it’s really – as one reviewer put it – the longest and probably best done architecture documentation ever done. I was looking for a book on the subject that really went into the architecture of such a system, not just a mere retelling of how great the code was and sparse commentary around it. From what I can tell so far, the book seems to live up to this promise.

XTP, JDBC and NIO: It’s a Dirty Job, But Someone’s Got To Do It

xtp-new-and-improved.jpgSo, I was thinking more about the topics in the last post and was wondering about how to go about the transformation of the JDBC implementation and what kind of cooperation would be required with the various driver implementations to pull it off. Originally, my thought would be to make the non-blocking transform at the boundary of the JDBC API, itself. The idea being that there would have to perhaps unpleasant details I’d have to deal with at the driver implementation level and I’d likely need the cooperation of the vendor in some fashion to carry it out.
Boy was I wrong. After playing with a few examples, I found that I can have this framework operate at the java.net.Socket layer and by doing so, I can completely bypass any cooperation requirement with the JDBC implementation, itself, and simply coopt any JDBC implementation as part of the continuation chain – i.e. the JDBC driver is treated just like the application code would be. This means that the transformation is actually even more powerful than I originally suspected and means I can transform any existing blocking code – within reason, of course – into a completely threadless, non-blocking form.
The upshot is that I can now put the X in XTP without even making anyone write a single line of code.

Non Blocking JDBC, Non Blocking Servlet APIs and Other High Mysteries

Most everyone who works with server infrastructure has figured out how to use NIO – in one form or another – to transform the standard blocking threaded server model into a far more scalable beast. Without going into boring details (see here, for example, if you need to investigate), the executive summary of the transformation is that previously servers had to pretty much dedicate a thread per inbound socket which would basically block on the socket between requests, waiting in desperate silence for some bytes to show up from the client. The reason why this had scalability issues is that the server had to dedicate a thread per client and threads are really frickin’ expensive resources – especially if most of the time they are blocked waiting for the client to do something.
What NIO allowed server architects to do was – at the simplest level – dedicate only a single thread to handle the waiting for a client socket to become active. Once a client socket was found which needed attention, this socket could be handed off to a thread pool where a thread could be picked up to handle the incoming request. This has a quite amazing effect on the scalability of the server as most of the times clients don’t do anything at all – something captured by “think time” in simple benchmarks. And since you’re blocked doing nothing, you can’t use the valuable thread resource to actually accomplish work for some client which actually does need your attention.
Now, this is actually pretty good stuff and has enabled infrastructure to scale far beyond where the old, blocking, architecture could ever touch. However, what’s pretty much universal in these server architectures – and perhaps is a well known dirty secret – is that once you get into the actual request processing, the system is back into the old same blocking regime. You can see this trivially, for example, in the Servlet API. When you – the application code – try to read some bytes from the client using the ServletInputStream, you’ll find that if the client hasn’t sent the bits, or the network is slow in getting them, or any number of reasons, your code will block waiting for those bits to show up. Likewise, if you try to write some output bytes to the client, using your ServletOutputStream, you’ll find that your code will be blocked until the OS can buffer those bytes for later output to the network. If those buffers are full you’ll simply block until they become available, or the client disconnects.
This isn’t any big revelation, but it is surprising to listen to people wonder why we can’t use NIO to take care of these issues so we can have even more scalability in the server. I mean, if the API is blocking – as all java.io.Stream APIs are – it’s kind of baffling to think that someone believes that we can magically make that blocking action go away and somehow eliminate the thread which is holding the execution state of the application code (and server invocation code) up until the point where the blocking read or write occurs.
Well, I’m pleased to say that I believe that I’ve discovered a way to magically make the blocking reads and writes “disappear” without having to change any of the JEE Servlet APIs. Well, “discovered” really isn’t the right word, seeing as how the technique is not really any new revelation to people “in the know”. However, doing this magic for pure and simple Java without redefining the existing APIs, nor requiring the application code to be rewritten in any way is something that I think is pretty magical and certainly at least deserves the characterization of a “discovery”. Further, this technique can be applied to JDBC or any other high value blocking API to transparently transform it into a threadless, non blocking API.

Continue reading

My leg is wet. He’s telling me that it’s raining, but he’s peeing on it.

Wow. I walked into the ballroom for the keynote and found that they had squeezed the room in half from the configuration it was in yesterday. We discussed this amongst ourselves and wondered about it out loud. Sitting down, we realized that perhaps this was intentional, to give the illusion that the room was packed.
Should have been a warning sign I paid attention to.
Sam Ramji got on the stage and did the prerequisite jokes about MicroSoft and Opensource. So it was funny in that uncomfortable way you have when your hated boss tells a decent joke or three in a row. But then Sam settled down into what was clearly a PR powerpoint campaign, letting us all know that M$ was, in fact, a pioneer in open source development and how rather than being something they were against, it was something they invented.
Geebus.
Anyways, it was probably the most boring keynote I’ve ever been to in my entire life and believe me, I’ve sat through some doozies of KeyNotestm at HarkonnenWorld over the years, not to mention the lower strata of Smalltalk conferences in Times Square that I have participated in.
I can’t claim that I was actually surprised by this, but I was disappointed… Contrasted with yesterday’s keynote by FSJ, it was a particularly bad choice for today.

Fake Steve Jobs

Count me as one of those who left the fan base of FSJ after he was revealed. But it was really only for a few months. Well, weeks really. Okay, 6 days. Almost a week. Basically, I couldn’t get away from the god damned blog because everyone kept sending me links and like a heroin junkie, I kept following them.
In any event, FSJ did the opening key note for this year’s EclipseCon. Pretty funny guy even in person – something that I was pretty worried about. Funny in writing is a different skill than funny in real time, in front of thousands. He was excellent.
Sadly, he shied away from the whole “Freetards” schtick that is actually a cornerstone of his blog entries. The Larry as Zod doppleganger bit was priceless. We like to call him the “Baron” here at House Harkonnen, but Zod is a pretty good stand in.