Eric Newcomer has an entry up asking Have We Got it all Backwards with Software Assembly? Basically, Eric comes to the conclusion that
The easy part should be developing the reusable services. The hard part should be their composition and assembly.
Myself, having lived through several waves of “the software component market is going to really take off this time” (Yourdon’s Decline and Fall of the American Programmer comes quickly to mind), I have a rather similar view of this whole debate.
I think it’s a big mistake to think that the analogy to mass production is the reusable software component. Quite frankly, the analogy to mass production is the disk copy – and with the internet, it’s the download. From my point of view, there simply is no issue with mass production in the software business. We can create far more copies of anything than anyone could possibly make use of.
Production isn’t the issue. Design is.
Ford’s assembly line innovation was a production system which made it possible to produce high volumes of something at a constant quality – note that Ford didn’t improve the quality. He just made the quality predictable.
Leaving aside the rather interesting rise of boutique manufacturing which focusses not on mass production but on small runs for small markets, if one looks at how automobile’s are actually created, you find that there’s a (relatively) small team of highly paid, incredibly smart people who spend an awful lot of time designing the automobile in question. It’s only after the automobile has been designed that the mass production aspect we’re so jealous of in the software industry kicks in (and then, only after another small group of highly intelligent engineers figure out how to mass produce the design).
What the software industry seems to be missing here is that the design (and/or architecture) part of the process is pretty much the same as it is in the software industry. The people who can come up with the innovative designs that solve real world problems are always going to be few in number and it’s something that no industry – and I do mean no industry – has ever managed to “mass produce”.
For another viewpoint, let’s look at the electronic hardware world where the idea of “components” is pretty close to the wet dreams of the software component proponent’s idea of a “software IC”. Again, even though there is a proliferation of standardized building blocks, standardized signaling busses…. standardized everything… you still don’t see a lot of “mass design”.
A good example of this is the iPhone. While there’s a number of new innovations in the phone itself – novel touch screen technology, though mostly packaging – the phone is put together with pretty much standard parts – i.e. Apple didn’t reinvent electronic components to produce the iPhone.
One thing that has become more and more clear to me over the years is that the real problem with software isn’t that we don’t have enough standardized tools and components. Eric does have a good point about POJOs, but from my POV, we’re still well in the dark ages from a design tool perspective that will really “unleash” the programmer in all of us.
No, the sad thing is – from the POV of those who are trying to commoditize the software industry – is that we have already pretty much hit zero cost in the production of software. Adding more cheap people to the mix who are just smart enough to push the big red button isn’t going to help things (and if people would ever remember their Mythical Man Month, they would stop thinking about this silliness altogether – but that’s simply never going to happen).
Rather, I think the capitalists who control the means of production are going to start figuring out that designing software is only going to get harder and require smaller teams of even more intelligent people. Find anyone who’s been in the software industry for a decade or longer and ask them if they want more people to work on their software project or smarter people.
And yet, we still have this bizarre notion in the software industry that – unlike every industry we’ve ever seen – somehow throwing armies of people who are literally an order of magnitude (and likely several more) worse at designing software will somehow compensate for paying top dollar for the small number of people who actually know what they’re doing. You can look at the state of the software industry and the general quality of what we produce to see how well this strategy is working out.
Maybe in the near future we’re going to figure out some magical set of components which will make all our lives orders of magnitude easier and provide the industrialists with their ultimate dream of high quality software manufactured with drones of low wage programmers, but I think it highly unlikely.
Software is hard to do and it’s never going to get much easier in my lifetime. Sure, we’ll solve some cool stuff that will become standardized components and these will allow the smart people to do even more. But the simple fact is that software development is getting harder to do as it moves more into the distributed realm, not easier.
I mean, the sad fact is that most programmers out there can’t even get a single threaded program to work, much less make it solve some hard problems. Add multi-threaded concurrency into the mix and I would guess that probably 60 to 70 percent of programmers out the wild will splat against the wall and slide down it like a wet sponge. Throw multiple cores into the mix – just keep it on the same machine – which introduces true concurrency into the mix and probably another 25% will join their fellows by sliding down the wall they just smashed into.
Throw fully distributed computing into the mix and I think you’ve winnowed down the current field into 1/100 of what you think you have out there (and I’m probably being optimistic).
It is very odd, to my way of looking at things, to think that there is some imperative to make “programming easier” – like it’s a natural “law” that programming must be possible for people who don’t have the inclination. It seems like it’s pretty hard for the people who make most of the money off this stuff we do to come to grips with the idea that this stuff may well be intrinsically hard to do and that while we will be successful at producing better and more efficient ways of enabling smart people to be even more productive than they already are, we most likely won’t be successful at making it possible for the masses to produce the ever more sophisticated kind of software that the marketplace is demanding.