Declarations and Self-Evident Truths for Enterprise IT
Like many holidays, the meaning of Independence Day is often overshadowed by its commercially derived incarnation: Cookouts, unsubtle retail sales and widespread detonation of pyrotechnics seem to obscure the original purpose of the celebration.
So, as we approach this fourth day of the seventh month, I return to the founding document itself for a semi-occasional read. As a citizen, I can see the power, purpose and poetry (TJ was nothing if not deft with the pen). As a writer, I can see—perhaps shamelessly—an angle for my ongoing ruminations on IT.
Which brings me, with tongue firmly pressed in cheek, to the following declarations:
Be it known that:
IT is beset by hardships, three in number:
I. Abundant and virulent growth in the number of systems that must be tended.
II. The unending cycle of changes, both great and small, caused by enhancements to and repair of defects in software systems.
III. A drought of IT resources, the volume of which number far fewer than harvests past.
Acknowledging thus, be it said that:
We, who carry out the duties of IT and depend upon such duties to carry out those of our own, are bound by certain truths that we hold to be pretty darn evident.
Certain among them:
I. The truth that development and operations groups are created equal, endowed of different but no less important capability and purpose, which together should create a greater and more powerful one, rather than a divided and conflicted two.
II. The truth that the pursuit of speed and agility as the principal and guiding aims for development shall not conflict with the pursuit of control and predictability as the aims for operations. Whereas these pursuits were once held in mutual opposition, they can be so held in agreement and to the comfort and security of both parties.
III. The truth that IT operations must make all reasonable efforts to excise unnecessary cost and complexity through conscientious reinvention and liberal use of machines to carry out certain lower-order duties once performed by the hand of man.
IV. The truth that, in the pursuit of such automated ends, IT shall not sacrifice the duties and talents of such workers, instead allowing these able men and women to dispatch functions of higher utility and station to the highest welfare of the business.
V. The truth that, in the further effort to excise cost, IT shall entrust applications and certain other technical assets to the hands of service providers where the benefits of such trust outweigh the taxes levied thereby.
VI. The truth that IT is no longer ancillary to the business, but in even the most rank and ordinary of circumstances, often the very heart of the business itself.
These truths bind us in the common view and conviction that IT must discard ways and manners of our history, however commodious and familiar they may be. They must replace such familiar ways with new ways, better in swiftness and economy.
It is upon these declarations that we set forth to a new day for IT.
Disintermediation of IT: Lessons from “Bubble 1.0″
**Register now for a related webinar on 6/24: “Focus on IT Agility”**
Back in the silly season of the late 1990s, all measures of reasonability were suspended in favor of a punch-drunk land grab for “eyeballs” on the path to IPO. While e-commerce was absurdly hyped, it was also an important new context for selling stuff that was bound to disrupt the established value chain.
E-commerce promised to take the friction out of the value chain—a more efficient way to connect supply with demand. To call a retailer “bricks and mortar” was the highest form of degradation; these out-of-touch progenitors of modern business were doomed to become footnotes in the annals of commercial history.
But no participants in the value chain were more maligned than the wholesaler or distributor. These folks made money by aggregating and matching supply and demand. But in the age of online commerce, was this role necessary? Many thought it wasn’t—and from this premise, their obituaries were written.
The fancy word for this was “disintermediation”—a fundamental shift in the value chain that rendered a set of participants irrelevant. It’s a concept that got a whole lot of ink in now-defunct and previously high-flying “new economy” publications.
Fast forward to 2010, and the same sort of dynamic is at work again.
Cloud computing—specifically, swipe-a-credit-card forms of public cloud—promises to disintermediate enterprise IT. As the story goes, in the face of long delays, business lines and developer groups will follow the path of least resistance, deploying their workloads to the public cloud. Applications will go rogue and traditional IT will become irrelevant—disintermediated by the cloud.
Do I believe it? Yes and no.
Bubble 1.0 did not change everything—but it did force everyone to change.
Participants in the value chain willing to rethink their roles and value propositions capitalized on the shift. Those who dug in their heels and shook their fists in angry protest? They’re featured in the footnotes.
There is a sober lesson for IT buried in the drunken silliness of Bubble 1.0: Don’t fight the forces of change. Some degree of skepticism is certainly healthy, but look at these punctuated shifts as new opportunity—the natural course of evolution.
That’s exactly what we’ll cover on Thursday, June 24th in a webinar discussion bringing together CTOs of newScale, rPath and Eucalyptus Systems to envision the future of IT in the age of cloud computing. Be sure to register to participate!
The premise of the discussion: IT must transform into something that looks a lot more like a public cloud. At the heart of this transformation is a new tool chain that allows IT to change the economics of IT service delivery and to remove the friction from the IT value chain. IT becomes a self-service provider of on-demand business, infrastructure and compute services.
So, will cloud computing change everything for IT? Probably not—but it will force everyone to change. Those who don’t change? We all know how that story ends.
The Changing Role of the IT Hero
Every IT organization relies on a handful of seriously talented technical artisans to take on the trickiest tasks in the data center. These folks are the script wranglers, the IT cowboys (and girls) who—through a brilliant infusion of intellect, competitive spirit and occasional subversion—make the data center work. They’re the heroes who make disproportionate contributions to the trade and we love them for it.
But times are changing and the role of the IT Hero is being reconsidered.
That is not to say that the IT Hero is any less loved or valued; in fact, the need for the IT Hero will persist ad infinitum. It’s just that circumstances are forcing IT leadership to look beyond the preternaturally gifted few to run the data center. As system scale and complexity compounds, change accelerates, and business lines demand more—faster—IT needs to clear this bottleneck.
Traditionally, the IT Hero wrote the scripts that made IT processes scale. But as software and deployment environments become ever more diverse and dynamic, these scripts grow like kudzu. What began as a way to economize IT has become a large cost center in its own right: Today, maintaining scripts is often as costly as maintaining IT systems themselves.
And guess who manages these scripts?
Perhaps this cartoon puts it best (source: http://geekandpoke.typepad.com/): 
That’s the IT Hero: Brilliant, but overly entwined with the fitness and survival of the data center. What happens if he or she leaves the building? What happens when scale and change explodes? Ever try recruiting heroes? They’re few and far between. Of course this ignores the more obvious point that IT has little appetite for any significant hiring in this age of “doing more with less.”
Beyond the sheer opacity and inscrutability of scripts, there are a variety of technical reasons this approach to automation fails in a time of scale and change. I won’t cover that ground here—it could consume many pages of commentary, and it’s expertly discussed by Erik Troan in his two-part series on the topic.
But the key point is that IT needs to shift from script-based to model-driven approaches to automation. While scripts provide detailed programmatic instructions that automate detailed procedures for how to perform a rote task, a model-driven approach allows IT to get out of the weeds and focus on the definition of the desired end-state rather than the exact steps to get there. Once the end-state is defined, the procedural execution is automated.
This model-driven approach to managing IT systems has several key advantages:
• IT Heroes can delegate—Traditionally, the thorniest IT issues have quickly escalated from front to backline support—the domain of the IT Hero. Model-driven automation abstracts complexity, putting control in the hands of a broader set of IT personnel and allowing heroes to deflect many of the inbounds.
• IT Heroes can focus on higher-value tasks—Expending effort to write and maintain low-level scripts for undifferentiated processes—in other words, processes that cannot yield a sustainable competitive advantage—is a wasted use of talent. This evolution allows IT Heroes to focus on higher value, more strategic contributions.
• IT leadership can sleep at night—With comfort that the knowledge of how processes are automated is explicit and documented, rather than implicit and trapped between the ears of the smartest few. If the IT Hero departs to Google, the lights stay on and data center processes continue to function.
What’s this mean for the best and brightest in IT? More interesting challenges to tackle, more business value to deliver. The truest IT Heroes are hardly threatened by their changing role. They’re up for the challenge, inspired to lead the way forward, and to prove that their value and contributions cannot be commoditized. The IT Hero lives on.
The Invisible Thread of Innovation
IBM Rational’s user conference, “Innovate 2010,” kicks off today with a bold and inspiring message about the importance of software development: “Software is the invisible thread of innovation.” It’s no doubt correct; software has become absolutely vital—in many cases, core—to the execution of business. For a new generation of web-oriented companies, software is nothing short of the business itself.
If you ask me, this is a message development organizations should embrace with the pride of a job well done—and IT operations should embrace as a catalyst for change.
Why? Because, today, there’s an impedance mismatch between dev and ops. Dev churns out new software and changes far more quickly than IT can respond. Dev produces this invisible thread of innovation, which IT struggles to consume.
IT often lacks the tools, process and culture to match the complexity, scale and change of software in production. This is the argument behind the DevOps movement, which has done a great job of shining light on the dark space between dev and ops.
It’s also an argument for the software distribution hub, which is about drawing the invisible thread of innovation from the source of software—custom, commercial and open source; to deployed business services—across physical, virtual and cloud. When changes are made to the software sources, it’s seamlessly flowed through the hub and to deployed systems and business services.
In this manner, this approach is about managing the entire software supply chain as an integrated thread—from the origin of the software to the business service in production.
The message? Software is undoubtedly the invisible thread of innovation, but innovation isn’t realized in dev—it’s realized in ops. This is why this thread must become less of a conceptual whiteboard ideal and more of an IT supply chain reality.
Envisioning A Software Distribution Hub
Modern manufacturing is a direct consequence of a forced transformation—a transformation resulting from changing economics and massive product and supply chain complexity.
The same sort of transformation is taking place today in enterprise IT—forced by similar challenges and following a very similar arc.
Manufacturing has evolved from manual to automated, ad hoc to repeatable. From art to science. Manufacturing has been forced to “industrialize.” Aiding in the effort are product lifecycle and supply chain management tools for streamlining manufacturing process and linking together previously isolated stages in the value chain.
Several factors drive manufacturing complexity:
Many inputs—raw materials and product sub-assemblies are sourced through direct and indirect means. Inputs must be made available for production runs.
Many actors—each phase of the manufacturing process involves many different roles and individuals who must work in harmony to produce the end product.
Massive scale—the volume of units shipped and diversity of product lines drives massive scale in the volume of raw materials, work in process and finished goods.
Constant change—changes in the specification and availability of inputs must be communicated and propagated downstream, while changes in demand are communicated and propagated upstream.
Frequent specialization—end products must be tweaked, tailored and localized to satisfy diverse market requirements.
Manufacturing is a useful analogy for the transformation taking place in enterprise IT. After all, these same challenges—many inputs, many actors, massive scale, constant change and frequent specialization—could just as easily describe the state of IT today.
Manufacturers have addressed complexity by creating a centralized infrastructure for controlled reuse of standardized components, collaboration between roles, and automation of key processes. They have created, in effect, a “hub” that automates and streamlines how inputs become outputs in a way that is efficient and scalable and based on principles of standardization and reuse.
They have integrated these standardized components all the way back through the supply chain.
IT has no such hub today, relying instead on independent, often duplicative efforts and disconnected, fragmented processes to construct and maintain a system.
For IT today, scale and change are compounding at the same time budgets contract and business line expectations increase. IT is undergoing a forced transformation—from art to science—much like manufacturing organizations of the past.
The transformation for enterprise IT should follow a similar path, where a centralized hub becomes the control infrastructure for managing IT system construction and change. This hub becomes the basis for managing and reusing standardized system components—components that are integrated back through the software supply chain.
You may think of this like a Rubik’s Cube, where the colored blocks represent reusable software components which—when combined—represent a new system.
These components are managed centrally and reused liberally to construct new systems. All the components and the systems themselves are deeply modeled and version controlled, ensuring that they’re well defined and controlled throughout their lifecycle. When updates are made to the components, change is flowed through the hub and cascaded en masse to hundreds or thousands of deployed systems.
With a software distribution hub, key challenges for IT become surmountable:
Many inputs—system software components are standardized and versioned, giving IT control over what software is put into production. The software supply chain is fully integrated—from the deployed business services all the way back to the original sources of the enabling software—custom, commercial and open source. Changes to software and configurations flow seamlessly from the sources to the consuming systems.
Many actors— rather than duplicating efforts, working at cross-purposes and creating conflicts in production environments, all contributors to the IT value chain are in perfect coordination. Standardized components are reused, not recreated. Where they’re reused is tracked. Changes to these components—in dev, test or production; by OS, middleware or application teams; or by third party software providers—is seamlessly propagated from their source, through the hub, and into production.
Massive scale—traditionally, adding systems meant adding people. With the software distribution hub, scale can grow at near zero marginal cost. Reuse of system components means massive leverage on the management of software in production.
Constant change—with change flowing through the hub, introducing change at any layer of the stack is simple, free of conflict and reversible. Changes to custom, commercial or open source software—or to configurations or business service definitions—seamlessly flow from their source to their destination.
Frequent specialization—one-size-fits-all gives way to customized systems, as standardized components are mixed and matched to address diverse business requirements. Because these components are controlled and managed centrally, there’s no added management overhead for this sort of system diversity.
Legacy manufacturing processes were forced to transform under the weight of complexity, scale and changing economic conditions. The same transformation—by the same forces—is underway for enterprise IT.
Today’s approaches are too slow, costly and chaotic to address modern system complexity and IT budgets. The only viable solution is a new approach—a distribution hub, with deep integration from the business service back through the software supply chain—that takes the cost and complexity out of managing IT environments in an age of massive scale, accelerating change, and elusive op ex budgets.
