Why Models Beat Scripting
I’ve made a couple of posts recently about Why Scripting is Evil. It has generated some conversation about what a script is, and what the alternatives are. One commentator must have anticipated this follow up post when he said:
So, somewhere in the process, a script is a necessary object, whether it is explicit (written by the user) or inherent (created through abstraction and based on the user’s description of desired outcome).
In short, the only way to do away with scripting is to create an application that abstracts the scripting process. Automated scripting would then be based on user descriptions of the desired outcome/goal rather than having the user describe the steps to reach said goal.
This really gets to the heart of how things need to change. I wouldn’t call it ‘abstracting the scripting process’ any more than I’d call a web browser ‘abstracting the network,’ but that doesn’t make it inaccurate.
What is exactly right is his characterization of the ideal state: user descriptions of the desired outcome/goal. We call this approach model driven. What that means is administrators can describe how a system should look, and let tools move the system from whatever state it’s in to the end state automatically.
This distinction between model driven and script driven has been evolving for a while. Tools like cfengine and puppet have supported model-driven configuration for years (the puppet web page describes itself as having a ‘declarative language’). They give administrators a language to (for example) specify the DNS server for a machine; the tools are responsible for knowing how to update the right files on the system in a safe manner. Before puppet and cfengine a simple tool called rdist was state of the art for managing distributed systems, and it was entirely script based. You don’t see it much anymore; rdist scripts are much more brittle than the more popular model based approach.
It’s probably obvious why I like this approach better than writing scripts for each change:
- Models are understandable. It’s much easier to look at a model and know that it’s correct than look at a script which implements the model.
- Models can be verified and have policies enforced against them.
- Models have a much simpler testing matrix.
- Models are invertible; just move back to the old version of the model.
- A model based approach supports a high level of rapid change and automation
All of this assumes you have a reliable way of applying the model to a production machine–but once you do, it works over and over. You’re splitting the what from the how. It’s exactly the same idea which lies beneath the Model-View-Controller paradigm, and it’s right for exactly the same reasons.
When I was speaking with a prospective customer last week they described the same idea using slightly different language. They described their goal as being able to completely describe the “goal state” of the system, and have “viewers” which could see how a system differed from the goal state. “Controllers” would be responsible for moving a system to its goal state. This is exactly the same idea as model based system management, just using different language (I’d just about finished reading Neal Stephenson’s book Anathem, so I obviously found the idea of systems moving between points in a Hemn space quite appealing!).
Model based apporaches are a compelling alternative to scripts. It’s the right approach for managing large numbers of machines without employing large numbers of people.
Is Op Ex the New Cap Ex?
Virtualization and cloud computing have surged on a promise for dramatically reduced capital expenditure through improved utilization of physical infrastructure—or the total elimination thereof. But, is this promise overstated? Perhaps.
Take virtualization, for example. On the surface, the cap ex ROI is a no brainer: a 4-5 X improvement in server capacity means fewer servers to buy. Right?
If history is prologue, the improvements in the economics of computing result in—not less spending—but a proportional increase in scale. When capacity is cheap, companies can justify deploying more workloads. This argument is corroborated by the Gartner Dataquest Worldwide Server Shipment Forecast, which indicates a sustained increase in shipment of physical servers over the next several years.
As Aristotle noted, “Nature abhors a vacuum.” In other words, when space is made available, things will always expand.
So, perhaps the cap ex reduction is a myth. But dig deeper and you’ll see the more strategic win: Projects that were once too speculative to cost justify are now fair game. Cheaper capacity means more innovation and responsiveness to business.
It also means a dramatic increase in the number of workloads that need to be managed. I’ve long contended that virtualization and cloud may be a cap ex boon and an op ex bust. Of course I’m now rethinking the “cap ex boon” (which is a shame because it sounded really great), but my contention about the “op ex bust” is firmer than ever.
Others are finally taking notice. Op ex may well be the new cap ex.
The perfect storm of massive scale, accelerating change and contracting budgets all point to op ex as the current devil for enterprise IT. This explains the renewed interest in automation and the rise of movements like DevOps—both aimed at dealing with scale, the need for speed, and mandates to do more with less.
What does this mean for IT personnel? The simplistic conclusion would be that they better freshen up their resumes. But, like the cap ex argument, that’s too hasty a conclusion. It means that they’ll have the tools they need to deal with scale. IT may not be in a great rush to hire these days, but the resources they have? They’re more important than ever.
What is a Script Anyway?
Shame on me. I didn’t define my terms.
Some folks were unhappy with my post about the evils of scripting, but not for the reason I expected. Instead, they thought I was attacking scripting languages. I’m a big fan of interpreted languages (same thing, different word); I’m the guy who put Python into Red Hat Linux in 1995, got the engineers building anaconda to use Python, and am building a company whose core IP is 120,000 lines of Python. Dynamic languages are fantastic, interpreting versus compiling is a nondiscussion (really, who cares?), and using #!/bin/sh to avoid some repetition makes perfect sense.
Now that I’ve described what I do like, let me talk about what I don’t. Specifically I meant to be talked about the merits of using scripts to configure servers. Installing software and tweaking config files are things I think scripts are bad for. So let me say what I mean by a script. Read more
Why Scripting is Evil
(this is the first in a two part series; the second part will be ready. later.)
There are two kinds of people in the world. Those who divide the world into two kinds of people and those who don’t.
Okay, and old joke, but I’m clearly the first kind of person. I try and split everything into two buckets. System automation solutions lend themselves to this two sizes fits all mantra, with the approaches splitting between scripting solutions and model based approaches.
It seems like most people think about scripting as the best approach to automation. Whether it’s simple shell (or PowerShell) scripts, attaching scripts to Opsware machine definitions, or running an inscrutable perl command line, scripting is king. Engineers and system administrators tend to break problems down into steps, and scripting is a way of codifying those steps and running them on lots of machines. Read more
Is “Agile ITIL” an Oxymoron?
Agile ITIL?
For many, that sounds utterly oxymoronic—like “jumbo shrimp” or “unbiased opinion.”
Agile: Lean, automated, adaptive.
ITIL: Lumbering, bureaucratic and rigid.
These are mutually contradictory, nonsensically combined concepts. More oil and water than peanut butter and chocolate. Right?
ITIL is about control, compliance and predictability. It’s about uptime and stability.
Agile? It’s about speed and change—the mortal enemies of uptime and stability.
So, what happens when Agile meets ITIL? Today, worlds collide.
Part of the issue is rooted in the divergent cultures and motivations of dev and ops:
- Dev: freewheeling, ideological, collaborative, self-organizing, and organic
- Ops: policy-driven, command-and-control, focused on uptime and compliance
When dev meets ops—when Agile meets ITIL—the velocity gains in application development are lost, caught in the purgatory of long, cumbersome release cycles.
Worse still is change. For IT, change portends nothing but punctuated pain and a passel of unintended consequences. Change is empowering for dev and dispiriting for IT.
According to Israel Gat, who, together with Michael Cote, blog as The Agile Executive:
1. The business needs to respond to change quicker than ever.
2. Various traditional IT management methods tend to discourage change and slow it down.
3. Competent Agile dev/test teams are now able to respond much quicker to the changes required by the business. The time gained in dev/test, however, could be completely wasted if IT Service Management fails to become (more) agile.
And there’s the key: When IT puts on the breaks, velocity gains—responsiveness, agility—are nothing but false hopes and empty promises. Agile becomes fragile. Agility becomes atrophy.
Gat proposes what he calls Agile Business Service Management—extending agility to IT.
In fact, Dr. Gat participated in an rPath webinar last year on this very topic—back when it was, if not radical, certainly something ahead of mainstream thinking.
Today, it’s the catalyst behind one of the frothier topics on Twitter: #DevOps.
The reality is ITIL is poised to get lean—to become agile.
Lumbering, bureaucratic and rigid are out of step with today’s high-velocity and low-overhead needs. Compounding scale, accelerating change and the need for speed have conspired with lean mandates and elusive budgets to put traditional IT practices on notice.
In support of this are movements like DevOps and manifestos like Visible Ops, which draws out the best principles of ITIL and presents them as specific, pragmatic steps toward the goal of high performance IT. At the same time, there’s something of a renaissance underway in IT automation software. This has further stoked the fire, helping advance these movements beyond concept and ideology and making ITIL agility achievable.
This is the topic we’ll cover in a webinar on Thursday, April 15th featuring two of the principal voices at the intersection of DevOps and Visible Ops: Damon Edwards, president of DTO Solutions and editor of Dev2ops.org; and Kevin Behr, co-author of The Visible Ops Handbook, which has fast become the bible for the thoughtful and progressive practice of IT.
Be sure to register for the webinar and join us on Thursday, April 15th.
So, is Agile ITIL an oxymoron? Perhaps. But that’s sure to change—it has to change. Soon, these words in combination will sound perfectly natural. But that’s just my unbiased opinion.
Enterprise IT’s Rise from the “Muck”
Amazon founder and CEO Jeff Bezos has often said that 70% of IT cost and effort is attributed to the “muck,” the heavy lifting that is so critical, but undifferentiated in delivering value. Of course, he’s making his case for public cloud—let Amazon don the hip waders so you don’t have to.
So it’s no surprise to hear Amazon CTO Werner Vogels beating the same drum.
Dan Wood’s Forbes.com post “$10 Million is the New $100 Million” examines what Vogels calls the “undifferentiated heavy lifting” that once burdened startups and is now obviated by the rise of cloud computing as an alternative to the costly and complex muck of IT infrastructure.
Woods rightly extends this “anti-muck” argument to the less trodden territory of application complexity. His excellent post “Virtualization’s Limits” calls application and other software complexity the next barrier to clear in this rise from the muck.
But I would take his story one important step further: These economies aren’t just about new innovations and making startups more capital efficient; they’re also the catalyst for the fundamental transformation of enterprise IT that’s happening today.
Woods isn’t alone in his campaign to shine light on complexity. The DevOps movement is gaining momentum. And IT automation is all the rage. Why? Because the world has changed in profound and fundamental ways:
- System scale is compounding by orders of magnitude
- IT is under pressure to become rapidly responsive
- Op ex budgets are adjusting down to the “new normal”
As such, IT is looking for ways to use automation to manage the complexity of software—to abstract away from the muck of enabling infrastructure and focus on applications and business services that deliver real and measurable value.
This is leading to what James Urquhart describes as “an operations disruption,” a new look at the requirements for managing software systems in modern IT. As a vendor at the center of this ops disruption, rPath sees it first hand: Demand from enterprises has surged as they look for ways to adapt to the new normal.
In general, the broader vendor community is responding to this demand with new offerings of their own. CA and VMware, in particular, are diving in headlong. Others sit more passively on the sidelines, trying to determine the right moves to protect their legacy franchises (if you’ve read Clayton Christensen now-classic text, you’ll recognize shades of The Innovators Dilemma).
Of course, disruption is uncertainty in the face of fundamental and permanent change.
But one thing is for certain: IT is being forced to change—to rise from the muck, once and for all.
