Doug's Oracle Blog

I'm pleased to see a few posts debating the (non)-application of quarterly Critical Patch Updates. Here are a few examples ...

Is Poor Security Hygiene Rampant?
Do DBAs care about Oracle’s latest Critical Patch Update?
Survey finds that 66% of Oracle users never install critical patches

All of which seem to have been kicked off by Pete Finnigan's blog about the survey. Pete talks a lot of sense in that blog and I agree that :-

"the issue has become psycological, a lot of companies beleive its difficult, that it will fail and that everything in the organisation needs to be regerssion tested."

I've talked about this subject recently and all the evidence I've seen supports the survey, although I'd venture the percentage of sites claiming to apply CPUs is higher in the survey! However, as I said in that series, my argument was never about what was right or wrong, but the reality I've experienced. Personally, I feel that sites should apply security updates and invest whatever is required (predominantly in man-power and a commitment to maintenance outages) to do the job. That's why I'm very pleased that my current site is grasping the CPU bull by the horns and, having encouraged the business to upgrade to 9.2.0.8, is pretty close to having the Oct 2007 CPU fully implemented.

Next up will be the Jan 2008 CPU. It's a never-ending story, after all, but I'm sure we'll get better.

 

It's so long now since the OUG Scotland Conference and I ended up leaving early because I wasn't feeling too good so I'm not going to attempt a review of the day. I asked a few others what they thought afterwards and the general view was that the day was pretty good, if perhaps too sales-orientated. Most of the morning was taken up with Oracle business-focussed presentations and I'm not sure that ever goes down very well at a UKOUG conference. I heard a particularly good review of Jason Arneil's presentation which I was hoping to see until it was rescheduled later in the day. I particularly enjoyed the last presentation I watched Jason give and then, just the other day, I noticed that Jason has a blog too so that should be worth following.

I think my presentation went reasonably well, based on the feedback. For UKOUG members, the presentation slides are here but I have to say I was very reluctant to hand them over because they're useless in isolation - just a few visual gags.

In keeping with the style of a keynote and the chap I was standing in for, I wanted something reasonably controversial but high-level. The original idea for the presentation came from a series of blogs that had been floating around in my head for a while, discussing 'The Reality Gap' or :-

"The difference between what Consulting Firms, Oracle Marketing, Technical Architects, Bloggers and Security Researchers say we should be doing and what most of us really are doing"

One of my favourite stories has always been The Emperor's New Clothes. Nothing drives me nuts like everyone toe-ing an almost religious 'line' which has no basis in reality. If I can play the little boy and cut through the misinformation a little that makes me feel better. In fact, I think it's what every consultant, contractor or whatever we want to call ourselves should be doing - asking tricky questions to get to the truth.

Having stolen blog material for a presentation, it's only fair that it should appear here eventually. For my first example, let's talk about Software Maintenance and, more specifically, upgrading to the latest versions of Oracle and applying patches on a regular basis.

I conducted a show-of-hands survey near the start of the presentation which went something like this (and thanks again for everyone who joined in, I was a little nervous that a Scottish crowd might be a little reticent!)

• Who is running Oracle? (Nearly everyone raised their hand. Good start.)

• Who has all of their databases on 9.2.0.8 or 10.2.0.x? (Less than a third kept their hands raised)

• Who has applied the July 2007 CPU to all of their databases? (Not a single hand remained in the air. In October.)

The most important thing is that I guessed the result in advance and, if it was so obvious to me before I met any of the respondents, isn't there a Reality Chasm between what Oracle Support, bloggers and security researchers write and what most companies are doing? Every time a CPU is released, it causes at least a minor flutter in the blogging community, yet it seems to pass by most corporate customers I work with (and I've worked with a few). Why? Well, because they have to actually roll the patch out across hundreds of databases, not just talk about it!

• To start with, you need a regularly scheduled outage on every database. (Plus all the associated Change Management)

• Next you need enough DBAs to plan, test and implement the change.

• Plus the people to perform regression testing? (Well, maybe not and you're prepared to take the risk that no new problems have been introduced)

Actually, that's just a small sample of the issues involved. I see patch management as possibly the most important challenge that DBAs have to address and the more work that Oracle put into making this easier, the better.

You and I probably both know that security patches are essential to ensure the security of the business data, but do you think every business truly understands that? They've got better, thanks to SOX legislation, but what do you think they want their DBAs doing - implementing shiny new applications or applying patches to existing applications when they won't see any functional improvement from their perspective? That's our job, though, isn't it? To educate the business about the importance of patching? Mmmm, but who pays us? Who is always looking to cut costs to the bear minimum (ah, the wonders of Capitalism). Who really controls IT departments these days?

However, none of the pros and cons matter to my specific argument and whether you agree or disagree with me about the importance of security patches isn't the issue. (Oh, but please can we mention company firewalls when we're discussing exposure risks, otherwise people are being disingenuous at least.) In fact, I spend a lot of time working to help companies apply patches more regularly so this isn't an argument about whether we should apply CPUs or not.

The issue is that I predict confidently (is that a guess, Alex?) that the majority* of Oracle customers aren't applying CPUs rigorously, so can we please stop kidding ourselves? Because until the issue of applying the patches is addressed, all the discussion about them might keep a few people busy, but to no material effect!

* Please note I did not say "all"

Updated later - here's an interesting related blog that I wanted to point out but thought I'd lost.