Jul 5: Server Technology Conundrums


[The following is a guest blog post courtesy of Peter Robson, prompted by this event that we both attended. Peter's a good friend, I love his writing style and he's expressed similar opinions in the past to me, so I asked if he would write them down for a wider audience. See what you think. Thanks, Peter.]

Recently I attended an Oracle User Group meeting on server technology. The speakers were excellent, the topics highly relevant to a contemporary interest in server technology, and the audience receptive and appreciative. But there was something wrong – each speaker was showing the audience how to address the failings or weaknesses of the Oracle database system. Given the very high initial purchase price of the product, and the equally daunting annual maintenance costs, this seemed something of an unsatisfactory juxtaposition.

There are three topics I want to address, tuning, upgrades and security.

On tuning. Attendees at this meeting were presented with a veritable avalanche of slides, showing graphs, both 2D and 3D, tables, even 4-dimensional matrices, all illustrating just how many variables can be measured and adjusted in Oracle. Specialists have carried out much serious work and research in this area. Customers in particular get enormous benefit from taking the advice of these highly skilled specialists. Indeed, the proliferation of technical gurus skilled in many of the arcane characteristics of Oracle is a further indirect endorsement of the product itself.

There are winners and losers here. Oracle wins – they need to spend less time on perfecting their product. The consultants win; because of the position Oracle adopts, they are guaranteed a job for as long as Oracle pursues this strategy. The cost of both the product (which one might perhaps begin to suggest is unfinished) and the cost of the down-stream consultancy advice is all born by the customer. In fact, it’s not just the price of purchase, but ongoing maintenance that irks many users. At something like 15 to 20% of the purchase price every year, the customer must pay this price to ensure they receive corrections (euphemistically entitled patches), as well as retain the right to receive the next major release. These privileges are generally described as ‘support’. One might just begin to wonder which party is actually being supported in this situation. It’s a beautiful catch-22 revenue stream, and once again the loser pays all.

Oracle provide some tools (AWR) to assist with tuning the database – but as an extra cost option. They tend to be inserted during the main installation, even if they are not explicitly licensed. It is quite easy to accidentally access one of these components (perhaps by looking at a dictionary view), resulting in the internal logging which Oracle have installed on the customer system recording the event. This will prompt an additional cost being charged to the customer..

But is it acceptable that a vendor should install a product on one’s own hardware, which generates logs using disk space and cpu resources, which the owner of those resources are not permitted to access without paying extra?

One used to have to use a PL/SQL package licensed as part of the Diagnostics pack in order to disable the Diagnostics pack, to ensure that accidental access to one of the monitoring dictionary views was prevented. Only after uproar in the user community was this anomaly resolved. This was a bizarre revenue stream – a costed item to make sure one does not use a cost item. In any event, one has to ask why should one have to pay extra to monitor and adjust the product already paid for? It’s tantamount to buying a car, and being told the dashboard instruments are extra. Worse than that, in fact – it’s like paying extra for the on-board computer that monitors and controls the engine management system.

A talk on upgrades described many of the difficulties in moving from one version to another, including most astonishingly the total corruption of data during its migration from one version of Oracle. The only way to avoid this situation is, apparently, a) don’t upgrade, or b) test, test and test again before going live with a new version. But why should the customer have to test a product so exhaustively for which they pay so much? Does the manufacturer have a lack of confidence in the suitability of a new release? One could be forgiven for believing that Oracle is in fact using the customer base as a means of final testing of new versions. It must explain why many customers are reluctant to buy into the first version of a new release. Of course many installations will upgrade without any problems, but the very fact that problems have to be anticipated is unacceptable given the investment involved in this product.

But the whole issue begs the question of why upgrades appear so thick and fast. I would suggest that this happens for two reasons – income stream, and the drive to develop technology for its own sake. Note the vested interest in the guru community of seeing their clients upgrade. Note also the way in which the underlying architecture of the database seems to change subtly with each new release, requiring a new edition to the guru handbook ‘How to Tune Oracle Version xx’. I do not believe that this is an environment in which the end user is best served. Unless there are particularly strong business reasons to do so, no organisation should ever upgrade just for the sake of it. In fact, I would say that the constant tinkering of the kernel by Oracle with each new release, probably by different, even competing coders within Oracle Corp, militates against the basic stability of the product. If so much testing is required following each new version, then too much must have been changed. Why?

Finally, the issue of security was the subject of another interesting presentation. It recently emerged into the public domain (following the MacKinnon hacking trial and other sources) that there are copies of Oracle embedded in the most secure sites in the US military and security sectors which still have not properly implemented password security. Anyone with a modicum of knowledge of Oracle default passwords can walk right in to many of these databases. As many competent hackers have done from all round the world (and countries from the Far East are notable here). One has to ask what sort of company is it that even allows a foolish customer to install their database product without FORCING them to impose the most fundamental of security constraints – primary password access. This rather worrying gap in basic security has to pose the question of how fit for purpose the Oracle dbms is for installation into a secure environment run by not quite competent users. Downstream security may indeed be excellent, but if the front door is left wide open, that downstream security is somewhat compromised.

So, great product, but it could be so much greater if more effort were directed to improvements truly for the benefit of customers, and not simply to maintain an income stream based on a technological merry-go-round.

Peter Robson
Posted by Doug Burns Comments: (25) Trackbacks: (0)

Trackbacks
Trackback specific URI for this entry

No Trackbacks

Comments
Display comments as (Linear | Threaded)

#1 - William Robertson said:
2008-07-06 17:17 - (Reply)

You seem to start out criticising Oracle for not bothering to finish their product, and then criticise them again for releasing too many fixes and upgrades. I agree there have been a lot of optimizer bugs in the fifteen or so years they've been working on it (ask me about column endpoints being ignored in 10.1 and suddenly critical in 10.2, or the fact that it only started using subpartition statistics in 10.2.0.4, several years after it started gathering them) but I think any product of this complexity is inevitably going to be a work in progress, and that is why they have upgrades.

I'm not sure what you're saying Oracle should do about the tuning issue, if there is one. It seems to me they have been trying to make the database more self-managing over the years - look at all those Automatic Whatever Management features in 10g, or 11g re-evaluating execution plans if a query runs slower than it did previously. Or is that tinkering for its own sake?

#2 - Connor 2008-07-08 14:14 - (Reply)

In terms of "Unless there are particularly strong business reasons to do so, no organisation should ever upgrade just for the sake of it", I used to be of the same opinion.

However, for anyone paying for Oracle support, then I've found that it actually pays to upgrade (and to upgrade often). Because upon encountering any significant problem with a non-current version, support typically ask you to upgrade or to at least replicate the issue on the current patchset...That's grief you don't need in the middle of crisis. Whilst not optimal, its better to upgrade in a controlled fashion.

#3 - Noons said:
2008-07-10 07:20 - (Reply)

Agree entirely with what Peter is saying.

I've been saying the same and much more, for years now: the TCO of Oracle has reached horrendous levels, while everyone seems to be concerned only with the vanilla list price.

And it's not just the "upgrade spiral", it's the very real cost of re-testing everything!

In case no one has noticed, hiring back cohorts of testers to re-test every single application every single time a patch comes out is just not sustainable.

And test we must: Oracle support themselves say so!

Any wonder a lot of sites just plain don't upgrade?

:-(

#3.1 - William Robertson said:
2008-07-10 08:27 - (Reply)

At least 11g makes it easier to test, or is that another of the tiresome new features you would prefer they didn't keep churning out to keep us on the upgrade treadmill?

#3.1.1 - chris_c 2008-07-10 16:54 - (Reply)

and at only 5840 of your British pounds a CPU what a bargain RAT is, seriously though it looks like a usefull feature but trying to get that through the assorted approvals from finance will be interesting..

#3.1.1.1 - Niall Litchfield said:
2008-07-11 20:53 - (Reply)

Actually of all the added extras RAT seems to me to be one of the easier to justify, since it competes directly with established and expensive testing tools - loadrunner etc - and many if not all organisations will have been bitten at least once by upgrades that had unpredicted performance side effects.

From the dbas point of view as well you get some nice side effects in the form of tools that are amenable to use in tuning and so on as well as testing.

#3.1.2 - Noons said:
2008-07-11 05:25 - (Reply)

Like Chris said: waaaaaaay too expensive and it should have been part of the base product ages ago instead of diddling around with the "no need for a dba" nonsense!

But then again, that would assume Oracle marketing would *listen* to what their clients are saying. Instead of charging on with whatever madness is the fancy of the jour and then trying to force it down the throats of all and sundry, needed or not.

Preferably with large doses of innuendo that anyone not accepting the greatest gift to Oracle-kind must of necessity be a "bad dba". And other such niceties of the last 10 years or so....


But, I digress.


The stuff in 11g is a very small and late step in the right direction.

But I'm afraid just capturing a "workload" is *not* complete and accurate testing, no matter what the latest deranged marketing definition of that might be.

RAT also means additional licencing and other costs - accurate testing must be captured first and it ain't coming out of thin air.

Which in a climate of cost contention is a great idea: I'm sure it's gonna take off like a bull on heat.

Just like all of Oracle's "great innovations" of the last decade. Fusion springs to mind...

#4 - Doug Burns said:
2008-07-13 23:35 - (Reply)

Well, the first thing to say is that, shortly after I posted the blog on Peter's behalf, I discovered he wouldn't have much opportunity to be online for a couple of weeks. So any lack of response from Peter should be put down to that, rather than disinterest in the comments.

I've been busy too. One more week to go at my current site and shed-loads of work to get through so I've been keeping track of the comments but haven't had much opportunity to respond. So, as a consolidated response ....

William said ...

look at all those Automatic Whatever Management features in 10g, or 11g re-evaluating execution plans if a query runs slower than it did previously. Or is that tinkering for its own sake?

I think Peter's point on those particular features was more about having to pay extra for them, particularly as ASH etc are enabled by default.

Although it might not have come across in his article, I know something he's been thinking about for a long time is 'why is Oracle so bl**dy complicated that we need gurus like Jonathan Lewis and the like'. As I pointed out to him, the percentage of Oracle professionals who have read Jonathan's book is tiny. I know, I work with a lot of them. Does that mean that every Oracle instance is on it's last legs? Not at all, but if you listen to people who are passionate about their subject and attend user group meetings, you're likely to hear more bad news than good, simply because it's more interesting.

Connor said ...
However, for anyone paying for Oracle support, then I've found that it actually pays to upgrade (and to upgrade often) ...

I agree with that strategy and have had a lot to do with encouraging it at my current site. Although problems are encountered when applying patch-sets, the correct solution is to get them fixed and move on. There is some pain associated with it and it would help if bug-fixes didn't introduce new bugs and the like, but it's better to have a constant, minor pain than end up with a software estate that's hopelessly out of date. Besides, which, you're stuffed for support if you don't, even if it does seem pretty expensive for what amounts to bug fixes.

However, ...

In terms of "Unless there are particularly strong business reasons to do so, no organisation should ever upgrade just for the sake of it", I used to be of the same opinion.

I'm sure we're all familiar with those systems that are running on 8.1.7.4 and just work! Why would we upgrade them if we don't need any new features. From a business perspective, what is the business gaining?

Noons said ...

And it's not just the "upgrade spiral", it's the very real cost of re-testing everything!

I agree. The costs associated with simply maintaining software and re-testing are horrific. So, I think the truth is that most people just suck it and see and hope for the best. If the upgrade fails, they back it out. It's ok talking about full regression tests but for the tiny tactical databases that make up a large part of a company's estate, it's far too expensive. Better to take the extra cost when you do run into problems (which I think is less often than people make out sometimes)

William said ...

At least 11g makes it easier to test

Yes, for a substantial additional cost, as others pointed out. As well as the extra manpower to learn to use it, implement it, etc. Don't get me wrong, I think RAT's a very useful feature indeed, but it's all just more license costs and more support percentages on top of that. There are times when the Oracle of today strike me as the IBM when I first started and we know what happened to them.

Maybe you think I'm being too cynical? Well, this paragraph jumped out at me from this recent article.

The fact is, Oracle has some potent defenses against a weak economy. Consider its high-margin, cash-generating maintenance business, which entails keeping products current for customers and providing "patches" to protect against potential problems.

Maintenance revenues -- reliable, annuity-like streams -- have grown some 24% a year during the past six years and show no sign of letting up. Oracle enjoys a maintenance-renewal rate of 90%, above the industry average in the mid-80% range. This year, maintenance revenue should come to about $10 billion, or just under half of Oracle's total revenue."

I know, Oracle is a business and welcome to the real world but the closer I get to the business decisions and the costs, the harder I find it to believe that they can maintain this approach forever. People are looking to cut costs. Oh look, what's that over there? A free database with small support fees. Mmmmm

Ultimately, though, these were largely Peter's thoughts and we had a very long debate about these issues before the blog post appeared. I don't agree with him about everything and might blog a longer response later, but I think he has some interesting points.

#4.1 - Joel Garry said:
2008-07-14 23:19 - (Reply)

I'm sure we're all familiar with those systems that are running on 8.1.7.4 and just work! Why would we upgrade them if we don't need any new features. From a business perspective, what is the business gaining?

Because eventually, they no longer "just work." Have you never seen a post that says I've got 8i and... or where can I download 8i?

#5 - Peter Robson 2008-07-16 21:07 - (Reply)

Well, back from holiday. Thanks Doug for explaining why no feedback from self. And thanks to you all for contributing to the debate (and encouraging me!).

I shall add more subsequently, but this is difficult. There are points on which I feel so strongly that I have to redraft repeatedly to avoid the text becoming ridiculously hyperbolic. Put it this way - the time, IMHO, is long overdue for a very cold, critical dispassionate examaination of what exatly is going on here...

more later,

peter

#6 - peter robson 2008-07-18 09:58 - (Reply)

I'm prompted to add some comments following an article in yesterday's Guardian Tech (17 July 2008) by Bruce Schneir. I couldn't believe what I was reading - had he already seen the stuff we are discussing? A couple of quotes will suffice:

"It's the system that's broken. There's no other industry where shoddy products are sold to a public that expects regular problems, and where consumers are the ones who have to learn how to fix them."

Read the article here:
http://www.guardian.co.uk/technology/2008/jul/17/internet.security

He argues for the acceptance of software liability by the manufacturer. And states:

"A lack of software liability is effectively a vast government subsidy of the computer industry. It allows them to produce more products faster, with less concern about safety, security, and quality."

Does that not describe the very issues being discussed in this blog entry?

Anyone who defends the current system has been seduced into accepting software which is sold without liability. IMHO this is totally unacceptable, and yet the market is complicit. Indeed, the market has a vested interest - lets really hit the gong - and database professionals who earn their living by fixing and tuning under-performing database systems, whether they realise it or not, have a vested interest in software products being sold which are inadequate in some way. Now don't misunderstand me - these people perform a valuable and essential service, but I am criticising the very fact that they have to provide this service!

To be objective, it is probably unfair to single Oracle out, as the problem spans across the entire software industry. But I am singling out for criticism anyone who regards this situation as acceptable. No it is not. Read that article by Schneier.

#7 - William Robertson said:
2008-07-20 09:38 - (Reply)

So to summarise, if you were put in charge of Oracle tomorrow you would:

1. Improve database performance
2. Remove all security vulnerabilities and take responsibility for any yet to be discovered
3. Release fewer patches
4. Reduce the prices.

I suspect that would raise some eyebrows at the board meeting.

I agree that the prices of enterprise edition and add-ons like partitioning and tuning/diagnostics packs are pretty horrifying, and as someone who rather depends on Oracle for a living I hope as much as anyone that it isn't about to be left high and dry by some new version of SQLBase that does it all for $20. However I don't presume to know what factors go into Oracle's pricing decisions. Do you? Without knowing anything about it I find it hard to have a strong opinion on the subject. Yes, when the revolution comes we'll storm the greedy corporate HQs and reduce licensing costs for Enterprise Edition.

Regarding the tuning guru industry, I wasn't at the conference but I think most of the performance bottlenecks addressed by the likes of Jonathan Lewis are shortcomings of application design and coding rather than faults in the RDBMS product itself. I wouldn't describe myself as a tuning guru by any means, but I nearly always find removing the hints and analyzing the tables works miracles. Even where there are configuration issues I'm not sure that we should expect Oracle to come automagically preconfigured for every possible platform and application type. I'd like to hear some specific examples of clear weaknesses of the core RDBMS product and pointless kernel tinkering before joining in the Oracle bashing.

#8 - Chris Adkin 2008-07-20 11:12 - (Reply)

Guru/Expert Community

With all IT technologies there is always an expert community, it happens that with the Oracle RDBMS this is
particularly large due to breadth of the technology and the amount of time it has been around for. On
a particular Ask Tom thread, Tom Kyte referred to how despite operating system platforms changing
in popularity every ten years (mainframe -> Unix -> Windows -> Linux), Oracle has always remained
the most popular choice of database engine. Also, it is a fact of IT as an industry that projects and
exercises where people really earn their corn and obtain valuable experience is when things go wrong.
As Doug alluded to, a user group get together wouldnt be that interesting if a succession of
presenters got on a platform and said, I've been running on software that is a major
release or two old, it all basically works, any questions ?. The 'expert' community is not however
without its failings, particularly with certain organisations that dominate the top results returned
by the major search engines and make bold claims about certain facets of the Oracle RBMS without
much test material or worked examples being supplied to back these up.

Upgrades

If Oracle did not produce a regular stream of upgrades many people would view the technology as being dead,
and who wants to base a significant chunk of their IT infrastructure on stagnant technology ?. Most DBAs
can probably name aleast one new feature that appears in a new release that makes their life easier.
Regarding patches, again if Oracle did not produce CPUs on a regular frequency there would be
some people up in arms about Oracle not taking critical flaws in the RDBMS seriously.
My bone of contention is not so much the technology being driven for the sake of it, but Oracle's tendancy to
bring out new features in the product and not address existing features which are half baked when first released.
DBMS_STATS auto sampling and habit of creating histograms for columns that do not require histograms and omtting
to create histograms on columns that most definitely require them when size auto is used, to name but one
example of this. There are some features which creep into the RDBMS which do most definitely smack of driving the
technology for the sake of it, the JVM in the kernal in 8i springs to mind, as do the rumours about fusion
technology findings it's way there also in 11g release 2.

On the subject of testing

I think that Oracle missed a trick in not making real application testing a free option. However in
the defence of Oracle, how many serious IT organisations do not re-test their applications if a
significant components in the underlying technology stack is upgraded, very few I would wager. To borrow
an American phrase, Oracle do nickle and dime their customer base at every turn and (again as Doug alluded to)
I can see a day when cheaper alternatives are considered more seriously at the low end, such as MySql.
There are also much cheaper database alternatives which some nedors site in their studies as saving
significant amounts of money compared to going down the route of clustered databases, refer to
the Terracotta web site for examples of this.

Security

This is a definite area for improvement and I can't see that it would be a major undetaking for Oracle to include
a secure installation option, that puts password aging, minimum password strenght, listener password security etc
without the DBAs having to explicitly set this up.

#8.1 - Doug Burns said:
2008-08-06 01:25 - (Reply)

Chris,

Whilst reading through tons of comments here, yours struck me because I found myself nodding agreement constantly as I read it.

Cheers,

Doug

#9 - peter robson 2008-07-20 23:08 - (Reply)

Thanks for your inputs - some good points.

William, you invite me to suggest what I would do if in charge. First off, stop patching a flawed system.

Next, redesign the Oracle engine so it corectly implements the relational model. Which means, for example, insisting on zero duplicate rows, prohibiting nulls, etc.

Get the physical implementation of the model right, and there would be far less requirement for upgrades and new releases. The product would become inherently more stable.

Chris, you suggest that if the frequency of updates reduces, the technology would be described as 'stagnat', and accordingly become less attractive to the purchasing community. Well possibly, but you have in fact touched on another driver in the IT world - fashion. I contend that the IT consumer is less than discriminating, and companies like Oracle understand this very well, and market accordingly! Can you blame them? So do many other companies. Which is why I think the case made by Schneir for software liability is so powerful. It truly represents a paradigm shift (a much abused term) in the marketing of software, and would radically redefine the market to the benefit of the customer.

The guru expert community exists to the extent that it does because the customer base is trying to build their commercial models without truly understanding what they are doing. I have found that a large number of the performance problems encountered by customers is quite simply because they don't get the basic model design correct in the first place. So hunting for 'better' performance is chasing a horse that has already left the stable (to mix metaphors). For example, it is still possible to find systems built using Oracle which don't have primary keys, thereby allowing duplicates rows. Fully normalised designs are not the norm (sic!). And still people believe that by adding extra memory, faster disks etc, they will improve performance. I'm sorry, but trying to improve a flawed design is a waste of money. So, the money is spent on unnecessary hardware upgrades, guru consultancy, and oh yes, Oracle maintenance.

Which brings us to the Oracle commercial model. It is designed, FIRST AND FOEREMOST, to make money. Oracle will charge you, the customer, as much as it can possibly charge and still keep customers. Make no mistake about it. In their early days, they sold an inferior product when compared to the likes of Ingres, Sybase and others, purely on their far superior marketing skills. Is it technically superior now? I don't know, but it certainly is not as good as it could be.

If Oracle got the basic model conformant with the relational model which they claim is their underlying model, a huge array of problems would simply vanish. And with them, many guru consultants, incidentally. Because customer sites would be forced to design models which were of a much higher standard. Constraints would be properly implemented. Dependencies correct, and so on. Tuning would become so much easier.

Doug rather gave the game away (confirming my point) that user groups are only interesting when people share problems. Not really - the best user group meetings are when exchanges prompt imaginative uses of a stable technology! That is not problem solving, that is going forward. Instead, our user group meetings are full of problem solving sessions. That is NOT progress. What a waste. Back full circle, I think.

#10 - William Robertson said:
2008-07-21 00:17 - (Reply)

So another couple of actions for tomorrow's board meeting:

5. Announce that Oracle version 11.2 will not support nulls.
6. Announce that Oracle version 11.2 will not support duplicates.

Good luck with those. Oracle already gets branded 'weird' for only having one kind of non-value (apparently most other products have two, and that's better). If the above were implemented Oracle would go out of business overnight, for the good reason that very few applications could be built under those constraints.

I must say I'm less sure where this is going with each turn this conversation takes.

#11 - peter robson 2008-07-21 15:32 - (Reply)

As I explained, Oracle is in the business of making money, not state of the art DBMSs. But they could if they chose, (perhaps in parallel with their existing offering) or if the customer base requested sufficiently energetically. And that means energetic user group forums, amongst other things. Would they go out of business? Knowing Oracle, perhaps not.

Give the customers a better DBMS, and they will save money, and stay in business. Too bad if Oracle goes under, but I doubt it.

Anyway, what's with two kinds of null? A null is a null - it means nothing. How can you have two kinds of nothing?! The support for nulls allows sloppy database design, which results in a continual quest for additional tuning. QED.

And who wants duplicates in their data tables anyway? Why?

#11.1 - Thomas Presslie 2008-07-28 07:41 - (Reply)

Fantastic thread! Can I introduce another perspective on the debate? Oracle is obviously primarily concerned with its shareholders and this will continue until the demise of Oracle. When will folks start looking more seriously at the open source alternatives? I find myself much more interested in the developments of PostgreSQL than the nice 'extra cost' new features in Oracle 11g.

#11.1.1 - William Robertson said:
2008-07-28 08:41 - (Reply)

Postgres does seem to be popular with other Oracle bashers - but do we have a favourite open source alternative RDBMS that bans nulls and enforces uniqueness in all tables and queries whether you ask for it or not? While we are at it, it should probably use Tutorial D in place of SQL. How about we dismiss Postgres along with Oracle and focus instead on Rel? That'll get rid of those pesky tuning gurus once and for all.

#12 - Peter Robson 2008-07-29 19:13 - (Reply)

William, you do go banging on about nulls and uniqueness. Do you really think these are desirable features?

In our house, we established corporate standards which a) banned duplicates, and b) banned nulls. But so much easier if the RDBMS does it for you. I do recall the Mimer RDBMS enforcing unique primary keys, and therefore banning duplicates.

As for TutD, well, we could, but that would require a whole new learning experience, which is not really necessary (whatever Date and Darwen may say). With care, SQL can be used in a relational fashion (indeed I am reviewing a text book by Date on this very subject right now). If the RDBMS vendors paid a little more attention to the basics of the relational model, and users understood a little more about these concepts, SQL could continue in use without invoking the ire of the purists.

#13 - William Robertson said:
2008-07-30 09:20 - (Reply)

It was you who brought them up!

To recap, your impression from the OUG conference was that Oracle is overpriced, overcomplicated, under-optimised and insecure, and that the overwhelming difficulty of configuring it to address its performance shortcomings feeds an industry of performance gurus. In the discussion that followed, you claimed that the core of Oracle's performance problems lay in its poor implementation of the relational model, for example allowing nulls and duplicates, which cause nothing but trouble and lead to inefficiency, complicated workarounds and guru proliferation.

Since this criticism could be applied to all commercial RDBMS products (and Oracle is better than most in supporting only one kind of empty, not two), I'm not really sure what it had to do with the your original point, but I was interested in seeing where it led since I do not believe it is possible to do without nulls (awkward as they are) in SQL, and I think the kind of implicit DISTINCT and mandatory PKs the relational purists argue for, although a better idea than banning nulls, would in practice cause more problems than it solved. In any case these measures would be easily circumvented by default values and surrogate keys, leading to yet more complication, inefficiency, patch releases, kernel tinkering and vampire-like performance gurus.

It seems to me that the post was an empty exercise in Oracle-bashing with no practical suggestions for improvements except that Oracle should reduce its prices.

#14 - peter robson 2008-08-02 18:45 - (Reply)

Hmm...

"Oracle is overpriced, overcomplicated, under-optimised and insecure, and that the overwhelming difficulty of configuring it to address its performance shortcomings feeds an industry of performance gurus"

Yes

"I do not believe it is possible to do without nulls (awkward as they are) in SQL"

Yes it is. It is absolutely possible to design to avoid nulls. Revisit your database design manuals.

"implicit DISTINCT and mandatory PKs the relational purists argue for, although a better idea than banning nulls, would in practice cause more problems than it solved"

Simply not true. Indeed the converse. See previous answer.

In light of having to make these answers, no further comment required!

#15 - Doug Burns said:
2008-08-06 01:24 - (Reply)

Crikey - look what happens when I turn my back for 5 minutes. Well, ok, two weeks maybe ;-)

I agree with William when he said this ....

I must say I'm less sure where this is going with each turn this conversation takes.

... to the extent that I almost wrote a seperate blog post containing my own thoughts, trying to focus on the initial issues that I found most interesting and pertinent and whether I agreed with them or not. But then it would appear out of context and might also prompt another minor comment storm. I've got enough to read through, as it is. So, picking up from comment number 7, here are a few thoughts that sprung to mind ...

What I will say from the outset is that I have no interest in getting into discussions about duplicate rows or NULLs. First, I don't see that they were what Peter's original piece was about even if it was him that introduced them later (in comment number 9). I don't think that was helpful. Second, knowing both people involved in the argument (albeit I only have a small electronic sample of William), I suspect they wouldn't actually disagree that much on most of these issues, despite the fact Peter's a relational purist. Having said that, I've never found Peter one for 'Oracle-bashing', either, although it looks that way on this thread. Maybe it's just frustration building up over time? I certainly don't see myself as an Oracle-basher and I have no plans to switch technologies. In fact, the most common criticism that's levelled at me day to day is that I'm too much of an Oracle evangelist! I have many witnesses to that.

Anyway, what I think I would focus on personally are the two issues :-

1) Security, Patching, Upgrades and Support.

I think there is a big difference between innovation and fixing what already exists. While there might appear to be a vested interest for Oracle Corp., Oracle contractors and consultants or whatever we want to call them, in introducing new features that will take time to master and might even need some additional effort to make them usable, I can live with that. I'd dread to think that we stop adding new features just because what's there already is good enough. I'm even happy if Oracle wants to charge people more for premium products, as I've stated here in the past. I wish ASH/AWR/ADDM were free but, if Oracle chooses to charge, then so be it.

However, adding new features is a completely different exercise to fixing what are, ultimately, bugs. It strikes me as amazing that Oracle can charge their own customers something like 15-20% of every initial pound spent, every year, simply to receive fixes to a product that they didn't get right in the first place. On top of that, I need to force businesses through a constant upgrade cycle on top of the support cost to be provided with fixes in the first place! Now, I'm not for a minute suggesting that Oracle are the only software company that do this, or the worst, but it seems like a bad joke to me that I'm the one who has to sit up at nights, upgrading working systems to the latest releases (and introducing new bugs in the process) because Oracle won't supply bug fixes for old releases. I *know* it's an expensive business supporting every release under the sun but, let's face it, I didn't introduce the bugs and yet the onus is on me (and my peers) and Oracle's customers to bail Oracle out of it's own mistakes. Bear in mind that we're talking about a multi-billion dollar organisation in a constant acquisition frenzy, not a 20-person software house. Of course, the markets wouldn't stand for them taking the huge hit of maintaining the versions their customers want to be maintained, but that doesn't mean they shouldn't. It was what Peter alluded to with the car industry comparisons. I'm sure car companies would rather perform a lot less testing and get to market quicker but there are, erm, regulations to keep them straight.

The patching and update issues were a direct response to Piet de Visser's 10g Upgrade presentation where he more or less stated that people might be better just sticking with old systems if they were working. The problem with that is that security holes are publicised as they are fixed via Critical Patch Updates which aren't available for those older versions. So people are stuck with a piece of software that has holes and the only way of closing them is to update to later releases which is a non-trivial exercise. When I buy a product I shouldn't *have* to pay an additional annual fee to ensure that any mistakes are fixed.

I'm no Oracle-basher but I do know that if you're ever going to have a 'Grid' or 'Software as a Service' then the quality of software and the way it's written needs to be addressed. I'm just thankful that Boeing or Airbus fly-by-wire software isn't developed in the same way as the majority of business software ;-) (Oh, and please, if it is, let's keep quiet about it. I have enough trouble getting my better half on a plane as it is.)

2) The apparent complexity of Oracle tuning.

Well, Peter and I debated this long and hard before I posted his thoughts. I'm not convinced that Oracle is impossibly hard to tune or that there's some great conspiracy of paid consultants saying so (see my comments above). In fact, there's more free information available than ever and perhaps that makes it *seem* hard, because there's so much knowledge floating around. One of the points I made to Peter was that, if he will hang around User Group conferences, he'll tend to hear the interesting stuff rather than the boring success stories. When I was teaching at Learning Tree, I learned a very important lesson about the difference between 'Oracle people' and 'SQL Server people'. I always liken Oracle/SQL Server to Unix/Windows. Oracle is extremely flexible and, as a result, there tends to be several solutions to any given problem, which can make it seem complex. I like that because I can look for the best solution. Oracle attracts those who want to understand how something works and be able to configure it well. SQL Server attracts those who want something that works with the minimum amount of fuss and when it breaks ... well ... there's a lot of shoulder shrugging it seems to me, but maybe I'm just biased. Peter might think that the range of options, or complexity, is a problem, but I don't. Oracle clearly do, however, or at least some of their customers do, because they've been focussing on manageability for years now.

The idea that Peter proposes that I have some vested interest in Oracle being broken is quite silly. As I've said to Peter, I might be more interested in working with Oracle because it's a very flexible and developing product, rather than working with something like Outlook or Word that requires no maintenance to speak of, but that doesn't mean that I want a broken product. If the product works well, there'll still be plenty of other interesting things to do (although I might need to move back into development).

Let me try an analogy based on the cars that Peter loves. Lots of people buy a Ford Focus and are blissfully happy. Peter chooses to drive a Morgan. Now, which is the better motor? Which takes more maintenance? Why even bother with a Morgan? (as I type, I have a feeling I may have the wrong model, but it probably doesn't matter for the argument).

When a business needs to be able to do something difficult, I'm confident that I'll be able to make Oracle do what they're looking for. It might turn out to be expensive, but at least the possibilities are there.

As I stated to Peter in emails before I posted his thoughts, the vast majority of Oracle databases I work on chug along quite happily with no problems at all. They might not be as efficient as they *could* be but, as William pointed out, most of the inefficencies tend to arise from poor database and application design and, I would add, those inefficencies are often hidden by the ludicrously powerful hardware they run on. However, for some critical systems and in some unusual cases, some performance tuning might be required and I, for one, am glad that those systems exist because they give me something interesting to do. I bet most car mechanics would rather work on a Ferrari than a Focus, even if the Focus requires much less attention.

Oh, to get back to the subject of the conference Peter and I attended - Piet de Visser suggested we never upgrade because it was such a pain; Pete Finnegan went on about security problems; Jonathan Lewis pointed out some of the interesting flaws he always does and I whined a bit about licensing? Why? Because part of Peter's criticism about my first version of the presentation that he'd watched was that perhaps I was too evangelical! There's no pleasing some people ;-)

Oracle-bashing? I don't think so, but Peter and I are probably old lefties and to watch a company make their customers pay for the companys own mistakes is likely to wind people up.

Now, I have a to do list the length of my arm so I'd better shut up ;-)

#16 - Martin Widlake 2008-08-06 18:46 - (Reply)

I think Peter’s initial posting is concise and very valid. Oracle does seem to constantly fail to get the balance right between extending it’s products and improving them. This is not Oracle Bashing, it is customer feedback. It is deeply vexing to have paid and continue to pay a lot of money only to so often hit problems that you would think would never have got through testing. When it works, Oracle is great, and I think most people find something new in every release that makes their working life easier (AWR is a real boon for tuning, leaving aside the thorny issue of licensing).

However, one area that is not covered by the original post or the plethora of good comments is the information that Oracle supplies, or rather the lack of it. I was one of those presenting at the event mentioned (though as Doug could not be bothered getting there in time for mine, he missed it – I shall hate him forever for that ) and I gave the presentation because I was so annoyed at the lack of information available about one aspect of performance, namely the automatic statistics gathering in V10.

I feel very strongly that Oracle could provide a lot more information than it does. Taking the automated stats gathering as an example, how can you trust something for which there is so little detail information on? How can you predict how it is going to mesh with your existing applications or plan new ones with it in mind when you have the situation where this automatic job makes it’s own decisions about what to analyze, when to analyze, at what level to analyze, when the cursors dependent on the analyzed objects are invalidated…?

It can be very hard/impossible to get this information from Oracle about how the software calculates or decides on what actions it will take. Working it out yourself is very time consuming, a luxury few of us have. Doug does say there is more information available than before but I feel most of this comes from other sources than Oracle Corp and I think the ratio of information to system complexity is actually going down.

The end result is that you may deride the product wrongly (something I have been guilty of); you may fail to appreciate the impact it will have; more critically, you may struggle to cope with issues it introduces; you certainly will not get the best out of it.

So you will not be a happy customer.

I can’t see what Oracle gains by not making more technical information available but I think I see what they would gain by being more open.

An incident that highlighted this for me was when I gave the same presentation a week or two later and included a small fact I had only discovered that day(*). One of the audience asked in a plaintive, vexed tone “HOW do you know that? I’ve been trying to find that out for months!”. He did not strike me as someone very happy with Oracle over that.

(*) How did I discover that information? Another long-suffering old-hand was good enough to tell me of course.

#17 - peter robson 2008-08-06 22:19 - (Reply)

Well, I'll keeop this VERY brief as Doug has written more than my original blog, and pretty well said the same things too!

So just a quick comment or two.

Agree entirely ref customers paying to have bigs fixed.

Ref cars - funny you should mention that, as I have several (and none of them is a Morgan!). Each for a particular role. I do not believe ANY car will ever meet all my motoring requirements, however much I re-engineer. Now lets see how far that analogy will go with Oracle... ;-)

Vested interest? Apologies, wrong choice of word. What I meant to say is that the complexity enables people like Doug et al to enjoy a career always chasing after Oracle. With every new release, here come the bugs. Oh good - some poor sodding company is going to have to pay someone to fix them...

Oracle / Sql Server and Linux / Windows? Neat. Probably true. But reading that paragraph, I got the impression that working with Oracle is enjoyable - not because its a well-rounded, complete product, meeting the customer needs with ease and efficiency (fit for purpose?), but because it provides plenty of opportunity to research and explore different solutions. I just have the feeling that it indulges the hacker instincts a little too much...

Having said that, I have to admit to enjoying working with sql, probably as much because it provides numerous different ways to do the same thing. But it does leave much to be desired.

Intrigued that Doug mentioned fly-by-wire. I was going to pose the rhetorical question 'would you fly in an aircraft based on an Oracle implementation' and decided not to. Thank you, Doug - you did it for me - complete with the answer!

QED - nothing more to say, is there? ;-)


Add Comment

Standard emoticons like :-) and ;-) are converted to images.
E-Mail addresses will not be displayed and will only be used for E-Mail notifications.
BBCode format allowed