Doug's Oracle Blog

Moving Home

Sun, 7 Aug 2005 21:18:00 +0000

Well, I've faced my final problem with Tripod's blogging tool. Jeff Hunter ran into problems trying to add a comment, so I've decided to move over to http://oracledoug.blogspot.com/ Cheers, Doug

Customer-driven IT

Wed, 3 Aug 2005 18:03:34 +0000

One of the most satisfying aspects of my job is helping people. My customers aren't necessarily IT literate so I get the satisfaction of helping them achieve something they otherwise wouldn't. For a long time I felt that IT consultants, developers, DBAs and the rest didn't show enough respect for or understanding of their customers needs. (I've no time for an arrogant DBA who thinks that users are just trouble and every activity is a precious favour granted. It's *great* helping people, fixing broken systems and suggesting new and improved approaches. We're lucky to have such interesting, well paid and challenging careers.) Over time the situation has improved, but I wonder if we haven't swung too far the other way. A common mantra these days seems to be 'IT only exists to serve the business and its activities need to be driven by the business'. This is fair enough, because IT doesn’t generate any revenue and so needs to assist the other areas of the business that do. What I *don't* think it means, though, is that I should do everything the business tells me to, without question. What's the point of businesses employing skilled resources and then just using them to blindly carry out activities in a manner decided by someone who doesn't have the first idea of the issues involved? One of the main casualties of this approach seems to be preventive maintenance. I'm losing count of the number of times I've highlighted that something is an accident waiting to happen and been told that avoiding that accident isn't a priority. Over time more priorities come and go, and the risk lingers. Eventually the accident happens. Everyone conducts a post-mortem and says 'Okay, we must fix these things and not let it happen again'. At a bad site it's soon forgotten because a new priority comes up. At a good site you get to fix things in the area that caused the problem. It won't end there, though, because preventative maintenance is constant. The business doesn’t particularly care about being on the latest patch release because there's no (easily measured) business benefit. Of course, when the doo-dah hits the fan and Oracle won't support you until you apply a patch to production (which will need to be tested in dev and test first), business managers will be screaming about what's taking so long to fix things! Another area of pain over recent years has been flailing around wildly trying any idea that crops up in a manager’s mind to fix problems, rather than slowing down, looking at the problem properly and coming up with a proven solution. If you're lucky, one of the quick fixes might work, but I still think that the chaos and wasted energy gets everyone down and it’s easy to introduce new problems. In fact, you're probably unlucky if a quick fix works, because people will settle on that approach from now on! Look, I love my customers to bits (really). But here's the thing … I wouldn't dream of suggesting a marketing strategy because I know absolutely nothing about marketing. So why do so many non-IT managers feel that they are qualified to make sensible judgements about IT strategy? The business pays my wages, but I sincerely hope that the reason they’re doing so is because I know some things that others might not and so I can give sensible guidance before they make their decision. It seems to me they're wasting a stack of money on expertise that they're not necessarily taking full advantage of. Phew! At least I didn't decide to talk about IT managers - too close to the bone, I reckon ;-)

Database In Depth - free chapter 1

Tue, 26 Jul 2005 21:15:17 +0000

Cool. When I used to teach courses, I liked to point attendees at the free online copy of chapter 1 of Expert One-on-one, the Tom Kyte book. I always thought that chapter was particularly inspiring and would encourage people to buy it once they'd read the sampler. Happily, I just noticed today that you can also get Date's chapter 1 for free. Hopefully it inspires your interest ... http://searchoracle.techtarget.com/searchOracle/downloads/Database_in_Depth_Chapter_1.pdf

A small follow-up

Mon, 25 Jul 2005 21:05:06 +0000

From a previous blog entry :- "... I noticed that the slave trace files are appearing in user_dump_dest. This is with 9.2.0.1.0 on Windows XP Home. I'm using 9.2 because I want to see the discrete SQL statements that the slaves are processing. So, just an interesting anomaly from what I've seen previously. I'll check it out on other versions / OS in future." Well, I've just tried it on 10.1.0.4 on XP at home and the px slave trace files are appearing in background_dump_dest, as usual.

Deferred Constraint Checking

Fri, 22 Jul 2005 16:26:58 +0000

In my previous blog entry I highlighted a few of the reasons that integrity constraint checking should not be deferred, stated by Date in his recent book (and by him and others in other papers). At the time and while looking at an entry over on Jeff Hunter's Blog I felt that functions might be a particular problem. My mind also went back to a chat with Carl Dudley and others after a Scottish SIG meeting when Carl had mumbled something about functions that had stuck somewhere in the back of my mind. So I thought I’d put together a simple example based on EMP and DEPT to illustrate some of the problems you could run into with deferred constraint checking. Here’s the data (unmodified from a standard SCOTT installation on 10.1.0.4)


SQL> SELECT empno, deptno, ename, sal
  2  FROM emp
  3  ORDER BY deptno, empno;

     EMPNO     DEPTNO ENAME             SAL
---------- ---------- ---------- ----------
      7782         10 CLARK            2450
      7839         10 KING             5000
      7934         10 MILLER           1300
      7369         20 SMITH             800
      7566         20 JONES            2975
      7788         20 SCOTT            3000
      7876         20 ADAMS            1100
      7902         20 FORD             3000
      7499         30 ALLEN            1600
      7521         30 WARD             1250
      7654         30 MARTIN           1250
      7698         30 BLAKE            2850
      7844         30 TURNER           1500
      7900         30 JAMES             950

14 rows selected.

SQL> SELECT deptno, dname
  2  from dept;

    DEPTNO DNAME
---------- --------------
        10 ACCOUNTING
        20 RESEARCH
        30 SALES
        40 OPERATIONS

So next I’ll create a simple function to test. The function just calculates the average salary for employees in the department specified by the p_dname parameter.


SQL> CREATE OR REPLACE FUNCTION func_avg_salary (p_dname dept.dname%TYPE) 
  2   RETURN NUMBER 
  3  IS
  4   n_avg_salary NUMBER;
  5  BEGIN
  6   SELECT AVG(emp.sal) INTO n_avg_salary
  7   FROM emp, dept
  8   WHERE emp.DEPTNO = dept.DEPTNO
  9   AND dept.dname = p_dname;
 10  
 11   RETURN n_avg_salary;
 12  END;
 13  /

Function created.

Note that anything this function does during this test could easily be more efficiently replaced by single SQL statements (avoiding PL/SQL altogether) but I mean to illustrate the problem at a simple level. I hope you can use your imagination to dream up much more worrying examples, involving more complex functions against a more complex model. It’s also worth thinking about whether a function might communicate outside of this transaction, for example by sending results out to file or via an email? Now I want to make sure that I have a deferred constraint to work with.


SQL> column table_name format a16
SQL> column constraint_name format a16
SQL> column column_name format a16

SQL> SELECT c.table_name, c.constraint_name, c.constraint_type,
  2   col.column_name, col.position
  3  FROM user_constraints c, user_cons_columns col
  4 WHERE  c.constraint_name = col.constraint_name
SQL> /

TABLE_NAME       CONSTRAINT_NAME  C COLUMN_NAME        POSITION
---------------- ---------------- - ---------------- ----------
DEPT             PK_DEPT          P DEPTNO                    1
EMP              PK_EMP           P EMPNO                     1
EMP              FK_DEPTNO        R DEPTNO                    1

SQL> ALTER TABLE emp DROP CONSTRAINT fk_deptno;

Table altered.

SQL> 
SQL> ALTER TABLE emp ADD CONSTRAINT fk_deptno FOREIGN KEY (deptno) REFERENCES dept
  2  INITIALLY IMMEDIATE DEFERRABLE;

Table altered.

Now that I have the deferrable foreign key constraint, I’m going to defer the checking until the end of the transaction.


SQL> SET CONSTRAINT FK_DEPTNO DEFERRED;

Constraint set.

That will let me insert some employees who work for the non-existent department 90.


SQL> insert into emp (empno, ename, job, mgr, hiredate, sal, comm, deptno)
  2  values (8000, 'BURNS', 'DBA', 7902, SYSDATE, 9000, 0, 90);

1 row created.

SQL> 
SQL> insert into emp (empno, ename, job, mgr, hiredate, sal, comm, deptno)
  2  values (8001, 'JONES', 'DBA', 7902, SYSDATE, 5000, 0, 90);

1 row created.

So now I *know* that I have inconsistent data. Or do I? (as James Burke may have said). In this case I do because I’m working with a very small number of statements; I checked which departments existed before I ran them; and I’ve just deferred the checking of the foreign key constraint. If I don’t know what I’ve just done, that’s what I would probably classify as ‘doing something stupid’. However, let’s use the function anyway.


SQL> COLUMN avg_sal FORMAT 999,990.90
SQL> 
SQL> select dname, loc, ROUND(func_avg_salary(dname), 2) avg_sal
  2  from dept;

DNAME          LOC               AVG_SAL
-------------- ------------- -----------
ACCOUNTING     NEW YORK         2,916.67
RESEARCH       DALLAS           2,175.00
SALES          CHICAGO          1,566.67
OPERATIONS     BOSTON

The results are correct really. For all of the departments that exist in the DEPT table, the function displays the average employee salary for that department. Of course, the two new employees that I just inserted are discounted because their department is non-existent. Whether this matters or not depends on the results you were expecting! Again, this is all pretty obvious, but I think the important thing here is that whoever wrote the function originally (some time in the distant past) was almost definitely expecting every employee to work for an existing department. That’s the whole point of having implemented the relational model and the constraints in the first place isn’t it? The author of the function could consider the possibility that there is invalid data in the database, but that’s going to require more work from the developer when the RDBMS should be guaranteeing this. Even if the developer of every function does check for any possible data anomalies – a big job across an application that seems a little pointless to me – am I going to have to check the code of every function to make sure that it performs these checks? All of which is not to say that I think deferred constraint checking is a terribly bad thing – I’m still quite a fan for those nightmarish initial data population exercises (if you’ve ever been hit by that, you’ll know what I mean). However, there are dangers and I’ve moved from a position of thinking that constraint checking could be deferred comfortably to thinking that it should only be used with care. Oh, and for those who haven’t used deferred constraint checking before, here’s what happens when I try to commit the invalid data


SQL> commit;
commit
*
ERROR at line 1:
ORA-02091: transaction rolled back
ORA-02291: integrity constraint (SCOTT.FK_DEPTNO) violated - parent key not found

The Multiple Assignment Operator and Deferred Constraints

Thu, 30 Jun 2005 13:52:24 +0000

I attended a day long seminar in Edinburgh last year with Chris Date lecturing and it was a rare treat. Thanks again to Peter Robson for organising this. Although I'm not the most academically-inclined techie, which I'm always honest about, I found what Chris had to say enlightening and mostly entertaining. There was one part that bothered me though and that was the whole section on ACID, deferred constraint checking and the need for a multiple assignment operator. I remember being quite bothered by Chris' insistence that database integrity must be maintained at statement boundaries, not transaction boundaries. As an Oracle DBA, my immediate reaction was to say (not out loud!) 'What about read-consistency? Isn't the main thing that at each commit point, the database is consistent? Who cares if it's temporarily inconsistent if I am the only transaction that can see those inconsistencies and that I knowingly introduced them?' Earlier this year, I heard a terrific presentation by Hugh Darwen at the Scottish SIG talking about NULLs. As an aside, he also mentioned the multiple assignment operator and consistency being maintained at statement boundaries, which kept me thinking ... I downloaded Multiple Assignment by Chris and Hugh Darwen and read through that, but I still wasn't convinced. So when I bought Database in Depth (which I talked about in my last posting) I cheated a bit and went straight to the section called "Why Database Constraint Checking Must Be Immediate" (not my capitalisation!). Having read that, I can see the points being made more clearly. It's a reasonably long section and I won't quote all of it (hopefully increasing the sales of the book in the process), but here are a few lines on the first three of the five reasons Chris gives for immediate constraint checking being mandatory. 1) "While it might be true, thanks to the isolation property, that no more than one transaction ever sees any particular inconsistency, the fact remains that that particular transaction does see the inconsistency and can therefore produce wrong answers" 2) "For if transaction T1 produces some result, in the database or elsewhere, that's subsequently read by transaction T2, then T1 and T2 aren't truly isolated from one each other (and this remark applies regardless of whether T1 and T2 run concurrently or otherwise)." 3) "We surely don't want every program (or other "code unit") to have to cater for the possibility that the database might be inconsistent when it's invoked." So I can't really argue with any of what Chris is saying there, other than to highlight that problems one and two are problems if you choose to do something stupid, like writing out values half-way through a transaction that another source could use, or introducing an inconsistency in your own transaction and not being aware of it. The third reason, though, seems to be a more likely problem and a variant of the first two. If I write a function that could be called inside another transaction, what can I say about the consistency of the database when my function is called? I think most database professionals would understand the great value of a database in which the data could never be inconsistent (particularly if you've worked with databases when the data is usually inconsistent). Perhaps it's an area that doesn't call for pragmatic workarounds, but rigid rules? All interesting stuff and I can't recommend this book highly enough.

Chris Date's new book

Fri, 24 Jun 2005 10:06:53 +0000

I've just started reading Chris Date's new book and I can recommend it very strongly if, like me, you're a bit lacking in relational theory. It was originally recommended to me by my friend Peter Robson, who was one of the reviewers. Although I've read a number of Chris' articles on dbdebunk and I had the pleasure of attending a one day seminar that Chris gave in Edinburgh last year, I'm finding this book more digestable. It's still rigorous, but I love the way that he's taken what some feel is a slightly irrelevant academic subject and made it sing (in my ears at least). There's still a little bit of :- "Let relations r and s have attributes X1, X2 ...." but I've found that the surrounding text makes it more bearable (for me). I think this is a first class book and thoroughly relevant to real-world database systems. Even if you don't agree with all of it's content (although I'm finding it all pretty agreeable so far!), you will be in a better position to state *why* you disagree! If you already have some of Chris' lengthier and more academic texts, maybe this book won't add anything to your knowledge, but if you've never read one this would be a good place to start. Put it this way. I got to bed late last night because I couldn't put it down and had to force myself to in the end. Sad, but true. The last book that happened with was Tom Kyte's Expert One-on-One, so it's a good sign ...

dba_registry (again)

Wed, 22 Jun 2005 14:02:19 +0000

(Switch humour on ...) Many thanks to Pete Finnegan for making sure that my embarassing mistakes kick around on recent entries at Orablogs! (and off) To quote Pete ...

Doug seems embarrassed a little by this but this is common mistake

Make that very embarassed, if you don't mind! Here are the reasons 1) I've got no business applying patches without reading the instructions properly. My excuse here would be that I was doing it at home so was a lot more casual than I would be at work. (Interestingly, the db at work which has this problem is a Dev/Test environment) 2) It's a classic case of the old Don Rumsfeld/Tom Kyte/J Lewis (and many others) point about the things you think you know. Just because I expect a patch utility that comes back with a success message to have run catpatch, I should have checked. Because it turns out I was completely wrong. 3) How can I slag off the standard of other people's DBA work when I can't do something as simple as this properly?!? (between you and me, their standards still make me look good enough for me ...) 4) It's still bugging me that I know I've applied patches, so have I run catpatch? I reckon that I have because at work I would sit and go through the instructions, just to be sure. So why didn't this spring to mind? I suspect that every patch I've ever applied has been through a command-line process on Unix too, so I'd already be in that 'step-by-step with instructions' frame of mind. When I'm running software at home, I tend to install new versions or upgrades rather than patches. More fool me, but not having your own Metalink account doesn't help .... I think there are a few issues here that genuinely aren't just about defending myself, but might catch out others :- a) If there's a GUI patch utility to apply a patch on Windows (and remember, this was the latest 10g patch I could find), it should run catpatch.sql too. Why wouldn't it? The least it could do is display a clear message saying 'Make sure your run catpatch.sql'. (Then again, I know there are installation instructions which make this clear. I checked them) b) The very fact that this story started with the discovery of a db at work that hadn't had catpatch run against it proves that I'm not the only one. The fact that there are blog entries about this in other people's blogs is another sign. c) Here's another possible approach. Why do the newly-patched executables allow you to open the database when the dictionary version is wrong? Given that Oracle has this information in the dictionary, why not check it? I'm sure I remember occasions when someone had forgotten to run an upgrade script and so the db wouldn't open (but better keep an eye on my growing senility here). All of which doesn't remove the egg from my face, but at least :- 1) I'm happy to admit my mistakes 2) I bet I'm not the only one!

dba_registry

Sun, 12 Jun 2005 22:19:30 +0000

It's true I learn something new all the time. Is it just me? Did everyone know about this? I was talking to one of the other DBAs at work on Friday who was telling me that one of the dbs we've taken over looks like it wasn't upgraded properly. I connected okay, v$version looked okay, so I asked him what he meant and he pointed me at dba_registry. (I've had to fake this slightly because it was 9i at work and I have 10g at home where I'm doing this - you'll see what I mean in a minute)

SQL> l
  1* select comp_id, comp_name, version, status from dba_registry
SQL> /

COMP_ID
------------------------------
COMP_NAME
----------------------------------------------------------------------------------------------------
VERSION                        STATUS
------------------------------ -----------
CATALOG
Oracle Database Catalog Views
9.2.0.1.0                      VALID

CATPROC

Oracle Database Packages and Types
9.2.0.1.0                      VALID

The database was using 9.2.0.4 binaries and other databases that were showed 9.2.0.4 in dba_registry. I have to be honest and say I didn't know about this. He pointed out that I should from when I've applied patches, but I know I've upgraded plenty of dbs without checking this (although I always check the log files of dictionary scripts I've run). Anyway, it was good to learn something new. So when I applied the 10.1.0.4.0 patch to my home set-up this weekend, I thought I'd check this view out again.


  1* select comp_id, comp_name, version, status from dba_registry
SQL> /

COMP_ID
------------------------------
COMP_NAME
----------------------------------------------------------------------------------------------------
VERSION                        STATUS
------------------------------ -----------
CATALOG
Oracle Database Catalog Views
10.1.0.2.0                     VALID

CATPROC
Oracle Database Packages and Types
10.1.0.2.0                     VALID

OWM
Oracle Workspace Manager
10.1.0.2.0                     VALID

JAVAVM
JServer JAVA Virtual Machine
10.1.0.2.0                     VALID

XML
Oracle XDK
10.1.0.2.0                     VALID

CATJAVA
Oracle Database Java Packages
10.1.0.2.0                     VALID

EXF
Oracle Expression Filter
10.1.0.2.0                     VALID

ORDIM
Oracle interMedia
10.1.0.2.0                     VALID

ODM
Oracle Data Mining
10.1.0.2.0                     VALID

CONTEXT
Oracle Text
10.1.0.2.0                     VALID

XDB
Oracle XML Database
10.1.0.2.0                     VALID

APS
OLAP Analytic Workspace
10.1.0.2.0                     VALID

XOQ
Oracle OLAP API
10.1.0.2.0                     VALID

AMD
OLAP Catalog
10.1.0.2.0                     VALID

SDO
Spatial
10.1.0.2.0                     VALID

EM
Oracle Enterprise Manager
10.1.0.2.0                     VALID


16 rows selected.

I obviously have a lot more stuff installed at home, or this view shows more information in 10g. However, the version numbers aren't 10.1.0.4.0 which means that it's obviously not a sure-fire way of checking an upgrade's been applied correctly. I suppose the dictionary may not have changed. I'm intrigued - is this something that's widely-known?

The importance of OFA

Fri, 10 Jun 2005 10:17:18 +0000

When discussing Cary Millsap and his work (for example during this week’s user group presentation or when teaching courses), I’ve often said that I wish he was more widely known for one of my favourite pieces of work - the various Optimal Flexible Architecture documents and standards. I think one of the benefits of having worked with Oracle for a long time is that I remember pre-OFA days so I know precisely what problems OFA was designed to address. Because Oracle eventually incorporated Cary’s ideas into the standard installation procedures, DBAs with less experience have become accustomed to the filesystem layouts, but I wish more of them had read the original standards or the installation manual or system admin guides. (It’s in different books on different versions and operating system ports). Otherwise you get a variation on OFA that’s been picked up by looking at an OFA directory structure and modifying it slightly in ways that totally miss the point. Here are some filesystems I’ve come across recently /myapp1/oradata/ /myapp1/oraindex/ /myapp2/oradata/ /myapp2/oraindex/ Which cause a few problems 1) We have large numbers of discrete filesystems for different applications. So if one application starts to run out of space, we might have free space kicking around on other applications’ filesystems, but it’s unusable! 2) So people start putting files in inappropriate directories e.g. take an myapp1-related datafile and put it in /myapp2/oradata. Which will, of course end up being deleted by some enthusiastic sys admin when the other application reaches its end of life. 3) If we suddenly find that we need more index tablespace space and less data tablespace space, do we either a) take everything offline so we can steal from oradata to give to oraindex; or b) stick an index tablespace datafile in the oradata filesystem? If it’s b), why didn’t we just call them both oradata in the first place?!?!? Then it gets worse. Under the /myapp1/oradata mount-point, we have environment-specific directories dev test support But the relevant instance names are (made these up) DOUGDEV DOUGTST DOUGSUPP What exactly was the point in using anything but the SIDs to identify which database the files belonged to? Trust me, there's loads of this stuff. In 2005. I worked it out – the first OFA paper I saw was in 1993. How can people call themselves DBAs when they still can’t get it after 12 years?!!? I think I need a lie down …