What's a 'Data Warehouse DBA'?

Doug's Oracle Blog

  • Home
  • Papers
  • Books
  • C.V.
  • Fun
  • Oracle Blog
  • Personal Blog

Jul 4: What's a 'Data Warehouse DBA'?

I've seen that question asked often in forums and mail groups. It's easy to understand why there's so much confusion because, for most DBAs, if you've seen one database, you've seen them all. Or rather, you haven't seen any of them. What I mean is that every database that lands in your hands is new, has it's own users, it's own problems, it's own performance profile. So learning and coping with new situations is an implicit part of being a DBA and if you're a *good* DBA, a data warehouse should be well within your capabilities.

The very first Oracle database I worked on was a data warehouse, although possibly not in the sense we understand today. It didn't use a Kimball-style data model, materialised views didn't exist in Oracle and the volumes of data were tiny compared to current warehouses. However, it did have a fact table for all of our customers, one for all of their accounts, a many-to-many join table between the two and tons of small? reference tables. We used it to run complex, user-defined queries to generate lists of customers for direct marketing mail-shots.

When recruitment agents started to ask me about my Data Warehouse experience, I was very blas in response because I know my skills are up to most jobs and was frustrated when they wanted to see 'Data Warehouse DBA' on my cv. To my mind a good DBA is more than capable of looking after Data Warehouses too and it doesn't help when slightly famous bloggers say things like (and this isn't a direct quote) 'I do data warehouses - they're different'. Which is weird, because you don't see people going around saying 'I do several thousand user OLTP systems'. My point here is that there is a certain cliqueness or possibly? elitism? about the DW crowd. Now, before I upset everyone (because all of my friends seem to be members of that crowd ;-), let me explain why I think they're like that.

I work with the DW development team quite a lot in my current workplace. One of the reasons for that (and I'm going to have to leave my modesty to one side for a moment) is that they could quickly spot that I knew what I was doing. That's the good thing about the DW crowd - they're quite demanding. Whilst working with them, I heard horror stories about what previous DBAs had done, so they are very twitchy about the skills of the people who work on their databases. In fact, we had a DBA start here not so long ago and the first bit of advice I gave him was 'when the DW guys ask you to do something, the chances are that they understand the reasons for it better than you do.' But that's because they spend all of their day working on the same couple of databases. They have tons of time to think through the issues. It's not because they're party to some mysterious knowledge that takes years to learn (or at least no more mysterious than just being a DBA on big databases).

To further counter my own argument, here are some things that I think are definite requirements for a DBA to work on Data Warehouses (even if they choose not to glorify themselves with the full title)

1) Knowledge of Partitioning, Materialised Views, Bitmap Indexes, Parallel Execution and, in fact, most things in the handily-supplied Data Warehousing Guide

2) Knowledge of the latest features because DW systems always seem to be on the latest release because they need the features.

3) Experience of working with big databases. This sounds obvious, but how do you go about backing up a 2Tb warehouse? Maybe it makes more sense to rebuild indexes than backup the tablespaces. Believe it or not, there *might* be good reason for having a tablespace with hundreds of files - just because you haven't seen one before doesn't make them bad ;-)

So I'm not saying working with Data Warehouses is easy, just that it's not some black magic art-form. I repeat - the most important thing is that you're a good DBA in the first place and that you're open-minded.

Actually maybe I've argued my way into a corner here and there is a lot to being a Data Warehouse DBA but it just doesn't seem that big a deal to me. Maybe I've got more experience than I think I have and I should start calling myself a Data Warehouse DBA, however .... The other week I had a conversation with a recruitment agent that spread over several phone calls and ultimately degenerated into a series of 'so how big was that database, then' questions, going through each of the sites I'd worked at. His client's requirement was for DBAs who'd worked on Terabyte warehouses and, as some of mine were only six or seven hundred gig, it was proving tricky. Eventually I gave it up as a lost cause but it is disappointing that this job title might be excluding good DBAs from jobs because they don't see it as being a specialised position and therefore don't see the need to change the job title on their cvs (or blog!).

To be honest, I'm feeling pretty depressed about the state of the DBA role this month, so expect this to be one of a series of rants.
Posted by Doug Burns Comments: (14) Trackback: (1)

Trackbacks
Trackback specific URI for this entry

What's a 'Development' DBA?
I theory, this should be a much more straightforward, less contentious question than my previous - 'What's a Data Warehouse DBA?'A development DBA looks after the development (and test) databases. It truly is as simple as that, but depends on the nature
Weblog: Doug's Oracle Blog
Tracked: Jul 21, 01:15

Comments
Display comments as (Linear | Threaded)

#1 - Howard J. Rogers said:
2006-07-04 18:49 - (Reply)

I'm reasonably confident that the 'What I say may not apply to you' thing has nothing at all to do with cliquiness, and everything to do with a recognition that what a lot of datawarehouse people do is "odd". In a world taught to understand 3rd normal form, materialised views and the concept of deliberate de-normalisation generally are just peculiar. When we all know indexes 'make things go faster', a bunch of people employing full table scans intelligently can be a wonder to behold. And so on...

Of course it's not magic, but it's less common than the OLTP-style database, which follows all the "right" rules that Uni-style relational database theory says should be followed, instead of seeming to break them at will.

Your own Point 1 in your list of things every Data Warehouse DBA should know gives the game away: Partitioning, Parallelism, Bitmap Indexes... they're all Enterprise Edition only, and sometimes extra license options on top of that. Which means: Data Warehousing is uncommon, uncommonly expensive to get to know... and at the end of that day, who *really* understands what on Earth the optimiser is doing with parallelism anyway?!

#2 - Paul Vallee said:
2006-07-04 19:54 - (Reply)

This is just another example of someone not having any idea how to actually interview somebody for a DBA job. Really, they get who they deserve. Their customers too. Now really, would you want to join a team full of people who pass this type of interview, anyway?

Paul

#3 - Pete_S said:
2006-07-04 20:09 - (Reply)

I have had people tell me that they want to start a career as a 'DW DBA' and ask how to start. The simple answer is learn to be a DBA and learn the craft well. The most important DBA skills come from general experience

#4 - Doug Burns said:
2006-07-04 20:22 - (Reply)

Ah well, I was trying to be contentious so I expected some interesting responses (and Pete Scott hasn't even turned up yet!) I was actually trying to think of something to argue with in Howard's comment but I can't really so I won't, except perhaps ....

"they're all Enterprise Edition only, and sometimes extra license options on top of that. Which means: Data Warehousing is uncommon, uncommonly expensive to get to know"

I don't know about that. Maybe I should be more explicit and talk about experienced contract DBAs (which should be obvious from their cv). Most of the time most of them will be using Enterprise Edition, if only due to management laziness when negotiating licensing agreements. I would also argue that most sites these days have at least one DW, so exposure to them may be intermittent, but not that uncommon.

"who really understands what on Earth the optimiser is doing with parallelism anyway?!"

Mr. Lewis, maybe. Certainly not me!

But, you know, the more I think about it the more I remember there were DW things that I had to encounter for the first time, but I still maintain any experienced DBA should have come across them a few times. I have to admit, though, that the DW designers and developers tell me that I'm wrong about that.

P.S. I *still* think they're cliquey ;-)

#5 - Doug Burns said:
2006-07-04 20:26 - (Reply)

It would appear that Pete has turned up ;-)

#6 - Doug Burns said:
2006-07-04 20:32 - (Reply)

Paul said

"This is just another example of someone not having any idea how to actually interview somebody for a DBA job."

It would be difficult to think of a more different interview than the ones with you. What was a shame, though, was that I suspect that the guy's reasons for being so suspicious and not a little tiresome were the same reasons for your positive interviews - he's had his fingers burned before now and didn't want to submit a lame duck cv.

I probably don't sell myself well enough in my cv either.

#7 - Pete_S said:
2006-07-04 20:44 - (Reply)

P.S. I still think they're cliquey

- I'd need friends to be cliquey

#8 - Andy C said:
2006-07-04 21:38 - (Reply)

Slight non sequitur but I used to do ETL type work (taking data out and loading data in). A lot of very important DBA's and business analysts proudly told me they worked on the 'Data Warehouse'.

When you probed a little deeper, it transpired they were wrong. By definition, not many companies have multiple DW's. They were actually working on operational systems that would have been prime candidates to feed a DW but were just in denial :-)

#9 - Peter Lewis 2006-07-04 21:53 - (Reply)

... does all this make DBAs dealing with Spatial databases even more prone to being in a clique than the DW chaps? Because Spatial DBs are less common than data warehouses? I don't think so. Cliques are probably not such a good thing anyway.

There are plenty of bits and pieces in the world of Oracle Spatial that don't necessarily fit easily when compared to a good old OLTP system. Hundreds of datafiles in a tablespace *can be* one of them.

Exposure to differing requirements and types of implementation makes for an ever broadening knowledge... but despite that I actually have a nagging feeling that I know proportionately less about Oracle as each day passes... but that might be an age thing... Oh well.

#10 - Doug Burns said:
2006-07-04 22:15 - (Reply)

Pete S said

"I'd need friends to be cliquey"

Oh, don't come that with me! I saw you, Mark Rittman and Jeff Moss hanging around darkened corners at UKOUG. I do believe there may have been a cauldron in the vicinity too ;-)

Andy C said

"A lot of very important DBA's and business analysts proudly told me they worked on the 'Data Warehouse'."

And that's at least part of what I'm getting at.

Peter said

"does all this make DBAs dealing with Spatial databases even more prone to being in a clique than the DW chaps? "

Oh, definitely! (That would be a joke actually)

#11 - Jeff Moss 2006-07-07 08:44 - (Reply)

That made me laugh...you described me to a tee...the bit about when a new DBA comes along on MY warehouse and I need something doing by them...first thought is always, "Who is this guy/gal ?" and do they really understand the database they are about to work on ?

It's not that most DBA's can't deal with a DW but that they just need a mindset which allows them to stop and think about the consequences of the actions they take and how the specifics of a warehouse can influence those actions - but there again...that's probably a characteristic all DBA's should have whatever the system they are working on!

Yes, there are specifics to a DW system as there are to an OLTP one but any good DBA will be able to handle either if they apply some common sense and have a good understanding of the features of the database that are relevant to that environment.

On my current DW we've taken the approach of spreading the knowledge of the environment around quite a few of the DBA team in order that we have good coverage. We've also spent quite a large amount of time documenting the architecture in order that a new resource arriving on the project can at least hit the ground running. It seems to be working well thus far.

Oh, and don't get me started on Recruitment Consultants!

PS - When do you want the cauldron back ?

#12 - David Aldridge said:
2006-07-19 14:52 - (Reply)

I explect that there are certain habits that come from OLTP administration that are not applicable to DW administration -- a preference for a SGA/PGA size ratio greater than 1 for example, where a data warehouse using PQ will have a ratio less than 1. That could trip people up.

PQ is generally a Big Deal, I think (as well you know, young man!) and so is the presence of lots of read-only tablespaces.

I expect that there are similar differences between RAC and non-RAC systems, whereby a RAC administrator has to consider all that the regular administrator considers plus a little more.

#12.1 - Doug Burns 2006-07-19 15:11 - (Reply)

Yeah there are a few differences, no doubt, but I don't know whether I should call myself a DW DBA because I know those things?

Do I need a label change like that? I guess so, because people seem to describe themselves as DW consultants, not just Oracle or database consultants. See what I mean? When did you last hear someone describe themselves as an OLTP DBA, or an Internet DBA?

#13 - David Aldridge said:
2006-07-19 17:39 - (Reply)

Maybe "DW Consultant" is shorthand for "Don't ask me anything about bind variables" :-D


Add Comment

Standard emoticons like :-) and ;-) are converted to images.
E-Mail addresses will not be displayed and will only be used for E-Mail notifications.
BBCode format allowed
 
 

Statistics on Partitioned Tables

Contents

Part 1 - Default options - GLOBAL AND PARTITION
Part 2 - Estimated Global Stats
Part 3 - Stats Aggregation Problems I
Part 4 - Stats Aggregation Problems II
Part 5 - Minimal Stats Aggregation
Part 6a - COPY_TABLE_STATS - Intro
Part 6b - COPY_TABLE_STATS - Mistakes
Part 6c - COPY_TABLE_STATS - Bugs and Patches
Part 6d - COPY_TABLE_STATS - A Light-bulb Moment
Part 6e - COPY_TABLE_STATS - Bug 10268597

Comments

Doug Burns about 10053 Trace Files - Different Plan in Different Environments
Tue, 02.04.2013 08:57
You're welcome. Now I just nee d to pull my finger out and ac tually come up [...]
Howard Rogers about 10053 Trace Files - Different Plan in Different Environments
Mon, 01.04.2013 23:08
Makes a big difference, so tha nks for that! With two brow ser windows, o [...]
stelioscharalambides.com about 10053 Trace Files
Sat, 30.03.2013 16:28

Upcoming Presentations

Bookmark

Open All | Close All

Syndicate This Blog

  • XML RSS 2.0 feed
  • ATOM/XML ATOM 1.0 feed
  • XML RSS 2.0 Comments
  • Feedburner Feed

Powered by

Serendipity PHP Weblog

Show tagged entries

xml 11g
xml ACE
xml adaptive thresholds
xml ASH
xml Audit Vault
xml AWR
xml Blogging
xml conferences
xml Cuddly Toys
xml Database Refresh
xml DBMS_STATS
xml Direct Path Reads
xml Fun
xml grid control
xml hotsos 2010
xml listener
xml Locking
xml oow
xml oow2009
xml optimiser
xml OTN
xml Parallel
xml Partitions
xml Patching
xml swingbench
xml The Reality Gap
xml time matters
xml ukoug
xml ukoug2009
xml Unix/Shell
xml Useful Links

Disclaimer

For the avoidance of any doubt, all views expressed here are my own and not those of past or current employers, clients, friends, Oracle Corporation, my Mum or, indeed, Flatcat. If you want to sue someone, I suggest you pick on Tigger, but I hope you have a good lawyer. Frankly, I doubt any of the former agree with my views or would want to be associated with them in any way.

Design by Andreas Viklund | Conversion to s9y by Carl