Consultant adamant in not using SDLC for data warehouse dev

Moderators: chulett, rschirm

Post Reply
chinek
Participant
Posts: 75
Joined: Mon Apr 15, 2002 10:09 pm
Location: Australia

Consultant adamant in not using SDLC for data warehouse dev

Post by chinek »

Hi,
In our data warehouse team, we have been using the SDLC method for our projects. Recently we have a new consultant (contracting in our company) who is adamant that SDLC is not suited for data warehouse projects. He's gone ahead in doing what he calls as prototyping, meaning he's building ETL jobs on the fly,produce results by hitting production dbs,get users to verify the data and refine their requirements and repeating that process. So when we questioned him about this he got very defensive and said that SDLC does not apply to data warehouse type development. Our issue is that he's been spending one month on his "requirements phase" and we don't know what the scope is,no functional specs,in fact no documentation at all.
His point is that the users do not understand or know what their requirements is so he's prototyping (or in his words producing pseudo-production reports) for them so that they then know what they want.
What should we do ? Or is he right in his approach ?
Our manager thinks very highly of him since he's from one of the big consulting firms..

Thanks.
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

Sounds like a perfect consulting project. He can spend as long as he likes doing whatever he wants! There is no scope or requirements so he doesn't need to produce anything. There is no documentation so you cannot check what he is doing or replace him.

I think a prototype work best if you assume you are going to discard it after it has done its job and build the proper version using SDLC. Sounds like he is doing development work directly into production without any SDLC and calling it prototyping. Can't see it being anything but a mess. We built prototype reports in an Access database and showed the reports to the users to get the requirements right. We then built it for real using DataStage and DB2 databases. We didn't touch prod because that's just asking for trouble.

Having said that I sure hope he's not from the same big consulting firm as me... :(
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

The correct term is a Spiral SDLC, which means a series of compacted Waterfalls that continuously feed and build on each other. Rather than a single, 18-month Waterfall SDLC that's supposed to build everything, you set small goals and in 90 days or less turn around iterative enhancements and improvements. This continues indefinitely, as you're never done. Source systems undergo enhancements for new data that the warehouse in turn has to capture. Data marts come and go as user requirements change.

Spiral, look it up.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

-craig

"You can never have too many knives" -- Logan Nine Fingers
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

There is some discussion on the spiral or iterative approach to project management over on GanttHead. There seems to be some agreement that scope creep can be a problem when you are constantly going for customer feedback. It can be hard to keep to a project schedule when you don't know what the end user is going to want. There was one suggestion to keep the iterations to a fixed length, eg. three weeks, and have a deliverable at the end of each period.

I think even with a prototype or spiral or iterative approach there still needs to be some definition of scope and scheduling up front and some documentation of requirements along the way.
chinek
Participant
Posts: 75
Joined: Mon Apr 15, 2002 10:09 pm
Location: Australia

Post by chinek »

Hi,

Thanks for all your inputs. I didn't know of the spiral lifecycle. Guess I do now. At the rate he's going I just think he's ripping the company off ,thankfully I'm not the one paying his wages.
From reading about the spiral method, there has to be some sort of controls but in our case, he's doing whatever he sees fit and there are no set deliverable or time constraint. I guess he'll be here for a long time. :cry:

ps vince,he's from another consulting co.
chinek
Participant
Posts: 75
Joined: Mon Apr 15, 2002 10:09 pm
Location: Australia

Post by chinek »

Hi,

Thanks for all your inputs. I didn't know of the spiral lifecycle. Guess I do now. At the rate he's going I just think he's ripping the company off ,thankfully I'm not the one paying his wages.
From reading about the spiral method, there has to be some sort of controls but in our case, he's doing whatever he sees fit and there are no set deliverable or time constraint. I guess he'll be here for a long time. :cry:

ps vince,he's from another consulting co.
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

chinek wrote:ps vince,he's from another consulting co.
Thought so. We are not allowed near a client site without waving our Proven Course Methodology or our MIKE DW methodology around. There are a very large number of DW projects out there that crashed and burnt because they didn't have a good methodology.
Post Reply