Hello,
I have been asked to take on the task of evolving ETL architecture, design and development standards specifically using Datastage and the Infosphere suite of tools.
I have started gathering and reviewing documents, presentations searching the web.
I look towards this group for suggestions, comments, ideas regarding the above.
Thank you for reading this post and any contributions.
/A
ETL Development standards, best practices, guidelines etc.,
Moderators: chulett, rschirm, roy
ETL Development standards, best practices, guidelines etc.,
ETL, DW, BI Consultant
-
- Participant
- Posts: 44
- Joined: Wed Sep 02, 2015 7:19 am
- Location: Brasilia, Brazil
Not that I recall on 'standards' and such. There has always be a Server Job Developer's Guide that shipped with the product or the online documentation that you can find here.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Here's another non-documented practice you should instill in your developers right from the start.
Create Diagnostic jobs for your database connectivity.
Connector Stage -> Peek
select * from table_x WHERE 1=2
You will save so much time just by that stupid little job using the same parms as your real etl job. All it does is validate connectivity. You would never run it on a daily or nightly flow. It would only be used if you change your connectivity credentials or if your environment got hosed and you simply want a ping test to your database.
Executing your suite of diagnostic jobs the day before your big GO LIVE roll-out... gold.
Create Diagnostic jobs for your database connectivity.
Connector Stage -> Peek
select * from table_x WHERE 1=2
You will save so much time just by that stupid little job using the same parms as your real etl job. All it does is validate connectivity. You would never run it on a daily or nightly flow. It would only be used if you change your connectivity credentials or if your environment got hosed and you simply want a ping test to your database.
Executing your suite of diagnostic jobs the day before your big GO LIVE roll-out... gold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: