DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
scorpion
Participant



Joined: 12 May 2005
Posts: 144

Points: 1182

Post Posted: Tue Mar 09, 2010 3:00 am Reply with quote    Back to top    

Hi All,

I just wanted to know that:

Is there any utility or tool avilable to make the code review and cross check the naming standars or other checks in job ,which allows in automated way.

For ex: if I have 1000 jobs running in prod/dev and now as part of the COE i decided to check whether all those jobs are following the standards or best practises.

Instead of going and opening the individual job one by one ,any automated utility is avlable?

Thanks in advance.

_________________
Thanx&Regards
scorpion
ArndW

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup

Joined: 16 Nov 2004
Posts: 16318
Location: Germany
Points: 92566

Post Posted: Tue Mar 09, 2010 5:22 am Reply with quote    Back to top    

No such tool is available. Some of us, myself included, have coded small applications to do such checking, but since each site has different standards that they wish enforced, such code is generally not re-usable.

_________________

Image
Rate this response:  
Not yet rated
PhilHibbs



Group memberships:
Premium Members

Joined: 29 Sep 2004
Posts: 1044
Location: Nottingham, UK
Points: 12411

Post Posted: Fri Mar 19, 2010 8:05 am Reply with quote    Back to top    

I have an Excel spreadsheet with VBA that gathers a load of information about a DSX file. It's quite sophisticated and thorough, but a lot of it is hard-coded to the way we do things, and we only use it for Server jobs. It isn't the best, most structured piece of code in the world ether. I'll share a copy with anyone who is interested, but I don't have anywhere to upload it so it will have to be via private email.

It gathers:
    Job category, name, type, timestamp, change history (in our own convention)
    SAP connection details
    SAP ABAP extract program names
    SAP IDocs
    Parameters & default values
    File stage details and file names (expanded with default parameter values)
    File schemas
    Version information (again, using our own convention), and a function to compare version info with an analysis of another project
    CRC of all constraints, derivations, and lookup keys

All information is stored in spreadsheet tabs. It takes progressively longer the more jobs you have, at around 2000 it can get very slow, so I tend to cut my DSX files first.

I need to add Stage Variable gathering as well, so that those can be checked against naming standards.

Search for "DataStage Analysis spreadsheet" to find the thread dedicated to this tool, my last post in that thread will have a download link for the latest version.

_________________
Phil Hibbs | Capgemini
Technical Consultant

Last edited by PhilHibbs on Mon Aug 02, 2010 3:27 am; edited 1 time in total
Rate this response:  
Not yet rated
kduke

Premium Poster


since February 2006

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 29 May 2003
Posts: 5227
Location: Dallas, TX
Points: 35013

Post Posted: Fri Mar 19, 2010 2:20 pm Reply with quote    Back to top    

Good idea. I would like to see the code.

_________________
Mamu Kim
Rate this response:  
Not yet rated
PhilHibbs



Group memberships:
Premium Members

Joined: 29 Sep 2004
Posts: 1044
Location: Nottingham, UK
Points: 12411

Post Posted: Tue Mar 23, 2010 8:53 am Reply with quote    Back to top    

PhilHibbs wrote:
I need to add Stage Variable gathering as well, so that those can be checked against naming standards.

This is now done, stage variables are gathered onto a tab with a count of the number of references to each, as with job parameters. PM me with your email address if you are interested in this project analysis tool.

_________________
Phil Hibbs | Capgemini
Technical Consultant
Rate this response:  
Not yet rated
kduke

Premium Poster


since February 2006

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 29 May 2003
Posts: 5227
Location: Dallas, TX
Points: 35013

Post Posted: Tue Mar 23, 2010 4:16 pm Reply with quote    Back to top    

Got it. Thanks.

_________________
Mamu Kim
Rate this response:  
Not yet rated
JoshGeorge
Participant



Joined: 03 May 2007
Posts: 612
Location: Melbourne
Points: 3747

Post Posted: Sun Mar 28, 2010 9:08 am Reply with quote    Back to top    

XMETA (repository) provides this information in different tables. You can check there only by writing queries or get the information into a spreadsheet to generate a report with the standards defined there.

Quote:
In the table name like DATASTAGEX%*%DSJOBDEFC2E76D84 you will get

CATEGORY_XMETA (For folder name)
NAME_XMETA (For job name))
DSJOBTYPE_XMETA (For job type) [ 0 is Server job, 2 is Sequencer, 3 parallel job ...]


Quote:
In the table name like DATASTAGEX%*%DSSTAGEC2E76D84 you will get

NAME_XMETA (For stage names)

_________________
Joshy George
Rate this response:  
Not yet rated
abc123



Group memberships:
Premium Members

Joined: 25 Aug 2006
Posts: 598

Points: 5344

Post Posted: Wed Jul 07, 2010 4:56 pm Reply with quote    Back to top    

Josh, I don't see a table called

DATASTAGEXNAME_XMETADSJOBDEFC2E76D84

for all job names?
Rate this response:  
Not yet rated
rohitash_saini
Participant



Joined: 08 May 2019
Posts: 4

Points: 16

Post Posted: Wed May 08, 2019 10:26 pm Reply with quote    Back to top    

Phil,
Can you please share this spreadsheet ?

_________________
Rohitash
Rate this response:  
Not yet rated
PhilHibbs



Group memberships:
Premium Members

Joined: 29 Sep 2004
Posts: 1044
Location: Nottingham, UK
Points: 12411

Post Posted: Thu May 09, 2019 5:36 pm Reply with quote    Back to top    

rohitash_saini wrote:
Can you please share this spreadsheet ?

Done, link updated in the first post of the main thread . Dropbox changed the way public shared files worked, it's now on Google Drive. If anyone has a better place to host it, please feel free to "mirror" it and I will include a link in my post.

_________________
Phil Hibbs | Capgemini
Technical Consultant
Rate this response:  
stuartjvnorton
Participant



Joined: 19 Apr 2007
Posts: 525
Location: Melbourne
Points: 3904

Post Posted: Thu May 16, 2019 7:37 am Reply with quote    Back to top    

MettleCI offers what it calls Compliance checking as part of its CICD processes, and it can also be run standalone through the Workbench.
It has a number of pre-canned rules to check things like naming standards, use of deprecated stages, adherence to best practices (no hardcoded DB credentials or file names, link sorts), etc. You can also write your own using Gremlin.

https://www.mettleci.com/mettleci.html#videos
Rate this response:  
rohitash_saini
Participant



Joined: 08 May 2019
Posts: 4

Points: 16

Post Posted: Tue May 21, 2019 1:51 am Reply with quote    Back to top    

Thank you Phil.

_________________
Rohitash
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours