Search found 97 matches
- Tue Aug 06, 2019 8:11 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: ISD job to load flat file to a table
- Replies: 6
- Views: 4979
Thank you both. Craig, you've reminded me of how simple this can be. We can't have Toad but I have checked quickly with SSMS and can do similar (need to resolve character set defaults). The scope, roughly, is that about 10 users would load about 2 files per month, and doing so without involving us (...
- Sat Aug 03, 2019 7:09 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: ISD job to load flat file to a table
- Replies: 6
- Views: 4979
ISD job to load flat file to a table
Hi experts, Looking for a good idea on how to load any flat file to SQLServer DB. We have plenty of flat files and some of them can be pretty large. The owners of these files would like test loading them into DB on an ad-hoc basis for exploratory purposes. The database would give them better perform...
- Tue Nov 20, 2018 2:57 pm
- Forum: General
- Topic: Automated deployments using Team Foundation Server?
- Replies: 4
- Views: 4608
Hi guys, I have not explained this well. The focus is on the automated deployment tools, ie. Jenkins, Mamboo or TFS in our case. What are some of the best practices these days. An option could be to have IIS objects: 1) compiled 2) exported 3) stored into version control 4) tagged with the relevant ...
- Tue Nov 20, 2018 4:03 am
- Forum: General
- Topic: Automated deployments using Team Foundation Server?
- Replies: 4
- Views: 4608
Automated deployments using Team Foundation Server?
Hi experts, Would you please be able to share your experiences with setting up automated deployment of DataStage, Information Analyzer and ISD jobs? What is the best approach these days? We will have 2 release Microsoft TFS Servers to support our non-Prod and Prod ETL servers. Looking to avoid insta...
- Tue Nov 20, 2018 3:45 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Checksum Stage limitation
- Replies: 2
- Views: 3050
Definitely strange if wrong hash result is produced for bigger number of columns. We have used about 5 columns in our own testing. Had to revert to using custom written MD5 hash stage because the built in checksum stage is appending '|' to the last input column. I wish that was not there but it is v...
- Thu Nov 08, 2018 12:08 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Custom built parallel stages
- Replies: 4
- Views: 4063
Hi gents, Many thanks for your opinions on this one. With IBM not being able to move away from the official stance of not providing typical support for custom written code (understandable) the explanations from senior experts helped in convincing powers to be that custom written C++ routines pose no...
- Wed Oct 31, 2018 1:09 am
- Forum: Information Analyzer (formerly ProfileStage)
- Topic: Column Analysis failing due to java heap space
- Replies: 5
- Views: 14528
Thanks a lot guys. We will almost definitely upgrade to 64-bit on Linux and hopefully within couple of months. This is the second time I am running IA on Windows and it is painful to say the least. Not just because of this failure, but the overall end user response times. Until then, and on advice f...
- Thu Oct 25, 2018 12:01 am
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Custom built parallel stages
- Replies: 4
- Views: 4063
Custom built parallel stages
Hi all, I have come across a custom built parallel stage within our environment that does a better job at MD5 hashing that the built-in Checksum stage because it does not use the trailing pipe character. The latter happens to be a problem for us because there are existing hashed values (produced by ...
- Sun Oct 07, 2018 7:34 pm
- Forum: Information Analyzer (formerly ProfileStage)
- Topic: Column Analysis failing due to java heap space
- Replies: 5
- Views: 14528
- Sun Oct 07, 2018 5:12 am
- Forum: Information Analyzer (formerly ProfileStage)
- Topic: Column Analysis failing due to java heap space
- Replies: 5
- Views: 14528
Column Analysis failing due to java heap space
Hi experts, Running on Windows system with 32 GB RAM, 8 CPUs and 4 node file. We have a flat file with 96 columns needing to be analyzed. When running on a sample of 20 records it runs incredibly slow, with only a few columns analyzed before eventually failing with " java.lang.OutOfMemoryError:...
- Thu Oct 04, 2018 3:03 am
- Forum: Information Analyzer (formerly ProfileStage)
- Topic: IA thin client does not see the flat file
- Replies: 0
- Views: 10836
IA thin client does not see the flat file
Hi All, Would somebody have an idea as to why is that the when attempting to run data quality rule from IA thin client, it is looking for file with the following pattern D:\SourceData.FileName2018Oct04 , instead of D:\SourceData \ FileName2018Oct04 So, it is separating the directory from a file name...
- Wed Aug 03, 2016 3:45 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Order of stage variables fails a job
- Replies: 4
- Views: 4008
This will very likely be due to buffering. Was trying to avoid changing too much of a job in production but prior to this there were two lookup stages linked one to each other. In dev and sit, they tell me it was never an issue, but it was never tested with large volumes. After changing from lookup ...
- Tue Aug 02, 2016 6:32 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Order of stage variables fails a job
- Replies: 4
- Views: 4008
Order of stage variables fails a job
Hi all, We have a parallel job that needs some improvements but is by no means complex. The last transformer in a job is preceeded by lookup stage that drops un-matched records. The transformer writes to dataset with some simple derivations. The job was running fine in non-prod environments, but sta...
- Wed Jun 01, 2016 8:57 am
- Forum: General
- Topic: SSL Certificates for Cross Project Compare
- Replies: 7
- Views: 10040
Hi Ray, Different project and still the same issue and I finally found a solution. Installing the certificate by following what used to be sufficient does not suffice anymore. So, our fix was: 1. Upon certificate warning click on 'View Certficate' button 2. Click on 'Certification Path' tab 3. You w...
- Tue Apr 19, 2016 8:29 am
- Forum: General
- Topic: SSL Certificates for Cross Project Compare
- Replies: 7
- Views: 10040