Installing Information Server 11.3.1 - surprises

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

That's a bad decision from IBM as the views are not really needed unless you want to use the Cognos Governance reports. So the views shouldn't be a mandatory part of the installation and they certainly shouldn't have hard coded schema names. They may have enforced the hard coded names in order to get the Cognos reports to work - so that in Cognos you just point to the right data source and you don't need to open up Framework Manager to change schema names.
JRodriguez
Premium Member
Premium Member
Posts: 425
Joined: Sat Nov 19, 2005 9:26 am
Location: New York City
Contact:

Post by JRodriguez »

ASorrell,

A note about #1. When installing 11.3.1 in a existing WAS shared environment allowing you to install without disabling security is an advantage. You don't want to disabled global security in a complex WAS shared environment, specially in production. WAS admins usually makes a strong case against disabling security even on weekends

There are other surprises that you might encounter if the IGC modules and the services tier will be hosted in a WAS dynamic cluster
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Another annoyance (to me, anyway) are the questions asked during setup about "creating" other databases, such as SRD, exceptions.

In fact the installer does not create them - it only registers them and generates the scripts that can be used to create them. To my mind this is "misleading advertising", even though it's accurately portrayed in the Planning, Installation and Configuration Guide. But who reads manuals?!
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Just out of curiosity, what is RLVIEWS ?

I'm currently installing 11.3.1 (again) and only the other three are created.

Further, there's no RLVIEWS findable in the Knowledge Center.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

There is no RLVIEWS in my 11.3.1 XMETA database (yet?)... My repository uses the bundled DB2. I also did not find any such references in any *.sql file from the install media. There are, however, plenty of SQL comments "-- DataStage operations database: create reference/lookup tables & views" so perhaps RL is short for reference/lookup.
Choose a job you love, and you will never have to work a day in your life. - Confucius
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

There's also REMVIEWS and CEFVIEWS for the Information Governance Dashboard, in the IGD schema.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

A couple of nice features.

To reduce I/O operations for large sorts, the Sort stage converts bounded length fields to variable length before the sort, and converts them back to bounded length after the sort. Sort optimization also improves run time during which implicit sorts are sometimes required, for example, in InfoSphere QualityStage matching and join steps.
By default the tsort operator optimizes bounded length fields by converting them to variable length before the sort and converts them back to bounded length after the sort. For records with many bounded fields where the actual size of the data is much smaller than the upper bound the optimization results in a large reduction of disk I/O.
For records that do not have large differences between data size and upper bound of bounded length fields, this optimization often causes slower sort performance. Set the APT_TSORT_NO_OPTIMIZE_BOUNDED environment variable to prevent the optimization of bounded length fields by the tsort operator.
Setting the APT_TSORT_NO_OPTIMIZE_BOUNDED environment variable when APT_OLD_BOUNDED_LENGTH is set has no effect as APT_OLD_BOUNDED_LENGTH also disables the bounded length optimization.

The Sequential File stage and associated import operator can now import records that do not contain all the fields in the import schema.
Set the APT_IMPORT_HANDLE_SHORT environment variable so that the import operator successfully imports records that do not contain all the fields in the import schema.
By default, records that do not contain all the fields in the import schema are rejected by the import operator. Set the APT_IMPORT_HANDLE_SHORT environment variable so that these records are imported successfully. The missing fields are given the default value for their type, or null if the field is nullable.
Setting the APT_IMPORT_HANDLE_SHORT environment variable disables optimization for fixed length schemas by the copy operator, which might cause slower performance for some fixed-length schemas.
The APT_IMPORT_HANDLE_SHORT environment variable is not supported for importing fixed-length records without a record limiter, as incorrect data might be imported.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Just catching up on this post.

JRodriguez - I didn't say the security setting was bad - just not documented as a requirement, which made it a surprise.

Also - I got the name RLVIEWS from the installer code when I looked into the Java app. It was checking to insure that nothing in the database was owned by those schemas (ie: unused). It may be something that didn't get implemented, but someone forgot to pull it from the installer.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Now there are two topics about the APT_IMPORT_HANDLE_SHORT environment variable. :)

That's great news about the sort optimizations.

Also on the topic of 11.3.x surprises (possibly 9.x too): If you go with the IBM XLC C/C++ compiler (at least this is true on AIX), beware of installing the compiler version 13.1. It currently has issues and must be backed out in favor of the 12.1 compiler version. Confirmed via PMR and testing...

They plan to correct the system requirements web site next week. It should not imply that compiler 12.1 and higher works OK. A few other compiler corrections planned to that site are to change "C" to "C/C++" and to remove g++ except for Linux. It currently lists g++ for various UNIX flavors but is not supported.
Choose a job you love, and you will never have to work a day in your life. - Confucius
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

I found another surprise last week, now resolved (I learned something new today!). Actually this would have been a surprise starting on version 9.1.2 as well.

See topic: DataStage 11.3.1 and umask 007 command
Choose a job you love, and you will never have to work a day in your life. - Confucius
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Another surprise for me today (may exist in previous versions).
orchadmin check objects to comments in the beginning of the configuration file.

Code: Select all

$ bin/orchadmin check
##E IIS-DSEE-TFRM-00003 While parsing ORCHESTRATE config file, near line 1: Expecting open brace at beginning of config file, instead found "#"
##F IIS-DSEE-TFCF-00014 15:11:48(001) <main_program> Fatal Error: The configuration does not contain the invoking node bfssydapd541.
:(
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

The 8.7 px job guide says that config file comments need to be in C style /* ..... */ format. I've not tested comments to know what really works, if any.
Choose a job you love, and you will never have to work a day in your life. - Confucius
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

I saw an IBM bulletin yesterday that indicated that Fix Pack 1 for 11.3.1 (= version 11.3.1.1) results in a problem in the Operations Console (JR52899). Here is a link.

https://www-304.ibm.com/support/entdocv ... wg1JR52899
Choose a job you love, and you will never have to work a day in your life. - Confucius
rameshrr3
Premium Member
Premium Member
Posts: 609
Joined: Mon May 10, 2004 3:32 am
Location: BRENTWOOD, TN

Post by rameshrr3 »

and to remove g++ except for Linux. It currently lists g++ for various UNIX flavors but is not supported.
This is misleading documentation, because the only time the compiler is invoked is when a (parallel) transformer or buildop stage are needed in the job design- which happens to be majority of times. The unix shared objects for the parallel transfomer ( Stored in $APT_ORCHHOME/lib ) are usually compiled on the native optimizing compiler for that particlar platform, and other than Linux, g++ is NOT the native optimizing compiler, and it could use different conventions for symbol naming etc and gcc generated objects or libraries will be incompatible with the trasnformer stage shared libs outside of linux.

You can get the parallel transfomer libraries by compiling a job with a parallel transfomer stage , going to the RT_SCnnn subdirectory and based on the *.trx file name , go to the correctsponding RT_BPnnn.O subdirectory , and running an ldd command on the generated Object (*.o) files and see which of them reside in the default $APT_ORCHHOME/lib path. Also the documentation misleadingly claims that static libararies can store parallel routine object files, but the way a compiler is invoked at compile time, it can only get routines from static objects or shared libraries - I havent found a reliable way of compiling a parallel transformer stage that calls a routine stored in a static library- even if i use various linker and compile flags-even though the transformer object code depends on a few standard static c++ libraries ( libC.a etc ) If anyone has had luck in this area on either AIX or Linux - please let me know.

I was thinking the mandate on using shared libraries or static objects for external routines is so that parallel execution is possible as number of nodes is not predictable in advance.
viewtopic.php?p=102127

I know a developer who mostly uses the Basic Transformer because of his bias towards the better Function library , but you dont get 7 key 'features' in the Basic Transformer. And on a grid environment the Basic transformer can become a bottleneck as it can only run on a conductor node.
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

The g++ reference was mistakenly misleading, and they did correct it on the system requirements site soon after we pointed it out.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Post Reply