Web Service Client stage - issue with xpath definition
Web Service Client stage - issue with xpath definition
I don't see anything that I can directly relate to this problem I'm encountering when using a Web Service Client stage - so here goes -
I'm trying to call a web service from a vendor, who supplied the wsdl definitions for the various services we need to call.
The wsdl for this particular service is only two nodes - one gets loaded with an xml payload, and the other with a string.
After I've imported the wsdl, the xpath definition for the second node type is text(). But the first node has a type of "*". Of course if you run the job with that value, it results in an "invalid token" error.
The only thing I've been able to get to work is to set the value to text(), like the second node. However, with that, DataStage interprets the xml payload with ;gt and ;lt, instead of left and right brackets. The end result is that the web service call fails.
I know that this is the cause of the failure, because if I run a trace on the job, capture the payload, convert the gt and lt values to left and right brackets, and run it in SoapUI, the call works perfectly.
(On the other hand, if I capture the data as is, and run it in SoapUI, I get the same error that I get in DataStage, which is "Object reference not set to an instance of an object".)
Has anyone run into something similar when call web services?
Thanks!
Tom
I'm trying to call a web service from a vendor, who supplied the wsdl definitions for the various services we need to call.
The wsdl for this particular service is only two nodes - one gets loaded with an xml payload, and the other with a string.
After I've imported the wsdl, the xpath definition for the second node type is text(). But the first node has a type of "*". Of course if you run the job with that value, it results in an "invalid token" error.
The only thing I've been able to get to work is to set the value to text(), like the second node. However, with that, DataStage interprets the xml payload with ;gt and ;lt, instead of left and right brackets. The end result is that the web service call fails.
I know that this is the cause of the failure, because if I run a trace on the job, capture the payload, convert the gt and lt values to left and right brackets, and run it in SoapUI, the call works perfectly.
(On the other hand, if I capture the data as is, and run it in SoapUI, I get the same error that I get in DataStage, which is "Object reference not set to an instance of an object".)
Has anyone run into something similar when call web services?
Thanks!
Tom
I guess one question for starters is.....does it ever work in a soap testing tool?
...on the & escape characters, it's hard to say what is going on...but "maybe" the xml data element will help you...this tells DataStage to ignore whatever xml character symbols may be in that string and to just leave it "as is". Not sure if that has anything to do with your problem, but it is something to try. Set it on the column level...
Ernie
...on the & escape characters, it's hard to say what is going on...but "maybe" the xml data element will help you...this tells DataStage to ignore whatever xml character symbols may be in that string and to just leave it "as is". Not sure if that has anything to do with your problem, but it is something to try. Set it on the column level...
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Hi Ernie - Thanks for your reply.
To answer your question - yep, it works perfectly in SoapUI, as long as I manually convert the < and %gt; to brackets (<>) first. (If I paste the soap envelope as is into SoapUI, I get the same error that DataStage produces.)
I had also tried setting the Data Element value to XML at the column level, but so far that has produced the same results.
To answer your question - yep, it works perfectly in SoapUI, as long as I manually convert the < and %gt; to brackets (<>) first. (If I paste the soap envelope as is into SoapUI, I get the same error that DataStage produces.)
I had also tried setting the Data Element value to XML at the column level, but so far that has produced the same results.
-
- Premium Member
- Posts: 425
- Joined: Sat Nov 19, 2005 9:26 am
- Location: New York City
- Contact:
Hi Tom,
Recently I ran into a similar issue. In our case the web service payload was defined to generate the payload with a node string and that caused that the returned XML payload was xml escaped ( < --< and > -->). This depend entirely on the web service provider...
We ended up downloading the payload to disk and performing a global replace to fix the XML tags...and continue processing the XML
It looks like you are facing a similar web service weakly typed defined with anode string which always by definition will encode the XML payload
Good luck!!
Recently I ran into a similar issue. In our case the web service payload was defined to generate the payload with a node string and that caused that the returned XML payload was xml escaped ( < --< and > -->). This depend entirely on the web service provider...
We ended up downloading the payload to disk and performing a global replace to fix the XML tags...and continue processing the XML
It looks like you are facing a similar web service weakly typed defined with anode string which always by definition will encode the XML payload
Good luck!!
Julio Rodriguez
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
You can provide the input payload yourself...use the WSTransformer Stage instead of the WS Client Stage, and have a dummy Transformer or RowGen in front of it (or dummy read from a file/database...whatever is most convenient).....
Put "the payload that works" into a Derivation for a longvarchar column (only column on the link that enters the WSTransformer). Then...in the Input link "message" tab, use the pull-down for "User defined message". Select your single column.
Now do the same thing on the output side. Declare one large column, longvarchar, and specify it in the user defined message pulldown.
You are in control. The communications via HTTP is still done by the stage and established when you imported the wsdl and selected the operation, but the payload is being constructed by you.
Get that working perfectly....
THEN.....you can work on upstream to build the correct payload, using a combination of xmlOutput Stage and Transformer to clean anything up if necessary (hint: the _IN Table definition that gets created when you import the WSDL is the one that you use on the input link side of the xmlOutput Stage.
One question... You noted above "If I paste the soap envelope as is into SoapUI" .....where did you get this "soap envelope" if not from SOAPui?
Ernie
Put "the payload that works" into a Derivation for a longvarchar column (only column on the link that enters the WSTransformer). Then...in the Input link "message" tab, use the pull-down for "User defined message". Select your single column.
Now do the same thing on the output side. Declare one large column, longvarchar, and specify it in the user defined message pulldown.
You are in control. The communications via HTTP is still done by the stage and established when you imported the wsdl and selected the operation, but the payload is being constructed by you.
Get that working perfectly....
THEN.....you can work on upstream to build the correct payload, using a combination of xmlOutput Stage and Transformer to clean anything up if necessary (hint: the _IN Table definition that gets created when you import the WSDL is the one that you use on the input link side of the xmlOutput Stage.
One question... You noted above "If I paste the soap envelope as is into SoapUI" .....where did you get this "soap envelope" if not from SOAPui?
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Either way, the solution I outlined above will do the trick. Manage the payload entirely on your own using the user-defined message methodology......
...and then work separately on ways to build that payload on the way in or to parse it on the way out, with or without xml stages, depending on the payload complexity.
Ernie
...and then work separately on ways to build that payload on the way in or to parse it on the way out, with or without xml stages, depending on the payload complexity.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Thanks Ernie.
Couple of questions just to clarify -
-When you mentioned adding the 'working' payload to a derivation (inside a dummy transformer I'm assuming), do you mean the entire soa envelope content? I'm assuming yes.
-Also, you mentioned the input and xml output stage. You also mentioned having an output from that stage. What is the target for that output?
Thanks again.
Tom
Couple of questions just to clarify -
-When you mentioned adding the 'working' payload to a derivation (inside a dummy transformer I'm assuming), do you mean the entire soa envelope content? I'm assuming yes.
-Also, you mentioned the input and xml output stage. You also mentioned having an output from that stage. What is the target for that output?
Thanks again.
Tom
I haven't done it in awhile, but if memory serves correctly, you just need the whole <body>. A decade or so ago I asked engineering to give us a default Table definition for this purpose...if you were ever curious why there is a "BuiltIn" Table Def called SOAPBody...well...that's why.
The trick of copying the whole thing that works into a Derivation is just to "test it". Meaning --- give a perfectly working body into the Stage and then let the Stage do the HTTP work.
If that works, then you can evaluate..."how do I build this soap payload manually"? ...that's where eventually you would replace the upstream Transformer with an xmlOutput Stage (or the xml Stage if you have been using it) or whatever else you need to "build" that exact soap content....
...in the end, you might have a Job that contains ....various source stages.....an xmlOutput Stage....a Transformer (for final adjustments)...the WSTransformer...another Transformer....an xmlInputStage....other downstream stages...
Ernie
The trick of copying the whole thing that works into a Derivation is just to "test it". Meaning --- give a perfectly working body into the Stage and then let the Stage do the HTTP work.
If that works, then you can evaluate..."how do I build this soap payload manually"? ...that's where eventually you would replace the upstream Transformer with an xmlOutput Stage (or the xml Stage if you have been using it) or whatever else you need to "build" that exact soap content....
...in the end, you might have a Job that contains ....various source stages.....an xmlOutput Stage....a Transformer (for final adjustments)...the WSTransformer...another Transformer....an xmlInputStage....other downstream stages...
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>