Page 1 of 1
Error when calling REST operation that calls a Datastage Job
Posted: Fri Dec 18, 2015 5:19 am
by pbttbis
Hi,
I have created a job with both input and output ISD stages. Job compiles fine (enable for webservices and mutliple instances options set).
In Information Server Console I create a new application, service and attach an operation to the job. The binding I am using is REST with the following settings:
Format: XML
HTTP Action: POST
Parameter Type: BODY
I tried the following two basic HTML pages to test:
--version 1
<title>REST with Forms</title>
<form action="
https://devetleng:9443/wisd-rest2/DataS ... aStageCall" method="post">
<input type="text" name="data" value="mydata" />
<input type="submit" />
</form>
--version 2
<html>
<body>
<form action="
https://devetleng:9443/wisd-rest2/DataS ... aStageCall" method="post">
<textarea rows="20" cols="100" name=" ">
<?xml version="1.0"?><requestElements><blah></blah></requestElements>
</textarea>
<input type="submit" value="Submit">
</form>
</body>
</html>
I am getting the following error: org.xml.sax.SAXParseException: Premature end of file.
Any ideas?
Thanks
Posted: Fri Dec 18, 2015 6:24 am
by eostic
Not sure exactly what is happening, but start with something vastly more primitive and simple in your first testing.
A good one to try is perhaps a simple Job that takes a single input argument and returns a single value in the response. One column on the output link from the ISDinput Stage and one column on the input link to the ISDoutput Stage. Do something to it in a transformer...... like "upcase" of a string.
Use GET. This way you can test it from a basic browser. Keep your xml type for the payloads.
Send the simple url, with your single argument...
http://server:port/wisd-REST/Application/Service/Operation?Arg=value
...get that working, with xml returned to your browser, before trying anything more complicated.
Why do you need POST anyway? ISD is best for stateless, non-i/o services requests.
Ernie
Posted: Fri Dec 18, 2015 7:27 am
by pbttbis
Hi Ernie,
My colleague Arnd says you know this stuff best. I believe you two have met on at least one occasion?
Yeah the job I am testing is a simple one like you suggest.
Testing it with GET it is successful.
Currently we have a webpage that interacts with a system via a POST. We are trying to insert datastage in between. Thus the POST should go to DataStage and is why I am trying to get it to work with a POST.
The requesting system's post is prepared in the format of version 2 in the two simple XML post examples I showed earlier.
Thanks,
Shaun
Posted: Fri Dec 18, 2015 3:19 pm
by eostic
Yeah...hard to say. I recall looking at that aspect of this binding a long time ago when it first came out, but haven't touched that aspect (POST with xml body) of the binding in years... I would probably punt on something like this and try working with the Text over HTTP binding instead. Let it just "pass thru" and take care of all the formatting for input and output on the DataStage side. Too hard to get it to conform otherwise. Doing it with Text over HTTP lets you more easily integrate with "whatever else" the client wants to send you....and deal with it inside of DataStage. Basically, the Text over HTTP binding is like putting "blindfolds" on ISD.......
Ernie
Posted: Mon Dec 21, 2015 7:27 am
by pbttbis
Thanks for the help so far Ernie.
I am now trying to get my ISD job to work with the binding "Text Over HTTP".
The HTTP data unit that looks like it should work with the client posting to the service I have selected is HTTP PAYLOAD.
https://www-01.ibm.com/support/knowledg ... ml?lang=en
I get a a blank page in the browser when submitting the post with my test pages. If i do a view source on the blank page Chrome shows me a ERR_CACHE_MISS
See nothing in the job log in DataStage Director. Nothing comes up in the SystemOUT.log and SystemErr.log files.
This tell me that my request is not been received by the job and that the problem lies in the binding configuration of the service.
Any ideas on how to troubleshoot what/where it is going wrong?
Posted: Mon Dec 21, 2015 8:17 am
by eostic
Mapping HTTP PAyload to a single column in the Job is the right approach.
I would have a simple ISD Job that has just one big column...nothing else. varchar and a long length, or maybe even char. send it to a transformer and then to a sequeential file. For a quick test, I would probably initially skip the isdoutput.
Just see if you can capture the payload correctly as an initial test. use "NONE and "NONE" on the sqeuential stage for quote character and delimiter.
Try a Server Job too, if that fails, just to check that something odd isn't going on in the plumbing.
Ernie
Posted: Mon Dec 21, 2015 10:29 am
by pbttbis
Parallel job forces you to have one (and only one) ISD Output Stage with an ISD Input Stage.
Server job allows to have just a ISD Input Stage but am still getting the same blank page and no processing in the job log.
Checking the plumbing in SystemErr.log, SystemOut.log and native_stderr.log. Nothing in there. Are there any specific log files I can check for application services?
Port 9443 the correct port to use with http://servername:port/wisd-txhttp/<application-name>/<service-name> ?
Posted: Thu Jan 07, 2016 3:35 am
by pbttbis
Marking this thread as resolved.
Ended up making use of a REST binding with Input and Output pass through handling all the logic in the job.
Great tool I ended up using for testing was the rest client from WizTools.org.
Learnt the hard way that having a passive stage with both an input and output within a job with both input and output ISD stage causes issues.
Posted: Thu Jan 07, 2016 6:27 am
by eostic
Ouch. Yes. It's too bad that the compiler doesn't check that for you...... : /
I recall learning that many years ago (not to do it in a Server ISD Job), as it is certainly a common technique for regular Server Jobs. It has something to do with the automatic serialization of the processes that occur when you do that....the Server engine automatically sequences it so that the input link to the passive stage must finish before the output link even gets started --- and in a real-time ISD Job, that creates a conflict.
Ernie
Posted: Thu Jan 07, 2016 3:38 pm
by ray.wurlod
Would it work if you enabled Inter Process Row Buffering in the server job? Just a thought. I still believe that blocking operations are incompatible with ISD.