How to reduce load times with SFDC Pack

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Developer9
Premium Member
Premium Member
Posts: 187
Joined: Thu Apr 14, 2011 5:10 pm

How to reduce load times with SFDC Pack

Post by Developer9 »

Hi Everyone,

We are currently using Data loader script for loading into Salesforce (Upsert)and script takes an average of 5 minutes to finish the upsert operation of 10 objects .
We are replacing the Data loader scripts with SF pack load job for each object loading ,each job taking an average elapsed time of 1:20 (min:sec)..so overall for object it taking around 25 minutes

Here our job schedule runs are daily deltas with upsert operation (below log shows it has only 1 record with 6 fields for the operation)and rest of the objects also have about few hundred records only.

Code: Select all

5/29/2018 11:13:35 AM job_sf,0: Data batch size is set to: 10,000
--WHY THERE IS A DELAY HERE----
5/29/2018 11:14:11 AM job_sf,0: Information: Found a parent field Agency__r_ExternalID__c
Information: Found a parent field Principal_Number__r_ExternalID__c
Information: Wait for: 30 seconds
5/29/2018 11:14:11 AM bond_allocation_sf,0: Loading finished. 1 rows are loaded into Salesforce.com
5/29/2018 11:14:11 AM bond_allocation_sf,0: Loading finished. 0 rows are rejected
5/29/2018 11:14:12 AM main_program: Step execution finished with status = OK.

5/29/2018 11:14:20 AM main_program: Startup time, 0:22; production run time, 0:46.
After observing the startup time and production run time are there any environment variables to reduce the job elapsed time less than minute or 30 sec?or this is the usual elapsed time for sfpack jobs for this record count?

Let me know any suggestions to reduce the job run time from current 25 minutes to lesser ?

Thanks.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

:!: New question = new post. FYI - I split this to its own discussion topic rather than leave it at the end of your previous SFDC post.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Developer9
Premium Member
Premium Member
Posts: 187
Joined: Thu Apr 14, 2011 5:10 pm

Post by Developer9 »

Thank you craig :)
Post Reply