Please use docs.servicenow.com for the latest documentation.

This site is for reference purposes only and may not be accurate for the latest ServiceNow version

Import Sets Performance Improvements

From Wiki Archive
Jump to: navigation, search
{{#if: }}
See Also

Winter 2009 Stable 1 introduces changes to the algorithm used to transform import sets from their staging table into their final destination. The new algorithm is about 60% faster in our tests and requires only about half as many round trips to the database.

Importing data via an import set requires two phases:

  1. The data is loaded from a data source into a staging table
  2. The data is transformed from the staging table into a target table


For the Winter 2009 Stable 1 build (and major releases starting with Spring 2009), the transform algorithm has been changed to operate in "blocks" of 100 records at a time as opposed to the previous algorithm which transformed a single record at a time. The newer approach allows the application server to pre-fetch a variety of information relevant to each block of records, reducing the number of unique interactions with the database and improving throughput.


Who should expect to see a performance improvement?

Any customer using import sets should expect to see a performance improvement in large transformations.

What kinds of transformations will benefit the most from these changes?

Transformations with a large number of reference or choice type columns will see the largest improvement.

What kind of transforations will benefit the least?

Transformations that with complex or unkeyed coalesce conditions will see a proportionately smaller benefit.