A data transform usually runs against only one table for each data transformation. Data transforms execute without any of the typical integrity checks that protect data, so use data transforms sparingly. Data transforms do not enforce security and they do not execute any validations or table triggers. If you can change your design or use a logic block to accommodate the changes to your table, you should consider those options before creating a data transform.
A data transform can be used in a development environment to modify data and can be included in a release to modify production data.
Some use cases where data transforms update data in a table include:
- A field in the table needs to be populated or the programming logic won't run correctly.
- A field in the table may need a specific value which not all the records in the table currently display, so the specific value needs to be entered into all the records in the table.
- A table has been split into two different tables, and data needs to be moved from one table to the other table.
To execute a data transform application developers create a data transform type logic block and then configure and test the logic block in the Data Transform Definitions application.
For more information see Data Transform Definitions application, Data transform logic blocks and Temporary tenants.
Data transforms development and deployment process
Application developers use a three-stage process for data transformations—develop, test, and deploy.
Here are the stages in the data transform process:
Process | Description |
| Develop and test | An application developer initiates the following steps for a data transformation:
|
| Deploy to a development environment | Deploy the data transform in your development environment. See Data Transform Execution application for more information. |
| Deploy to production environment (Solution Developers and Nextworld base only) | Data transforms move downstream like other metadata objects. Nextworld performs the following actions as part of a new release:
|
Data transform configuration for system administrators
System administrators configure data transform executions in the Data Transform Execution application.
In the Data Transform Execution application, you must:
- Name the execution plan.
- Add all the data transforms to be executed in the execution plan.
- Use the Environments field to specify all the environments the data transforms are designated to execute in.
There are optional configurations, such as:
- To run the data transforms without committing any changes to the database, select the Draft Mode checkbox. This allows you to see if there are any errors in the transformed records.
- To include all direct and indirect dependencies in the execution plan, select the Include Dependencies checkbox. To make it optional to include dependencies in the execution plan, do not select the Include Dependencies checkbox. Only dependencies listed in the execution plan run, and they run in dependent order.
For more information see Data Transform Execution application.
Data Transform Execution application
System administrators use the Data Transform Execution application to execute one or more data transforms.
There are several considerations for configuration, such as:
- To verify that a data transform can complete without errors prior to permanently altering the data, select the Draft Mode checkbox to run the data transforms without committing any changes to the database.
- If all direct and indirect dependencies must be included in the execution plan, select the Include Dependencies checkbox. To make it optional to include dependencies in the execution plan, do not select the Include Dependencies. Dependencies listed in the execution plan run in dependent order.
- During execution, data transforms execute in all environments listed in the Environments field.
Additionally, when a data transform runs to update a business record it changes the Modified Date and Modified By User values of the record. The Modified By User value of the record uses the system user credential.
- If the data transform creates a new record, it doesn’t keep the original Created By User or Created Date values from the original business record.
- If a data transform inserts a record, that record has the system user credential for Created By User and the current date time for Created Date field.
For more information see Data Transform Definitions application.
Data transform best practices
These best practices ensure that data transforms are configured consistently throughout Nextworld.
Use of data transforms should be identified as early in the development process as possible, ideally during the design phase.
Data transforms should run quickly and repeatedly without re-processing all the records again in order to limit environment downtime.
A data transform should be responsible for only updating one table. If one table is dependent on the success of another table’s data transform, this can be handled by configuring the transform’s dependency on the Data Transform Record.
Use a Set Message of type Data Transform Message to communicate critical and summary information only.
Data transform logic block best practice
Bulk Update is the preferred method of transformation, as it grabs all the records that match the filter criteria and updates them with the fields set in the Field Mappings.
There should only be one logic block for a table. If there is an existing data transform for the table you need to convert, add to that logic block instead of creating another one.
Data transform logic blocks cannot be built over Header/Detail tables. If you want to transform a Header/Detail table, build an individual transform for the header, and one for each detail related to that table.