The following topics review the different configuration steps:
- Data import batch integration best practices
These best practices ensure that data import integrations are configured consistently throughout Nextworld.
- Integration table mapping for data imports
This topic reviews the common fields that must be mapped in data import definitions for integration tables.
- Data import integration table configuration
Application developers and system administrators create integration tables in order to process and format external data before using it in internal tables and applications.
- Data import integration application configuration
Application developers and system administrators create integration applications over an integration tables. Business users then use the integration application to transfer external business data into Nextworld tables.
- Integration logic blocks
When processing records as part of an integration there will likely be one or more logic blocks which will process most of the business logic required to successfully import. These logic blocks fall into one of two types and how they are used strictly depends on where in the import flow the logic needs to be run. The two types are implementation logic blocks and process logic blocks.
Integration table mapping for data imports
This topic reviews the common fields that must be mapped in data import definitions for integration tables.
Standard integration table field mapping
Configure the following data items for standard integration tables:
| Data item | Notes |
|---|---|
IntegrationStatus | Default to value of |
nwExternalId | Key to external system, used for external purposes only. |
Header detail integration table field mapping
Configure the following data items for header detail integration tables:
| Data item | Notes |
|---|---|
IntegrationStatus | Default to value of |
GroupLevel1 (required) | A way to group or sort data. Use |
IntegrationAction | Use |
IntegrationDetailAction | Use |
nwExternalId | Key to external system, use for external purposes only. Use |
nwExternalDetailId (optional) | Key to external system, use for external purposes only. Use |
Historical (optional) | During migrations, historical data must be imported into the data table. Since this usually applies to an entire batch of data and not individual records, the historical indicator does not need to be a column in the CSV file. Instead the Historical field can be set to a constant |
Data import integration application configuration
Application developers and system administrators create integration applications over an integration tables. Business users then use the integration application to transfer external business data into Nextworld tables.
Create an application and use the same name as the integration table it is built over For example, the BillsOfMaterialIntegration application is built over the BillsOfMaterialIntegration table.
Select type Advanced List as your Application Type.
Field placement
The following fields must be included in the advanced list header:
nwImportRunNumberIntegrationStatusPurgeProcessed
Sort Profile
For integrations over a header detail table, on the Sort Profile tab in application builder, you should create a sort with the following definition:
nwImportRunNumber- DecendingGroupLevel1- AscendingnwImportRowNumber- Ascending
For integrations over a standard table, on the Sort Profile tab in application builder, you should create a sort with the following definition. Depending upon your reason for having an integration for a standard table, your application sort could vary or be different than below.
nwImportRunNumber- DecendingnwImportRowNumber- Ascending
Related objects
When you create a menu entry point for your integration application, give it the same name as the application without the upper camel case naming convention. For example, the BillsOfMaterialIntegration application becomes Bills Of Material Integration in the menu.
If you create an application settings record for the integration application, use the following naming convention: <table name>Settings. For example, the application settings name for the BillsOfMaterialIntegration application is BillsOfMaterialIntegrationSettings.
General configuration
Configure the following on the Actions page in the Application Builder application.
Application Links
All Header Detail and Standard integration applications should include an application link to the IntegrationsErrorLog mini app. The purpose of this mini app is to allow the user to track every integration table record that does not successfully import to the destination table. Both the IntegrationsErrorLog mini app and its associated application setting, IntegrationsErrorLogSettings, are established within Nextworld for re-use. Configure the following application link:
| Link Name | Link Destination | Application Setting Name | Form |
|---|---|---|---|
ErrorLog | IntegrationsErrorLog | IntegrationsErrorLogSettings | List |
Data mappings for Header Detail and Standard integration applications should include nwImportRunNumber as a filter field for displaying errors. Header Detail integration applications should also include GroupLevel1 as a filter field.
Row Actions
After you create the ErrorLog application link, you must add it to the Row Actions subtable for both Header Detail and Standard integration applications. Configure the following row action:
| Link Name | Display Name | Link Type |
|---|---|---|
ErrorLog | View Errors | Application (Save Not Required) |
If you are configuring a Header Detail integration application. Add a row action to reset records that have the same group number from an Error status to a New status. Configure the following row action:
| Logic Block | Display Name | Link Type |
|---|---|---|
Reset<hd table name>IntegrationStatus | Reset Group Status | Logic Block In Trans Boundary |
List Form Actions
All Header Detail and Standard integration applications should include list form actions so that the user can use logic blocks to process their integrations. Configure the following form actions:
| Logic Block | Display Name | Link Type |
|---|---|---|
Integrate<table name> | Process Run | Logic Block In Trans Boundary |
Reset<table name>IntegrationStatus | Reset Run Status | Logic Block In Trans Boundary |
Purge<table name>Integration | Purge Run | Logic Block In Trans Boundary |
Learn more about integration logic blocks in the Integration logic blocks topic.
Data import batch integration best practices
These best practices ensure that data import integrations are configured consistently throughout Nextworld.
External data CSV file
Use column headings consistent with Nextworld data items instead of using default column headings like A, B, C and so on. Using the default headings can lead to confusion and possible data mapping errors.
Data Import Definitions
If an attachment is still linked to the data import definition from a previous import, the data from the old attachment is processed alongside the new data. Delete any existing attachments before running a new import to avoid processing the same data unnecessarily.
Subtables
For importing subtables, create a separate integration table for each subtable. This keeps the .csv file more organized and concise, even though multiple import runs may be needed. For example, the Directory table contains multiple subtables. In order to integrate, Directory has an integration table, DirectoryIntegration, along with the integration tables for the subtables, such as DirectoryEmailsIntegration and DirectorySocialMediaIntegration for the Emails and Social Media subtables.
Managed and Direct integrations
An integration table is always necessary to import Header Detail tables, and is recommended for Standard tables with large volume integrations and business critical data with dependencies. In the case of importing a Main table without large volume integrations and business critical data with many dependencies, use the data import tool to import directly into the table.
Use the nwImportRunNumber data item for both Header Detail and Main table integrations.
Use the nwImportRunNumber and GroupLevel1 data items to group data for Header Detail table integrations.
Logic Blocks
It is not recommended to include any Logic Blocks on the Data Import Definition itself. When certain conditions are met, data imports can leverage a "copy" command in the database instead of importing rows line by line. Using a logic block on the Data Import Definition will disable that ability.
You may have an integration table built for a table that has one or both of the following:
- Action block
- Table trigger
If you do, then you must ensure that the Set Message action contains a populated Data Source Name field. If you do not specify a Data Source Name, then import errors that occur during integration may not get written to the correct integration record.
Workflow
If the table being integrated to has an workflow trigger, do not rely on this transition during integration. Instead, have a logic block use the Queue Workflow State Change action to queue a workflow trigger to move the records to the next state automatically. We do not want to flood the workflow queue. This could prevent more important work from happening and create a backlog that could potentially take days to clear if the amount of data being integrated is large enough.
Common Fields
If the table being integrated to has common fields, the integration may perform better for extremely large data sets if those common fields are removed from the target table and are instead populated through logic.
Integration logic blocks
Add progress and summary message types to the Integration<table name> logic block so the user can get progress and summary updates about each run.
Add the PurgeProcessed data item to the integration table as a work field. Configure the Integrate<table name> logic block so that when the Purge Processed check box is selected, all successfully processed transactions within a run number are deleted.
Add a View Errors action to the IntegrationsErrorLog mini app to connect the DeleteIntegrationsErrorLogMessages logic block to it so that displays all errors of a given run number in the mini app.
A purge logic block allows the deletion of records by run number. A reset logic block tallows for a status reset of records from ERROR to NEW by run number. Learn more in: