The following topics review the different configuration steps:

  • Data import batch integration best practices
  • Integration table mapping for data imports
  • Data import integration table configuration
  • Data import integration application configuration
  • Integration logic blocks
 

Integration table mapping for data imports

This topic reviews the common fields that must be mapped in data import definitions for integration tables.

Standard integration table field mapping

Configure the following data items for standard integration tables:

Data itemNotes
IntegrationStatus

Default to value of NEW.

nwExternalId

Key to external system, used for external purposes only.

Header detail integration table field mapping

Configure the following data items for header detail integration tables:

Data itemNotes
IntegrationStatus

Default to value of NEW.

GroupLevel1 (required)

A way to group or sort data. Use Header type for this output.

IntegrationAction

Use Header type for this output.

IntegrationDetailAction

Use Header type for this output.

nwExternalId

Key to external system, use for external purposes only. Use Header type for this output.

nwExternalDetailId (optional)

Key to external system, use for external purposes only. Use Header type for this output.

Historical (optional)

During migrations, historical data must be imported into the data table.

Since this usually applies to an entire batch of data and not individual records, the historical indicator does not need to be a column in the CSV file. Instead the Historical field can be set to a constant TRUE that is added to every record being imported to the integration table

Data import integration application configuration

Application developers and system administrators create integration applications over an integration tables. Business users then use the integration application to transfer external business data into Nextworld tables.

Create an application and use the same name as the integration table it is built over For example, the BillsOfMaterialIntegration application is built over the BillsOfMaterialIntegration table.

Select type Advanced List as your Application Type.

Field placement

The following fields must be included in the advanced list header:

  • nwImportRunNumber
  • IntegrationStatus
  • PurgeProcessed

Sort Profile

For integrations over a header detail table, on the Sort Profile tab in application builder, you should create a sort with the following definition:

  • nwImportRunNumber  - Decending
  • GroupLevel1  - Ascending
  • nwImportRowNumber  - Ascending

For integrations over a standard table, on the Sort Profile tab in application builder, you should create a sort with the following definition. Depending upon your reason for having an integration for a standard table, your application sort could vary or be different than below.

  • nwImportRunNumber  - Decending
  • nwImportRowNumber  - Ascending

Related objects

When you create a menu entry point for your integration application, give it the same name as the application without the upper camel case naming convention. For example, the BillsOfMaterialIntegration application becomes Bills Of Material Integration in the menu. 

If you create an application settings record for the integration application, use the following naming convention: <table name>Settings. For example, the application settings name for the BillsOfMaterialIntegration application is BillsOfMaterialIntegrationSettings.

General configuration

Configure the following on the Actions page in the Application Builder application.

Application Links

All Header Detail and Standard integration applications should include an application link to the IntegrationsErrorLog mini app. The purpose of this mini app is to allow the user to track every integration table record that does not successfully import to the destination table. Both the IntegrationsErrorLog mini app and its associated application setting, IntegrationsErrorLogSettings, are established within Nextworld for re-use. Configure the following application link:

Link NameLink Destination
Application Setting Name
Form
ErrorLogIntegrationsErrorLogIntegrationsErrorLogSettingsList

Data mappings for Header Detail and Standard integration applications should include nwImportRunNumber as a filter field for displaying errors. Header Detail integration applications should also include GroupLevel1 as a filter field.

Row Actions

After you create the ErrorLog application link, you must add it to the Row Actions subtable for both Header Detail and Standard integration applications. Configure the following row action:

Link Name
Display Name
Link Type
ErrorLogView ErrorsApplication (Save Not Required)

If you are configuring a Header Detail integration application. Add a row action to reset records that have the same group number from an Error status to a New status. Configure the following row action:

Logic Block
Display Name
Link Type
Reset<hd table name>IntegrationStatusReset Group StatusLogic Block In Trans Boundary

List Form Actions

All Header Detail and Standard integration applications should include list form actions so that the user can use logic blocks to process their integrations. Configure the following form actions:

Logic Block
Display NameLink Type
Integrate<table name>Process RunLogic Block In Trans Boundary
Reset<table name>IntegrationStatusReset Run StatusLogic Block In Trans Boundary
Purge<table name>IntegrationPurge RunLogic Block In Trans Boundary

Learn more about integration logic blocks in the Integration logic blocks topic.

Data import batch integration best practices

These best practices ensure that data import integrations are configured consistently throughout Nextworld.

External data CSV file

Use column headings consistent with Nextworld data items instead of using default column headings like A, B, C and so on. Using the default headings can lead to confusion and possible data mapping errors.

Data Import Definitions

If an attachment is still linked to the data import definition from a previous import, the data from the old attachment is processed alongside the new data. Delete any existing attachments before running a new import to avoid processing the same data unnecessarily.

Subtables

For importing subtables, create a separate integration table for each subtable. This keeps the .csv file more organized and concise, even though multiple import runs may be needed. For example, the Directory table contains multiple subtables. In order to integrate, Directory has an integration table, DirectoryIntegration, along with the integration tables for the subtables, such as DirectoryEmailsIntegration and DirectorySocialMediaIntegration for the Emails and Social Media subtables.

Managed and Direct integrations

An integration table is always necessary to import Header Detail tables, and is recommended for Standard tables with large volume integrations and business critical data with dependencies. In the case of importing a Main table without large volume integrations and business critical data with many dependencies, use the data import tool to import directly into the table.

Use the nwImportRunNumber data item for both Header Detail and Main table integrations.

Use the nwImportRunNumber and GroupLevel1 data items to group data for Header Detail table integrations.

Logic Blocks

It is not recommended to include any Logic Blocks on the Data Import Definition itself. When certain conditions are met, data imports can leverage a "copy" command in the database instead of importing rows line by line. Using a logic block on the Data Import Definition will disable that ability. 

You may have an integration table built for a table that has one or both of the following:

  • Action block
  • Table trigger

If you do, then you must ensure that the Set Message action contains a populated Data Source Name field. If you do not specify a Data Source Name, then import errors that occur during integration may not get written to the correct integration record.

Workflow

If the table being integrated to has an Effective Dateworkflow trigger, do not rely on this transition during integration. Instead, have a logic block use the Queue Workflow State Change action to queue a Logic Block Controlledworkflow trigger to move the records to the next state automatically. We do not want to flood the workflow queue. This could prevent more important work from happening and create a backlog that could potentially take days to clear if the amount of data being integrated is large enough. 

Common Fields

If the table being integrated to has common fields, the integration may perform better for extremely large data sets if those common fields are removed from the target table and are instead populated through logic.

Integration logic blocks

Add progress and summary message types to the Integration<table name> logic block so the user can get progress and summary updates about each run.

Add the PurgeProcessed data item to the integration table as a work field. Configure the Integrate<table name> logic block so that when the Purge Processed check box is selected, all successfully processed transactions within a run number are deleted.

Add a View Errors action to the IntegrationsErrorLog mini app to connect the DeleteIntegrationsErrorLogMessages logic block to it so that displays all errors of a given run number in the mini app.

A purge logic block allows the deletion of records by run number. A reset logic block tallows for a status reset of records from ERROR to NEW by run number. Learn more in: