Collect

Register Tables to Source

Tables are registered to sources so that when a target refresh is conducted, all active tables are pulled from the source and published to the target. Tables can be individually added or added as a group through the publish group feature.

Table names are the same for both source and target systems unless a table rename option is enabled. Transparent tables use an ODBC connection to download data. Pool tables use a RFC DLL to pull data from SAP tables.

There are two ways to register tables to a source:

  • Import a Group Table – Used to import System Type tables delivered with Common. This method facilitates the process of registering tables by importing content delivered with Common into Collect.
  • Manually – Used if the source is not a System Type delivered with Common.

A user can also Use the Manual Data Services Package Type.

Continue with Encrypt Columns.

Import Groups Tables

The import group table(s) process imports System Type group tables from Common into Collect to facilitate the process of adding sources. This process also imports the table’s indices and rules. The Fetch process builds this list or a user can download it from the Knowledge Tier.

To import group tables:

  1. Click Collect > Targets on Navigation pane.
  2. Click Sources icon for a target.
  3. Select a source.
  4. Click Import Group Tables on the Page toolbar.

    NOTE: The page automatically displays in Edit mode.

    View the field descriptions for the Group Import page

  5. Select a System Type group to import from Common as a source from System Type Group ID list box. Refer to Common documentation on System Types for detailed information.
  6. Select a schedule for when the target is refreshed with source data from Schedule ID list box, if the target should be refreshed on a schedule. Otherwise, the target is refreshed on-demand via the Build and Refresh icon on the Targets page. Refer to Common documentation on Create Schedules for detailed information on creating and modifying a schedule.
  7. Click Publish Schema Owner check box to enable it, which imports the table schema owner to the target and overrides the default Target Source schema owner.

    NOTE: Systems like Oracle EBS must have this option checked. Because SAP only has one schema owner for all tables, this options is not required for SAP.

  8. Click Publish Rules check box to enable it, which loads rules assigned to the tables in the group import to the rule configuration table to update the tables when processed.

    NOTE: If Publish Rules is unchecked, rules assigned to the tables do not load into the rule configuration table. In this case, there are no rules to process against the table. Once a rule is loaded into the table, there is no sync process between the Group Import and the current Target Source Table.

  9. Click Save.
  10. Click Import Table Groups to import all group tables, indices and rules into the Target Source; a confirmation message displays.
  11. Click Ok.

Manually Register Tables to Source

The unique ID for the target source table has been updated with version 7.4.3 to include the table rename.

Using Table Rename functionality, users can register multiple tables with large datasets (for example, BSEG) but with different Table Renames (for example Company Code). This allows users to extract data from target sources on a more granular level, pulling the correct data into tables named, for example, BSEG_Company One, BSEG_Company Two, and so on during the refresh.

NOTE: When extracting data for package types SAP RFC and BOA RFC, any trailing, empty spaces in columns are removed by default. This option is set on the Vertical View of the Tables page.

To manually register a table to a source:

  1. Click Targets in the Navigation pane.
  2. Click Sources for Target.
  3. Click Tables for Source.
  4. Click Add.

    View the field descriptions for the Tables page

  5. Enter a name in TABLE field.
  6. Update PACKAGE TYPE list box if the default value is not applicable.

    NOTE: The PACKAGE TYPE controls the method in which target tables are refreshed with source data. Refer to Package Types for detailed information on each supported package type.

    NOTE: Refer to Use the Manual Data Services Package Type for more information about using this package type.

  7. Enter text in TABLE RENAME field, if the table needs to be renamed in the target database.

    NOTE: The table name in SQL is copied directly from this field, and no special characters are allowed.

    NOTE: If the Table Rename field is populated, the Rename Table Template field on the Target Sources page has no effect.

  8. Update PRIORITY field if the default value is not applicable.

    NOTE: The PRIORITY field controls the order in which tables are processed.

  9. Verify ACTIVE check box is enabled to include the table in a source refresh.
  10. Click Save.
  11. Click Vertical View for a Table.
  12. Click Edit.

    View the field descriptions for the Tables page's Vertical View

  13. Enter a brief table description in Description field.
  14. Click Advanced Settings tab.
  15. Enter a value in Where Clause Override field if data must be filtered during a download process.

    NOTE: The Where Clause Override is an extra clause that conditions to the download SQL, e.g., WHERE SPRSL NOT IN (‘a’,’b’,’c’,’d’) and Where ColumnKey like ‘2007%’.

    NOTE: When a Where Clause Override is added to a table that is set up as a Data Services package the initial build will require an additional build. Immediately after the first build runs, the user must rerun the build. The second build will use the clause entered in the Where Clause Override field to refresh the table’s contents when a refresh is run. After the second build, the clause entered in the Where Clause Override field will be up to date for every subsequent build, and the user is not required to build the package more than once.

  16. Enter the database’s schema for the table in Table Schema Owner field.
  17. Select a package from Package Name list box.

    NOTE: If the Package Type is set to Manual, the Package Name field must be populated. For Manual Data Services package types, select from the list of available jobs in the Data Service Repository assigned to the target on the Targets page’s Vertical View.

    NOTE: If the package name does not display in the list box, confirm that the package has been named following naming conventions. Refer to Create Packages for more information.

  18. Update Schedule ID list box if the default value is not applicable.

    NOTE: The Schedule ID determines when the refresh process runs. If a schedule is set at the table level, the source schedule for the table is ignored. Schedules are created and maintained in Common. Refer to Create Schedules for more information.

  19. Update Schedule Group list box if the default value is not applicable.

    NOTE:   The Schedule Group field controls the schedule to build packages, refresh or filter on tables in a group.

  20. Click Synchronous check box to allow packages for Data Services jobs execute synchronously.

    NOTE: With synchronous execution, the build package jobs respect the queue and wait until jobs are finished before starting another job. This option applies to the data services Package Types.

  21. Click Schedule Single Thread check box if the table is to be downloaded in a queue with all other single-threaded tables.

    NOTE: If Schedule Single Thread is enabled, a Schedule Group must be assigned in order for tables to be downloaded in a single thread. If Schedule Single Thread is disabled, tables are downloaded in a logical grouping. Schedule Single Thread is ignored if Package Type is “DBMoto to Mirror” or "Data Services." 

  22. Update Rfc Records Per Call field if the default value is not applicable.

    NOTE: The Rfc Records Per Call only applies to RFC package types (BOA RFC, SAP RFC, and SAP TEXT). This field controls the number of records returned in a single block for Collect background process to parse and insert into the database.

  23. Click Save.

Use the Manual Data Services Package Type

Use this package type to use advanced features within Data Services Jobs, for example change data capture.

Prior to adding this package type configure the connection information used by SAP Data Services at the Target Source level. Refer to Configure Source for SAP Data Services for more information.

To use the package type:

  1. Create the job, dataflow, source, transformation and target in SAP Data Services.

NOTE: Use the naming conventions for Collect listed in Common. Select Common > Configuration > Modules > Collect > Parameters – Collect > Click Data Services Settings tab. Use the naming conversion in the table below.

 

Download Job Settings

Download Job Name Format

Displays name format for the job that executed a download of source table to a target table. Default value is #SOURCE#_#TARGET#_#SOURCETABLE#_imp.

Download Job Data Flow Name Format

Displays the format for the Data Flow object in a Download job. Default value is DF_#SOURCE#_#TARGET#_#SOURCETABLE#.

Download Job Source Table Data Flow Display Name Format

Displays the format for the source table display name in a Data Flow object of a Download job. Default value is #SOURCETABLE#_#SOURCE#.

Download Job Target Table Data Flow Display Name Format

Displays the format for the target table display name in a Data Flow object of a Download job. Default value is #TARGETTABLE#_#TARGET#.

Download Job Source Table Description Format

Displays the format of the source table description in a Download job. Default value is #SOURCETABLE# table from #SOURCE#.

Download Job Target Table Description Format

Displays the format of the target table description in a Download job. Default value is #TARGETTABLE# table from #TARGET#.

Download Job Schema Name Format

Displays the format of the Schema Name object in a Download job. Default value is TFM_#TARGETTABLE#.

  1. Add a table to the Source. Refer to Manually Register Tables to Source for more information.
  2. Select Manual Data Services in the Package Type list box.
  3. Click Save.
  4. Click Vertical View for the registered table package.
  5. Click the Advanced Setting tab.
  6. Click Edit.
  7. Select the Data Services job from the Package Name list box.
  8. Click Save.

NOTE: The Build process should not be run for any manual package types.

NOTE: The Manual Data Services package type does not execute against an SAPAPPSERVER source.

NOTE: The table must exist in the Collect target database for the package refresh to work.