Huawei Cloud OBS

This page shows how to set up and use the Huawei Cloud OBS destination. The Huawei Cloud OBS destination loads data to a Huawei Cloud Object Storage Service (OBS).

Create a new Huawei Cloud OBS Destination

Follow the steps below to add a new Huawei Cloud OBS destination to Xtract Universal:

1. In the main window of the Designer, navigate to Server > Manage Destinations. The window “Manage Destinations” opens.
   
2. Click [Add] to create a new destination. The window "Destination Details" opens.
   
3. Enter a Name for the destination.
4. Select the destination type Huawei Cloud OBS from the drop-down menu. A list of connection details opens.
5. Fill out the destination details to connect to the destination.
6. Click [OK] to confirm your input.

The destination can now be assigned to extractions.

Destination Details

The destination details define the connection to the destination.

Authentication

Access Key ID (AK)

Enter the access key of the Huawei Cloud OBS account. For more information on how to create access keys, see Huawei Cloud Support: Creating Access Keys (AK and SK)

Secret Access Key ID (SK)

Enter the secret access key of the Huawei Cloud OBS account. For more information on how to create access keys, see Huawei Cloud Support: Creating Access Keys (AK and SK)

Region

Select the region of the data storage.

Connect

Click [Connect] to establish a connection to the storage account. If the connection is successful, "Connected" is displayed next to the button.

Bucket

This setting only becomes available after a connection to the storage account is established.
Select a bucket. The SAP data is extracted into the selected bucket. Click [ ] to refresh the list of available buckets.

Misc

Folder path

Creates a folder structure within the bucket. Script expressions are supported, see Use Script Expressions as Dynamic Folder Paths.

• Create a single folder by entering a folder name without slashes: [folder]
  
• Create subfolders using the following syntax: [folder]/[subfolder_1]/[subfolder_2]/[..]

File Format

Select the required file format. You can choose between CSV, JSON and Parquet.

CVS Settings

The settings for file type CSV correspond to the settings of the Flat File CSV destination:

• CSV Settings
• Convert / Encoding

JSON Settings

To use the JSON file format, no further settings are necessary.

Parquet Settings

The settings for file type Parquet correspond to the settings of the Flat File Parquet destination:

• Compatibility Mode

Connection Retry and Rollback

Connection retry is a built-in function of the Huawei Cloud OBS destination. The retry function is activated by default.

Connection retry is a functionality that prevents extractions from failing in case of transient connection interruptions to Huawei Cloud OBS. Xtract Universal follows an exponential retry strategy. The selected exponential strategy results in seven retry attempts and an overall timespan of 140 seconds. If a connection is not established during the timespan of 140 seconds, the extraction fails.

The retry function is applied after receiving one of the following HTTP errors or exceptions:

• 503 Service Unavailable
• 504 Gateway Timeout
• WebExceptionStatus.ConnectionClosed
• WebExceptionStatus.ConnectFailure
• WebExceptionStatus.Timeout
• WebExceptionStatus.RequestCanceled
• WebExceptionStatus.SendFailure
• WebExceptionStatus.NameResolutionFailure

Rollback covers scenarios where extractions do not fail due to connection failures to Huawei but e.g. due to an error when connecting to SAP. In those cases Xtract Universal tries to remove any files from the Huawei Cloud storage that were created in the course of the extraction.

Assign the Huawei Cloud OBS Destination to an Extraction

Extractions write data to their assigned destination. Follow the steps below to assign a destination to an extraction:

1. In the main window of the Designer, select an extraction.
2. Click [ Destination]. The window “Destination Settings” opens.
   
3. In the “Destination Settings” window, select a destination from the dropdown list.
   
4. Optional: edit the destination settings.
5. Click [OK] to confirm your input.

When running the extraction, the extracted SAP data is now written to the destination.

Destination Settings

The destination settings only affect the extraction that the destination is assigned to. To open the destination settings, select an extraction in the main window of the Designer and click [ Destination]. The window "Destination Settings" opens.

File Name

Determines the name of the target table. The following options are available:

Append timestamp

Add the timestamp in the UTC format (_YYYY_MM_DD_hh_mm_ss_fff) to the file name of the extraction

Use Script Expressions as Dynamic File Names

Script expressions can be used to generate a dynamic file name. This allows generating file names that are composed of an extraction's properties, e.g. extraction name, SAP source object. This scenario supports script expressions based on .NET and the following XU-specific custom script expressions:

#{Source.Name}#

#{Extraction.ExtractionName}#

#{Extraction.Type}#

#{Extraction.SapObjectName}#

#{Extraction.Timestamp}#

#{Extraction.SapObjectName.TrimStart("/".ToCharArray())}#

#{Extraction.SapObjectName.Replace('/', '_')}#

#{Extraction.Fields["FISCPER"].RangeSelections[0].LowerValue}#

#{Extraction.Fields["FISCPER"].RangeSelections[0].UpperValue}#

#{Extraction.Context}#

#{Extraction.Fields["[NameSelectionFiels]"].Selections[0].Value}#

#{Odp.UpdateMode}#

#{TableExtraction.WhereClause}#

Column Name Style

Defines the style of the column name. Following options are available:

Convert dates

Converts the character-type SAP date (YYYYMMDD, e.g., 19900101) to a special date format (YYYY-MM-DD, e.g., 1990-01-01). Target data uses a real date data-type and not the string data-type to store dates.

Year 0

Converts the SAP date 00000000 to the entered value.

Year 9999

Converts the SAP date 9999XXXX to the entered value.

Invalid values

If an SAP date cannot be converted to a valid date format, the invalid date is converted to the entered value. NULL is supported as a value. When converting the SAP date the two special cases 00000000 and 9999XXXX are checked first.

Folder

Writes the extracted data to a specific folder structure within the bucket. If the specified folder does not exist, it is created.

• Create a single folder by entering a folder name without slashes: [folder]
  
• Create subfolders using the following syntax: [folder]/[subfolder_1]/[subfolder_2]/[..]

Use Script Expressions as Dynamic Folder Paths

Script expressions can be used to generate a dynamic folder path. This allows generating folder paths that are composed of an extraction's properties, e.g., extraction name, SAP source object. The described scenario supports script expressions based on .NET and the following XU-specific custom script expressions:

#{Source.Name}#

#{Extraction.ExtractionName}#

#{Extraction.Type}#

#{Extraction.SapObjectName}#

#{Extraction.Timestamp}#

#{Extraction.SapObjectName.TrimStart("/".ToCharArray())}#

#{Extraction.SapObjectName.Replace('/', '_')}#

#{Extraction.Context}#

#{Extraction.Fields["[NameSelectionFields]"].Selections[0].Value}#

#{Odp.UpdateMode}#

#{TableExtraction.WhereClause}#

#{Extraction.Fields["[0D_NW_CODE]"].Selections[0].Value}#

#{Extraction.Fields["[0D_NW_CHANN]"].RangeSelections[0].LowerValue}#

#{Extraction.Fields["[0D_NW_CHANN]"].RangeSelections[0].UpperValue}#

#{Extraction.Fields["0D_NW_CODE"].Selections[0].Value}#

#{Extraction.Fields["0D_NW_CHANN"].RangeSelections[0].LowerValue}#

#{Extraction.Fields["0D_NW_CHANN"].RangeSelections[0].UpperValue}#

Compression

Compression is only available for the csv file format, see Destination Details: File Format.

GZip

The data is transferred compressed and stored as a gz file.

File Splitting

Writes extraction data of a single extraction to multiple files. Each filename is appended by _part[nnn].

Max. file size

The value set in Max. file size determines the maximum size of each file.

Empty Files

If this option is active, empty result sets create an empty file in the target environment. Deactivate this option to not create empty files.