Flat File Parquet
This page shows how to set up and use the Flat File Parquet destination. The Flat File Parquet destination loads data to a Parquet Database destination.
Create a new Flat File Parquet Destination
Follow the steps below to add a new Flat File Parquet destination to Xtract Universal:
- In the main window of the Designer, navigate to Server > Manage Destinations. The window “Manage Destinations” opens.
- Click [Add] to create a new destination. The window "Destination Details" opens.
- Enter a Name for the destination.
- Select the destination type Flat File Parquet from the drop-down menu. A list of connection details opens.
- Fill out the destination details to connect to the destination.
- Click [OK] to confirm your input.
The destination can now be assigned to extractions.
Destination Details
The destination details define the connection to the destination.
Output directory
Enter the folder path to save the destination flat files in. If the entered folder does not exist, a new folder is created.
Note
To write flat files to a network drive, you need to:
- Enter the File output path in UNC format e.g.,
\\Server2\Share\Folder1. - Grant write permission to the directory to the Xtract Universal service or run the service under a user account that has write permission.
Compatibility mode
The following compatibility modes are available:
- Pure
- Spark (does not support the data types used in pure mode, so other data types need to be used)
- BigQuery (formats columns names to be compatible with Google BigQuery, see BigQuery Documentation: Column Names)
| SAP | Pure / BigQuery | Spark |
|---|---|---|
| INT1 | UINT_8 | INT16 |
| TIMS | TIME_MILLIS | UTF8 |
Assign the Flat File Parquet Destination to an Extraction
Extractions write data to their assigned destination. Follow the steps below to assign a destination to an extraction:
- In the main window of the Designer, select an extraction.
- Click [ Destination]. The window “Destination Settings” opens.
- In the “Destination Settings” window, select a destination from the dropdown list.
- Optional: edit the destination settings.
- Click [OK] to confirm your input.
When running the extraction, the extracted SAP data is now written to the destination.
Destination Settings
The destination settings only affect the extraction that the destination is assigned to. To open the destination settings, select an extraction in the main window of the Designer and click [ Destination]. The window "Destination Settings" opens.
File Name
Determines the name of the target table. The following options are available:
| Option | Description |
|---|---|
| Same as name of SAP object | Copy the name of the SAP object. |
| Same as name of extraction | Adopt the name of the extraction. |
| Fully qualified extraction name | Adopt the name of the extraction, including the path of the extraction. This option avoids conflicts, when the extraction names are not unique. Use the fully qualified extraction name when the same extraction name is used in multiple extraction groups. |
| Custom | Define a name of your choice. |
Append timestamp
Add the timestamp in the UTC format (_YYYY_MM_DD_hh_mm_ss_fff) to the file name of the extraction
Use Script Expressions as Dynamic File Names
Script expressions can be used to generate a dynamic file name. This allows generating file names that are composed of an extraction's properties, e.g. extraction name, SAP source object. This scenario supports script expressions based on .NET and the following XU-specific custom script expressions:
| Input | Description |
|---|---|
#{Source.Name}# | Name of the extraction's SAP source. |
#{Extraction.ExtractionName}# | Name of the extraction. |
#{Extraction.FullyQualifiedExtractionName}# | Name of the extraction. If the extraction is part of an extraction group, the name of the extraction group is included in the extraction name. This option avoids conflicts, when the extraction names are not unique. |
#{Extraction.Type}# | Extraction type (Table, ODP, BAPI, etc.). |
#{Extraction.SapObjectName}# | Name of the SAP object the extraction is extracting data from. |
#{Extraction.Timestamp}# | Timestamp of the extraction. |
#{Extraction.SapObjectName.TrimStart("/".ToCharArray())}# | Removes the first slash '/' of an SAP object. Example: /BIO/TMATERIAL to BIO/TMATERIAL - prevents creating an empty folder in a file path. |
#{Extraction.SapObjectName.Replace('/', '_')}# | Replaces all slashes '/' of an SAP object. Example /BIO/TMATERIAL to _BIO_TMATERIAL - prevents splitting the SAP object name by folders in a file path. |
#{Extraction.Fields["FISCPER"].RangeSelections[0].LowerValue}# | Lower value of the range selection. |
#{Extraction.Fields["FISCPER"].RangeSelections[0].UpperValue}# | Upper value of the range selection. |
#{Extraction.Context}# | Only for ODP extractions: returns the context of the ODP object (SAPI, ABAP_CDS, etc). |
#{Extraction.Fields["[NameSelectionFiels]"].Selections[0].Value}# | Only for ODP extractions: returns the input value of a defined selection / filter. |
#{Odp.UpdateMode}# | Only for ODP extractions: returns the update mode (Delta, Full, Repeat) of the extraction. |
#{TableExtraction.WhereClause}# | Only for Table extractions: returns the WHERE clause of the extraction. |
Column Name Style
Defines the style of the column name. Following options are available:
| Option | Description |
|---|---|
| Code | The SAP technical column name is used as column name in the destination e.g., MAKTX. |
| PrefixedCode | The SAP technical column name is prefixed by SAP object name and the tilde character e.g., MAKT~MAKTX |
| CodeAndText | The SAP technical column name and the SAP description separated by an underscore are used as column name in the destination e.g., MAKTX_Material Description (Short Text). |
| TextAndCode | The SAP description and the SAP technical column name description separated by an underscore are used as column name in the destination e.g., Material Description (Short Text)_MAKTX. |
| Text | The SAP description is used as column name in the destination e.g., Material Description (Short Text). |
Convert dates
Converts the character-type SAP date (YYYYMMDD, e.g., 19900101) to a special date format (YYYY-MM-DD, e.g., 1990-01-01). Target data uses a real date data-type and not the string data-type to store dates.
Year 0
Converts the SAP date 00000000 to the entered value.
Year 9999
Converts the SAP date 9999XXXX to the entered value.
Invalid values
If an SAP date cannot be converted to a valid date format, the invalid date is converted to the entered value. NULL is supported as a value. When converting the SAP date the two special cases 00000000 and 9999XXXX are checked first.
Existing files
If a flat file by the same name already exists in the target directory, the following actions can be taken:
| Option | Description |
|---|---|
| Replace file | The export process overwrites existing files. |
| Abort extraction | The process is aborted, if the file already exists. |