Execute SSIS Packages between Cloud Data Stores and a Private Network Data Storage
The following article describes a secure data integration approach to execute SSIS packages that access an SAP source system located in a private network (domain).
To perform data integration securely in a private network environment without a direct line-of-sight from the public cloud environment, you can install a self-hosted IR in your on-premises environment behind a firewall, or inside a virtual private network. The self-hosted integration runtime only makes outbound HTTP-based connections to the internet. The Xtract Connection (SAP connection) with the property ConnectByProxy = True is used to implement access to the local network using the Self-Hosted Integration Runtime (SHIR).
Prerequisites
The following prerequisites are mostly based on the official Microsoft documentation for using Azure integration runtime (IR) as well as Self-Hosted Integration Runtime (SHIR).
- Download and install the latest version of self-hosted IR and the additional drivers and runtime on your on-premises machine or Azure virtual machine (VM).
- Create an SSIS package on a development environment (Visual Studio) and execute it successfully.
- Publish the SSIS project to an SSIS in Azure Data Factory (ADF).
Step-by-Step Instructions
- Configure and start the Azure Data Factory IR with the necessary files to run the Xtract IS for Azure extension in ADF.
- Download and install the SHIR on the local Windows server (on-prem), see Microsoft Integration Runtime.
- Register the Integration Runtime (Self-hosted), see Configure a self-hosted IR via UI.
-
If applicable: Enable SSIS package execution on self-hosted IR node using the following cmd command:
-
Execute the XtractISSetup.exe file to install Xtraxt IS and use Xtract IS Data Flow Tasks.
- If your SSIS package uses one of the following components, download and install the relevant drivers:
- Object Linking and Embedding Database (OLEDB), see OLE DB Driver 18 for SQL Server
- Open Database Connectivity (ODBC)
- ADO.NET connectors
- Azure Feature Pack, see SSIS 2017 Feature Pack for Azure
Note
If the Integration Runtime (Self-hosted) node encouters errors during execution, error logs are provided using the Windows Event Viewer.
Related Links:
- Microsoft Documentation: .NET Framework system requirements
- Microsoft Documentation: Integration runtime in Azure Data Factory
- Microsoft Documentation: Self-hosted integration runtime
- Microsoft Documentation: Troubleshoot self-hosted integration runtime
- Microsoft Documentation: Assess SSIS project\packages for executions in Azure
Written by: Fabian Steffan, Maia Reichert-Baisch