Azure Data Factory / Synapse Pipelines
Azure-native ETL/ELT orchestration.
Access mode: Read-only
Required information
| Field | Details |
|---|---|
| Subscription ID | Azure subscription containing the ADF/Synapse resource. |
| Resource group | Resource group name. |
| Factory / Workspace name | ADF factory name or Synapse workspace name. |
| Authentication | Azure AD service principal. |
| IAM role | Read-only: Data Factory Reader on the factory resource. Or Reader on the resource group. |
Network considerations
Azure Management API: HTTPS to management.azure.com. No VPN needed.
ADF Managed VNet / Private Endpoints: Affects ADF’s connectivity to data sources, not Flume’s API access to ADF.
Credential and auth management
Preferred: Azure AD service principal. Data Factory Reader role. Standard client credentials flow.
Git integration: If ADF is Git-connected, also access the repo directly for pipeline JSON definitions.
Token: Standard Azure AD OAuth2. Flume handles refresh.
Validation checks
| Check | Method | Expected result |
|---|---|---|
| Authentication | GET /subscriptions/<id>/resourceGroups/<rg>/providers/Microsoft.DataFactory/factories/<factory>?api-version=2018-06-01 | Returns factory metadata |
| Pipeline listing | GET .../pipelines?api-version=2018-06-01 | Lists pipelines |
| Dataset listing | GET .../datasets?api-version=2018-06-01 | Lists datasets |
| Activity runs | POST .../queryPipelineRuns with time range | Returns recent runs |
Every connection starts from the pre-engagement checklist and goes through the universal validation protocol before production sign-off.