Azure Data Factory trigger is not initiated when uploading a file using Java SDK
Symptoms:
- Uploading files using Java SDK class DataLakeFileClient fails to initiate the ADF trigger.
- Uploading files using Azure Portal initiates the trigger as expected.
- ADF trigger is configured to ignore empty blobs.
Analysis:
- Whenever a file is created using ADLS Gen2 REST API, there are 3 operations called at the backend i.e. CreateFile, AppendFile and FlushFile.
- Microsoft.Storage.BlobCreated event is triggered when a blob is created or replaced. Specifically, this event is triggered when clients use the CreateFile and FlushWithClose operations that are available in the Azure Data Lake Storage Gen2 REST API.
- As CreateFile API created a 0 byte blob, the event will not be triggered.
Resolution:
- You can initiate the trigger by adding FlushWithResponse method with close parameter as true.
Example:
boolean close = true;
File file = new File("C:\\Users\\mahmoudsamy\\route.txt");
fileClient.uploadFromFile("C:\\Users\\mahmoudsamy\\route.txt");
fileClient.flushWithResponse(file.length(), retainUncomittedData, close, httpHeaders, requestConditions, timeout, contect);
Note: Please make sure to pass the correct parameters when using FlushWithResponse to avoid unexpected results.
Published on:
Learn moreRelated posts
How to back up on-premises Windows VMs to Azure
Introducing Markers in Azure Maps for Power BI
We’re announcing a powerful new capability in the Azure Maps Visual for Power BI: Markers. This feature makes it easier than ever for organiza...
Azure Boards additional field filters (private preview)
We’re introducing a limited private preview that allows you to add additional fields as filters on backlog and Kanban boards. This long-reques...
What’s new with Azure Repos?
We thought it was a good time to check in and highlight some of the work happening in Azure Repos. In this post, we’ve covered several recent ...
Part 1: Building Your First Serverless HTTP API on Azure with Azure Functions & FastAPI
Introduction This post is Part 1 of the series Serverless Application Development with Azure Functions and Azure Cosmos DB, where we explore ...