Azure Data Factory trigger is not initiated when uploading a file using Java SDK
Symptoms:
- Uploading files using Java SDK class DataLakeFileClient fails to initiate the ADF trigger.
- Uploading files using Azure Portal initiates the trigger as expected.
- ADF trigger is configured to ignore empty blobs.
Analysis:
- Whenever a file is created using ADLS Gen2 REST API, there are 3 operations called at the backend i.e. CreateFile, AppendFile and FlushFile.
- Microsoft.Storage.BlobCreated event is triggered when a blob is created or replaced. Specifically, this event is triggered when clients use the CreateFile and FlushWithClose operations that are available in the Azure Data Lake Storage Gen2 REST API.
- As CreateFile API created a 0 byte blob, the event will not be triggered.
Resolution:
- You can initiate the trigger by adding FlushWithResponse method with close parameter as true.
Example:
boolean close = true;
File file = new File("C:\\Users\\mahmoudsamy\\route.txt");
fileClient.uploadFromFile("C:\\Users\\mahmoudsamy\\route.txt");
fileClient.flushWithResponse(file.length(), retainUncomittedData, close, httpHeaders, requestConditions, timeout, contect);
Note: Please make sure to pass the correct parameters when using FlushWithResponse to avoid unexpected results.
Published on:
Learn moreRelated posts
This Month in Azure Static Web Apps | 09/2024
We are back with another edition of the Azure Static Web Apps Community! :party_popper: September was yet another month ...
IPv6 Adoption: Enhancing Azure WAF on Front Door
The transition to IPv6 is a significant step for enterprise corporations, reflecting the evolution of internet technology and the need for a l...
Introducing the Data-Bound Reference Layer in Azure Maps Visual for Power BI
Imagine managing a nationwide sales team and needing to understand how your sales align with factors like population density, competitor locat...
GitHub Copilot for Azure: 6 Must-Try Features
As developers, we are constantly seeking tools that streamline our workflows and boost productivity. … Enter GitHub Copilot for Azure, now in ...
Unlocking the Best of Azure with AzureRM and AzAPI Providers
With the recent release of AzAPI 2.0, Azure offers two powerful Terraform providers to meet your infrastructure needs: AzureRM and AzAPI. The ...
Azure Communication Services Ideas Board: Share your feedback with the product team
Innovation is not a solitary pursuit, and we recognize that some of the best ideas come from you, our Azure Communication Services community. ...
Engage with the Azure Community Services Ideas Board: Your Voice Matters
Innovation is not a solitary pursuit, and we recognize that some of the best ideas come from you, our Azure Communication Services community. ...
Optimizing custom copilot (agent) performance with Azure Load Testing: A comprehensive guide
As we move into the next phase of digital transformation, the role of custom copilots is set to become increasingly pivotal. By leveragin...
Azure Storage - TLS 1.0 and 1.1 retirement
Overview TLS 1.0 and 1.1 retirement on Azure Storage was previously announced for Nov 1st, 2024, and it was postponed recently to 1 year later...