Loading...

Azure Data Factory trigger is not initiated when uploading a file using Java SDK

Azure Data Factory trigger is not initiated when uploading a file using Java SDK

Symptoms

 

  •   Uploading files using Java SDK class DataLakeFileClient fails to initiate the ADF trigger. 
  •   Uploading files using Azure Portal initiates the trigger as expected. 
  •   ADF trigger is configured to ignore empty blobs. 

 

Analysis

 

  • Whenever a file is created using ADLS Gen2 REST API, there are 3 operations called at the backend i.e. CreateFile, AppendFile and FlushFile.
  • Microsoft.Storage.BlobCreated event is triggered when a blob is created or replaced. Specifically, this event is triggered when clients use the CreateFile and FlushWithClose operations that are available in the Azure Data Lake Storage Gen2 REST API.
  • As CreateFile API created a 0 byte blob, the event will not be triggered. 

 

Resolution

 

  • You can initiate the trigger by adding FlushWithResponse method with close parameter as true.

Example

 

        boolean close = true;

File file = new File("C:\\Users\\mahmoudsamy\\route.txt");

        fileClient.uploadFromFile("C:\\Users\\mahmoudsamy\\route.txt");

        fileClient.flushWithResponse(file.length(), retainUncomittedData, close, httpHeaders, requestConditions, timeout, contect);

 

Note: Please make sure to pass the correct parameters when using FlushWithResponse to avoid unexpected results.

Published on:

Learn more
Azure PaaS Blog articles
Azure PaaS Blog articles

Azure PaaS Blog articles

Share post:

Related posts

Long-term data retention up to 10 years: Announcing Private Preview of Azure Backup for Azure Cosmos DB

Azure Backup for Azure Cosmos DB is a new option that lets you securely protect and recover your Azure Cosmos DB data for compliance, audit, a...

1 hour ago

Assessing Your Azure Data Factory for Migration to Fabric Data Factory

As organizations modernize their data integration workflows, moving from Azure Data Factory (ADF) to Fabric Data Factory is an important miles...

7 hours ago

Video: Copilot Studio – Connect Azure SQL As Knowledge

SQL is where the worlds data is stored. And in this video I’m going to ... The post Video: Copilot Studio – Connect Azure SQL As Knowled...

1 day ago

One Azure, Many Logins: How Users Access Microsoft’s Cloud Safely

Users can access Microsoft Azure through several flexible and secure methods, depending on their role, device, and workload needs. The most co...

3 days ago

Unlocking New Possibilities: Microsoft Azure Hyperscale AI Computing with H200 GPUs Accelerates Secure AI Innovation in Azure for U.S. Government Secret and Top Secret

As artificial intelligence continues to reshape industries and redefine the boundaries of innovation, Microsoft is proud to announce a leap fo...

4 days ago
Stay up to date with latest Microsoft Dynamics 365 and Power Platform news!
* Yes, I agree to the privacy policy