Loading...

Azure Data Factory trigger is not initiated when uploading a file using Java SDK

Azure Data Factory trigger is not initiated when uploading a file using Java SDK

Symptoms

 

  •   Uploading files using Java SDK class DataLakeFileClient fails to initiate the ADF trigger. 
  •   Uploading files using Azure Portal initiates the trigger as expected. 
  •   ADF trigger is configured to ignore empty blobs. 

 

Analysis

 

  • Whenever a file is created using ADLS Gen2 REST API, there are 3 operations called at the backend i.e. CreateFile, AppendFile and FlushFile.
  • Microsoft.Storage.BlobCreated event is triggered when a blob is created or replaced. Specifically, this event is triggered when clients use the CreateFile and FlushWithClose operations that are available in the Azure Data Lake Storage Gen2 REST API.
  • As CreateFile API created a 0 byte blob, the event will not be triggered. 

 

Resolution

 

  • You can initiate the trigger by adding FlushWithResponse method with close parameter as true.

Example

 

        boolean close = true;

File file = new File("C:\\Users\\mahmoudsamy\\route.txt");

        fileClient.uploadFromFile("C:\\Users\\mahmoudsamy\\route.txt");

        fileClient.flushWithResponse(file.length(), retainUncomittedData, close, httpHeaders, requestConditions, timeout, contect);

 

Note: Please make sure to pass the correct parameters when using FlushWithResponse to avoid unexpected results.

Published on:

Learn more
Azure PaaS Blog articles
Azure PaaS Blog articles

Azure PaaS Blog articles

Share post:

Related posts

Eliminate LLM Cold starts: Load models up to 6x Faster with Azure Blob Storage and Run:AI Model Streamer

Stop paying for idle GPUs while model weights copy to disk. Stream them straight into GPU memory instead with Run:AI Streamer from Azure Blob ...

2 days ago

Exponential backoff and circuit breaker for Service Bus-triggered Azure Functions

Use exponential backoff and circuit breaker patterns in Azure Functions with Service Bus SDK bindings. Control retry storms, protect dependenc...

7 days ago
Stay up to date with latest Microsoft Dynamics 365 and Power Platform news!
* Yes, I agree to the privacy policy