Azure Data Factory trigger is not initiated when uploading a file using Java SDK
Symptoms:
- Uploading files using Java SDK class DataLakeFileClient fails to initiate the ADF trigger.
- Uploading files using Azure Portal initiates the trigger as expected.
- ADF trigger is configured to ignore empty blobs.
Analysis:
- Whenever a file is created using ADLS Gen2 REST API, there are 3 operations called at the backend i.e. CreateFile, AppendFile and FlushFile.
- Microsoft.Storage.BlobCreated event is triggered when a blob is created or replaced. Specifically, this event is triggered when clients use the CreateFile and FlushWithClose operations that are available in the Azure Data Lake Storage Gen2 REST API.
- As CreateFile API created a 0 byte blob, the event will not be triggered.
Resolution:
- You can initiate the trigger by adding FlushWithResponse method with close parameter as true.
Example:
boolean close = true;
File file = new File("C:\\Users\\mahmoudsamy\\route.txt");
fileClient.uploadFromFile("C:\\Users\\mahmoudsamy\\route.txt");
fileClient.flushWithResponse(file.length(), retainUncomittedData, close, httpHeaders, requestConditions, timeout, contect);
Note: Please make sure to pass the correct parameters when using FlushWithResponse to avoid unexpected results.
Published on:
Learn moreRelated posts
Record Scanner for vinyl collectors cuts costs with Azure Cosmos DB vector search
by Artur Drozdz, Founder of Record Scanner (recordscanner.com) If you’re like me, there’s at least one room in your home with an entire cabine...
🚀 Introducing the New VS Code Extension for Azure Cosmos DB
We’re excited to share that the Azure Databases extension for Visual Studio Code is now officially rebranded as the Azure Cosmos DB extension!...
AI-based T-SQL Refactoring: an automatic intelligent code optimization with Azure OpenAI
This article presents an AI-powered approach to automating SQL Server code analysis and refactoring. The system intelligently identifies ineff...
Azure Boards integration with GitHub Copilot (Private Preview)
Several months ago, GitHub introduced the public preview of its Copilot coding agent, a powerful new capability that allows you to assign GitH...
What is Azure Key Vault and How It Secures Microsoft Dynamics 365 CRM Systems?
Azure Key Vault is a service by Microsoft Azure that helps securely store and manage sensitive information such as API keys, connection string...
Azure AI Foundry Model In Copilot Studio Custom Prompts
Any custom model created in Azure AI Foundry can be used in Copilot Studio. This ... The post Azure AI Foundry Model In Copilot Studio Custom ...