Performing simple ADLS Gen2 Storage REST API operations using CURL

The blog points to performing simple ADLS Gen2 REST API operations such as List, Create, Update and Delete using CURL utility.

 

Let’s look at some of the CURL command syntax to perform REST API operations and will be making use of SAS as the auth scheme for most of the operations. For the operation of setting up ACL, we will be making use of Bearer token scheme. We need to take care of the pointers below while performing the operations via CURL:

 

• Ensure the URL is formed correctly as per the operation you are trying to perform.
• The mandatory header needs to be passed and can be done using “-H” parameter.
• Ensure you are appending/removing extra ‘?’ to the SAS token in the URLs accordingly.
• Http verb can be GET, PUT, PATCH or DELETE as provided by the CURL specifications.

Filesystem - List

Filesystem - List - REST API (Azure Storage Services) | Microsoft Docs

 

Syntax:

curl -i -X <HTTP Verb> -H "x-ms-version: 2019-12-12" -H "x-ms-date: <Date and Time in GMT format>" "https://<StorageAccountName>.dfs.core.windows.net?resource=account&SASToken"

 

In the below snippet, the operation is being performed on the account sampleadlgen2account and it lists the filesystem present inside it as below:

 

Path - List

Path - List - REST API (Azure Storage Services) | Microsoft Docs

 

Syntax:

curl -i -X <HTTP Verb> -H "x-ms-version: 2019-12-12" -H "x-ms-date: <Date and Time in GMT format>" "https://<StorageAccountName>.dfs.core.windows.net?recursive=true&resource=filesystem&SASToken"

 

In the below snippet, the operation is being performed on the filesystem named mycontainer of the account sampleadlgen2account and it lists the filesystem in the recursive format:

 

 

In order to create a File in the filesystem, 3 set of API’s need to be called in sequence i.e. CreateFile(Put) -> Append File(Patch) -> Flush File (Patch). Let us now try calling each of them one by one as per their required set of headers

 

Create File

Path - Create - REST API (Azure Storage Services) | Microsoft Docs

 

Syntax:

curl -i -X <HTTP Verb>  -H "x-ms-version: 2021-06-08" -H "x-ms-date: <Date and Time in GMT format>" -H "Content-Length: 0" -H "Content-Type: text/plain" “https://<StorageAccountName>.dfs.core.windows.net/<FileSystemName/FileName?resource=file&SASToken”

 

In the below snippet example, a Put operation was performed over the filesystem named mycontainer and the file name is sampletestfile. Note that we need to pass content length initially as 0. The file was created successfully.

You could further validate using Azure Portal or Azure Storage explorer as per your feasibility.

 

Append/Flush File

Path - Update - REST API (Azure Storage Services) | Microsoft Docs

 

The second API that needs to be called is to Append the data to the file that was created by the above call. For that we made a call to the Update API in which we passed the content to be written to the file along with its length.

 

curl -i -X <HTTP Verb>  -H "x-ms-version: 2021-06-08" -H "x-ms-date: <Date and Time in GMT format>" -H "Content-Length: Content Length in Bytes" -H "Content-Type: text/plain" -d "Content of the file" “https://<StorageAccountName>.dfs.core.windows.net/<FileSystemName/FileName?action=append&position=0&SASToken”

 

At this point there is still no update to the content size.

 

The last is to call a Flush file operation and that is done using the same Update API with Flush parameter.

 

curl -i -X <HTTP Verb> -H "x-ms-version: 2021-06-08" -H "x-ms-date: <Date and Time in GMT format>" -H "Content-Length: 0" -H "Content-Type: text/plain" “https://<StorageAccountName>.dfs.core.windows.net/<FileSystemName/FileName?action=flush&position=<flush position>&timeout=<timeout value>&close=true&SASToken"

 

 

Read File

Path - Read - REST API (Azure Storage Services) | Microsoft Docs

 

Syntax:

curl -i -X GET -H "x-ms-version: 2021-06-08" -H "x-ms-date: <Date and Time in GMT format>" "https://<StorageAccountName>.dfs.core.windows.net/<FileSystem>/<Filename>?SASToken"

 

In the below example, we tried performing the Read File operation on the same file name sampletestfile present in the container named mycontainer that we created as part of above operations

 

 

Delete FileSystem

Filesystem - Delete - REST API (Azure Storage Services) | Microsoft Docs

 

Syntax:

curl -i -X <HTTP Verb> -H "x-ms-version: 2020-08-04" -H "x-ms-date: <Date and Time in GMT format>" https://<StorageAccountName>.dfs.core.windows.net/<FileSystemName>?resource=filesystem&SASToken

 

In the below example snippet, we deleted a filesystem named tes from the account storage account

 

 

Similar operation can be performed on the file level basis as well provided the path is provided appropriately

 

Set Access Control

Path - Update - REST API (Azure Storage Services) | Microsoft Docs

 

Syntax:

curl -i -X <HTTP Verb> -H "x-ms-version: 2020-08-04" -H "authorization: Bearer <Bearer Token>" -H "x-ms-date: <Date and Time in GMT format>" -H "x-ms-acl: <ACL Permission in appropriate format>"https://<StorageAccountName>.dfs.core.windows.net/<DirectoryName>/?action=setAccessControl"

 

 

Hope this helps!