Interpreting Script activity output json with Azure Data Factory\Synapse analytics
Script activity in Azure Data Factory\ Synapse analytics is very helpful to run queries against data sources mentioned here in this document.
When we use two or more queries in the script activity, it is important to understand the output json of script activity to write expressions based on the output in the subsequent activities.
Consider the below Pipeline design:
We have two select queries as follows in script activity, and each of which will give a resultSet.
select top 2 * from tbl_adf;
Select * from tbladf
When debugged, it will give output as below.
So, as per line #2, resultSet count =2. It is because, we have two select queries in the script activity.
In case we want to get the Total Revenue value from Row#1, we have to write below expression.
@activity('Script1_copy1').output.resultSets[0].rows[0]['Total Revenue']
where, resultSets[0]: First select query result
rows[0]: first row in resultSets[0]
Same way, if we want to get the Total Revenue value from Row#2, we have to write below expression.
@activity('Script1_copy1').output.resultSets[0].rows[1]['Total Revenue']
where, resultSets[0]: First select query result
rows[1]: second row in resultSets[0]
And, the below expression gets the rowcount from each resultset.
@activity('Script1_copy1').output.resultSets[0].RowCount
@activity('Script1_copy1').output.resultSets[1].RowCount
So, by understanding the structure of output json, we are able to write expressions to access individual elements of the output of any activity in ADF\Synapse.
Published on:
Learn moreWe can help you with Interpreting Script activity output json with Azure Data Factory\Synapse analytics
If you want help implementing, troubleshooting, or improving this product, contact us and we’ll point you in the right direction.
Related posts
We're moving!
We’re moving to the Analytics on Azure Tech Community! All new Azure Synapse Analytics content will be published there. In the next few days a...
Upgrade to Azure Synapse runtimes for Apache Spark 3.4 & previous runtimes deprecation
It is important to stay ahead of the curve and keep services up to date. That's why we encourage all Azure Synapse customers with Apache ...
ADF\Synapse Analytics - Replace Columns names using Rule based mapping in Mapping data flows
In real time, the column names from source might not be uniform, some columns will have a space in it, some other columns will not. For exampl...
Synapse Connectivity Series Part #4 - Advanced network troubleshooting and network trace analysis
Continuing the series of this blog posts I would like to go more advanced on troubleshooting connectivity issues. I would like to thank also&n...
Boost your CICD automation for Synapse SQL Serverless by taking advantage of SSDT and SqlPackage CLI
Introduction Azure Synapse Analytics Serverless SQL is a query service mostly used over the data in your data lake, for data discovery,...
Metadata-Based Ingestion in Synapse with Delta Lake
Overview The crucial first step in any ETL (extract, transform, load) process or data engineering program is ingestion, w...
Missing Fields Added to Dedicated SQL pool Diagnostic Settings Logs
Over the past year, customers have informed the team there were a set of key columns missing in the standalone Dedicated SQL pools (formerly S...
Using Azure DevOps with Synapse Workspaces to create hot fixes in production environments
Have you ever deployed a release to production only to find out a bug has escaped your testing process and now users are being severely impact...
Azure Synapse MVP Corner - March 2023
About this blog series Microsoft Most Valuable Professionals, or MVPs, are technology experts who passionately share their knowledge with the ...
CI & CD With Azure Synapse Dedicated SQL Pool
Author(s): Pradeep Srikakolapu is a Program Manager in Azure Synapse Customer Success Engineering (CSE) team. Automating de...