Realtime information sharing between z/OS Applications and Azure
Title: Realtime information sharing between z/OS Applications and Azure
The IBM Z Digital Integration Hub (zDIH) is designed to facilitate real-time consumable information flow at scale between mainframe applications and hybrid cloud, without disrupting those core business applications and associated service-level agreements (SLAs). Z Digital Integration Hub integrates core business applications running on z/OS with broader Azure cloud architecture to accelerate transformation.
With IBM Z Digital Integration Hub, you can curate necessary information as opposed to moving all the raw data originating from the many core applications and related data sources. Then, mainframe-optimized technologies are used to communicate and store that curated/aggregated information in memory. You can surface that information through various standards-based interfaces, including through event-based mechanisms such as Kafka or open standard-based APIs. IBM zDIH is particularly well-suited for use cases involving Systems of Record on IBM z/OS®, providing a robust solution for streamlined and secure application integration.
Overview
For IBM® Z application modernization, a crucial focus is establishing an optimal, effective, scalable, and real-time information flow between on-premise core Systems of Record and cloud-based applications. Real-time information from Core Systems of Record is critical across industries. For example, in banking, Core Systems of Record generate high-value information needed in real-time including deposits, Automated Clearing House (ACH) payments, card processing, insurance claims processing, and many more. IBM zDIH addresses existing challenges for obtaining mainframe data using alternative approaches, including:
- Insufficient Currency of Information: Arising from latency introduced by data migration
- Lack of Event-Based Integration: Absence of an event-based integration approach
- Inability to Separate Query and Update Traffic: Resulting in an unpredictable impact on the Systems of Record
- Data Format and Content Challenges: Systems of record data may lack desired information or be in formats that are not consumable by hybrid cloud applications
- Difficulty in Maintaining Transactional Order in Migrated Data: Leading to labor-intensive reconciliation procedures and inaccuracies
- Challenges with data regulations and consistency: Arising while keeping multiple copies with data migration
Since the focus is on real-time information, the recommended approach is fundamentally hybrid—having an optimized cache implementation on z/OS co-residing with critical core systems and enabling the information to flow to numerous cloud applications on Azure. This approach enables enterprise-wide application modernization and digital transformation efforts.
Architecture
In this reference architecture and application demo, an example front-end application is hosted in Microsoft Azure which integrates with IBM Z Digital Integration Hub running on z/OS which, in turn, efficiently integrates with core business applications co-residing on z/OS. The front-end demo application provides a visualization of a user’s banking account transaction history. Additionally, the application retrieves user information from Microsoft Fabric and provides notifications to the user about account conditions.
IBM Z Digital Integration Hub (zDIH) allows for the near-real-time information sharing between core business applications and Microsoft Fabric to satisfy these use cases. When a user checks their account balance transaction history, the information request is sent directly to IBM zDIH caches, which maintains currency of the account transactions without needing to copy data into data warehouse solutions. In doing so, IBM zDIH prevents data proliferation and meets currency requirements at scale. When a user views or modifies their general profile information, these requests are sent to Microsoft Fabric and retrieved from OneLake.
Finally, IBM zDIH can be configured in an events-based approach to proactively share curated information via Kafka to Microsoft Fabric event streams . As new transactions of interest are processed on the System of Record, account notifications can be processed through these event streams and trigger Data Activator flows to emit the notification to downstream applications and consumers. Combining the capabilities of Microsoft Fabric and IBM zDIH allows clients to quickly develop new solutions using near-real-time information from IBM Z.
Workflow
- User authenticates with the frontend .net/Java application or Power Apps
- User identity is verified with Microsoft Entra
- User dashboard is populated via 2 queries. a. Query account balance through REST interfaces to zDIH caches b. Query Customer 360 data from Microsoft Fabric & One Lake
- User account transactions are processed in z/OS Core applications and transactions or balance is cached in zDIH
- Cached zDIH data is pushed to Eventstream part of Microsoft Fabric
- Data Activator is triggered on low balance or high transaction threshold
- Frontend application receives push notification and presents warning to user
- Azure Application Monitor and Application insights can be used to monitor Power Platform, the application APIs and security.
- Azure Application Monitor and Application insights can be used to monitor Power Platform, the application APIs and security.
Components
- IBM Z Digital Integration Hub (IBM zDIH) provides real-time information flow at scale, protects production environments from unpredictable inquiry traffic with adaptability and offers flexible interactions such as REST API , JDBC, and event-based architectures. IBM zDIH includes a DevKit for no-code/low-code approach to accelerate deployments.
- Azure API Management – APIs enable digital experiences, simplify application integration, underpin new digital products, and make data and services reusable and universally accessible. Azure API management provides a hybrid, multi-cloud management platform for APIs across all environments.
- App Service is a fully managed platform for building, deploying, and scaling web apps. It supports various programming languages and frameworks, offering seamless integration with Azure services. It provides auto-scaling and high availability, simplifying app deployment and management. Visual Studio is a powerful integrated development environment (IDE) that you can use to write, edit, debug, and build code, and then deploy your web app.
- Microsoft Power Platform on Azure –increases agility across your organization by allowing you to rapidly build low-code apps using Power Apps to modernize processes and solve tough challenges.
- Azure Monitor - Azure Monitor helps you maximize the availability and performance of your applications and services. It delivers a comprehensive solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises environments. This information helps you understand how your applications are performing and proactively identify issues that affect them and the resources they depend on.
- Azure ExpressRoute - ExpressRoute lets you extend your on-premises networks into the Microsoft cloud over a private connection facilitated by a connectivity provider. With ExpressRoute, you can establish connections to Microsoft cloud services, such as Microsoft Azure and Office 365.
- Azure Site Recovery (ASR) is a disaster recovery solution provided by Microsoft Azure that helps protect and recover applications and workloads running on virtual or physical machines. It provides business continuity and minimizes downtime in the event of planned or unplanned outages.
Alternatives
- In place of using the ExpressRoute gateway, another option is to utilize the Azure VPN Gateway. The Virtual Network Gateway enables secure site-to-site connectivity (connecting on-premises network to Azure virtual network) through encrypted tunnels.
Scenario Details
- Microsoft Power Platform / Power Apps is a low code / no code option to create a web-based UI that will in turn connect to the above developed services. This architecture illustrates both a low code Power Apps client as well as a custom web app client.
- IBM Z Digital Integration Hub (IBM zDIH) includes a zDIH DevKit for a low code / no code approach to accelerate zDIH deployments
Potential use cases
It's crucial to define use cases that align well with IBM zDIH to maximize its value proposition. Common use cases exhibit the following characteristics:
Integration with core Systems of Record (SoR) on z/OS®: IBM zDIH enables seamless integration with core Systems of Record (SoR) running on the z/OS platform. This integration ensures that critical and high-value business operational information generated by mainframe-based SoR systems, such as customer information, transaction records, and running account balances, can be efficiently consumed and analyzed in real-time.
Real-time information flow: IBM zDIH facilitates real-time information flow by enabling the continuous capture, processing, and analysis of relevant transactions from SoR systems. This real-time information flow ensures that organizations have access to up-to-date and actionable insights, allowing them to transform their operations, provide improved customer experiences, as well as new revenue opportunities.
Event-oriented processing: IBM zDIH supports event-oriented processing, allowing organizations to capture and analyze relevant events as they occur in real-time. This capability enables organizations to detect and respond to critical events promptly, such as fraudulent transactions, customer requests, and market fluctuations, helping to mitigate risks and enhance operational efficiency.
Separation of query processing from update transactions: IBM zDIH provides the opportunity to separate query processing from update transactions, ensuring thatunpredictable inquiry traffic can service current information in real-time while also avoiding disruptions to SoR systems.
Transformation of System of Record (SoR) information models into a more consumable model aligned to consumers: IBM zDIH facilitates the transformation of SoR data into a more consumable information model, making it easier for business users and applications to consume mainframe data, ensure accuracy, and accelerate time to value of new digital initiatives,
Cost advantages for inquiry processing: IBM zDIH offers cost advantages by separating inquiry processing from transaction processing, utilizing high-zIIP eligibility, optimizing resource utilization and leveraging the scalability of the IBM Z platform .
Considerations
The following considerations, based on the Azure Well-Architected Framework, apply to this solution.
Reliability
Reliability ensures your application can meet the commitments you make to your customers. For more information, see Overview of the reliability pillar.
- IBM zDIH has built-in support for high-availability and disaster recovery. Additonally, zDIH is built on IBM's mainframe platform, known for its unparalleled reliability and availability features. This ensures that critical workloads running on zDIH are highly resilient against failures and disruptions.
Security
Security provides assurances against deliberate attacks and the abuse of your valuable data and systems. For more information, see Overview of the security pillar.
- Azure Active Directory (Azure AD) provides a broad spectrum of security features and capabilities to protect identities, applications, and data, and provide authentication and authorization of users and applications. The seamless integration of Azure AD with OAuth enables secure authentication and authorization for applications.
- IBM zDIH provides robust security built-in to the product as well as leveraging the exceptional security capabilities of the IBM Z platform to mitigate business risks, safeguard application data, and ensure long-term security for your systems.
Cost Optimization
Cost optimization is about looking at ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Overview of the cost optimization pillar.
- IBM zDIH is highly zIIP eligible and can be used to separate inquiry traffic from update traffic to core business applications on z/OS in an optimized Command Query Response Separation (CQRS) for cost optimization. Additionally, zDIH provides readily consumable information via standard interfaces to further enhance development efficiencies of downstream applications running on Azure.
- Azure provides various licensing options for the Power Apps platform as well – that can be controlled and managed with respect to the total number of users, logins allowed, page views etc.
Use Azure Pricing Calculator to estimate the cost of implementing the solution.
Operational Excellence
Operational excellence covers the operations processes that deploy an application and keep it running in production. For more information, see Overview of the operational excellence pillar.
- IBM zDIH enables real-time information flow to support faster decision-making, improves responsiveness to changing conditions, and enables organizations to capitalize on opportunities more effectively.
- IBM zDIH DevKit provides a no-code/low code approach for application generation capabilities. This increases developer productivity and accelerates application time-to-market. The generated applications follow standardized conventions and can be integrated into your DevOps pipelines.
Performance Efficiency
Performance Efficiency covers the operations processes that deploy an application and keep it running in production. For more information, see Overview of the performance efficiency pillar.
- IBM zDIH efficiently enables real-time information flow at scale leveraging in-memory techniques and the unparalleled performance of the IBM Z platform.
Contributors
This article is maintained by Microsoft. It was originally written by the following contributors.
Principal authors:
- Bhaskar Bandam, Senior Technical Program Manager from Microsoft
- Ivan Dovgan, Senior Technical Staff Member, Chief Architect – z/OS Hyperscaler Integrations from IBM
Other Contributors:
- Venkat Ramakrishnan, Senior Technical Program Manager from Microsoft
- Amy Hieter, Product Manager, IBM Z Digital Integration Hub
- Mythili Venkatakrishnan, Distinguished Engineer, IBM Z Financial Services CTO from IBM
- Nasser Ebrahim, Senior Technical Staff Member, IBM Z Digital Integration Hub
Next Steps
- For more information, please contact [email protected]
Related Resources
