- Platform Specific Architecture for Azure (Microsoft Azure)
- Azure: Overall Architecture
- Azure: Basic Architecture
- Azure: Virtual Hostname/IP
- Azure: High Availability
- Azure: Disaster Recovery
- Azure: Data Tiering Options
- Azure: XSA
Following diagram is illustrating complete version of Reference Architecture for SAP HANA tailored for Azure (Microsoft Azure).
For detailed explanation of individual modules please see individual sections in Generic SAP HANA Architecture.
You can also review official Azure Reference Architecture and other documentation:
- Azure: SAP HANA infrastructure configurations and operations
- Azure: SAP HANA (Large Instances) architecture
Link to generic content: Module: Basic Architecture
Not every instance type is supported for productive SAP HANA usage.
Official list of SAP certified instance types is available at SAP: The SAP HANA Hardware Directory. This should always be used to decide whether the particular instance type is supported for SAP HANA or not.
Azure specific list of certified instances with additional details can be found in following references:
SAP HANA Storage Configuration for Virtual Machines is coming in following flavors:
- Azure Premium SSD - cheaper storage that meets SAP KPI requirements for most of the SAP HANA workloads
- Azure Ultra disk - high-performance storage intended for most demanding SAP HANA workloads
- Azure NetApp Files - special option for SAP HANA Host Auto-Failover (in single Availability Zone)
Recommended disk setup for each option is described is following sections.
SAP HANA Storage Configuration for HANA Large Instances is always based on NFS storage as supported deployment options are covered in Azure: Supported scenarios for HANA Large Instances.
| Filesystem | Name | Device type | Comment |
|---|---|---|---|
| / | Root volume | P6 or P10 | |
| /hana/data | SAP HANA data | P20, P30 or P40 | |
| /hana/log | SAP HANA logs | P20 | |
| /hana/shared | SAP HANA shared | P20 or P30 | For Single-Node deployments |
| /hana/shared | SAP HANA shared | Azure NetApp Files | For Scale-Out deployments |
| /usr/sap | SAP binaries | P6 | |
| /backups | SAP HANA backup | P20, P30, P40, P50 | For Single-Node deployments |
| /backups | SAP HANA backup | Azure NetApp Files | For Scale-Out deployments |
Instance specific sizing recommendations are available at Azure: Azure Premium SSD.
Note: Cost conscious storage configuration is available for use, however, it is not officially supported by SAP.
| Filesystem | Name | Device type | Comment |
|---|---|---|---|
| / | Root volume | P6 or P10 | |
| /hana/data | SAP HANA data | Ultra disk | Alternatively can be on one volume with logs |
| /hana/log | SAP HANA logs | Ultra disk | Alternatively can be on one volume with data |
| /hana/shared | SAP HANA shared | P20 or P30 | For Single-Node deployments |
| /hana/shared | SAP HANA shared | Azure NetApp Files | For Scale-Out deployments |
| /usr/sap | SAP binaries | P6 | |
| /backups | SAP HANA backup | P20, P30, P40, P50 | For Single-Node deployments |
| /backups | SAP HANA backup | Azure NetApp Files | For Scale-Out deployments |
Instance specific sizing recommendations are available at Azure: Azure Ultra disk.
Note: Please be aware about Ultra disk limitations as described in Azure: Ultra disk - GA scope and limitations.
| Filesystem | Name | Device type | Comment |
|---|---|---|---|
| / | Root volume | P6 or P10 | |
| /hana/data | SAP HANA data | Azure NetApp Files | For bigger systems one volume per mount point |
| /hana/log | SAP HANA logs | Azure NetApp Files | For bigger systems one volume per mount point |
| /hana/shared | SAP HANA shared | Azure NetApp Files | |
| /usr/sap | SAP binaries | P6 | |
| /backups | SAP HANA backup | Azure NetApp Files |
Instance specific sizing recommendations are available at Azure: Azure NetApp Files.
Note: Please be aware about Azure NetApp Files limitations as described in Azure: Azure NetApp Files - Important considerations.
Key limitation is fact that Azure NetApp Files are not Availability Zone aware and can cause performance degradation when not deployed in close proximity (for example following High Availability takeover).
As visualized on the Overall Architecture diagram - in Azure both Availability Zones are using one subnet stretched across multiple Availability Zones. This is achieved by Network Virtualization, however, side effect of this approach is that classical ARP cache invalidations are not working - this is having big impact on implementation of Cluster IP in Azure.
Impact on implementation of Cluster IP in Azure is described in Azure: High Availability.
Link to generic content: Module: Virtual Hostname/IP
This chapter describes recommended implementation of Virtual Hostname and Virtual IP for an SAP HANA installation on the Azure cloud.
The implementation is based on assigning a Secondary static private IP address to an existing network interface of the Azure Virtual Machine (VM). A Secondary static private IP address can be reassigned to another VM and so it can follow SAP HANA instance relocation. For more details see Azure: Assign multiple IP addresses. This Secondary static private IP address is associated with the Virtual Hostname which is used during SAP HANA instance installation.
- define Virtual IP (in the same subnet as the network interface) and Virtual Hostname for SAP HANA Instance
- assign Virtual IP as Secondary static private IP address to existing network interface (see Azure: Add IP addresses to a VM
- configure OS to use Virtual IP Address
- make Virtual Hostname resolvable (e.g. updating
/etc/hosts) - install SAP HANA instance with the Virtual Hostname (see SAP: Administration Guide - Default Host Names and Virtual Host Names)
Note: to reassign the Virtual IP to another VM you need first to remove it on the existing VM and after that to assign it on the new VM (see Azure: Remove IP addresses)
A network communication initiated from a remote system to the Virtual IP of SAP HANA instance is established and takes place between remote system IP and the Virtual IP (Secondary static private IP address). The Primary private IP address on the same interface is not involved in this communication.
In contrast to an inbound connection, when SAP HANA instance initiates a network connection to a remote system the Primary private IP address is used as source IP instead of Virtual IP (Secondary static private IP address).
If there is requirement to use Virtual IP as the source IP, it could be achieved by means of linux routing. The core of the idea is to add route specifying source address like ip route add <network/mask> via <gateway> src <virtual_ip> (see Routing for multiple uplinks/providers).
Link to generic content: Module: High Availability
Best practice for deploying SAP HANA is to stretch High Availability cluster across Availability Zones. Each Azure Availability Zone is physically separate infrastructure, therefore deploying High Availability across Availability Zones ensures that there is no shared single-point-of-failure (SPOF) between primary and secondary SAP HANA system. This approach is significantly increasing overall resiliency of the High Availability of the solution.
List of existing Availability Zones for individual Azure Regions is available here: Azure: Azure geographies.
Most critical factor for selecting Availability Zones is network latency. Latency between individual Availability Zones can significantly differ and therefore it is important to measure network latency using SAP niping tool (see SAP Note 500235: Network Diagnosis with NIPING for additional information) and select Availability Zones with minimal latency.
Furthermore, it is important to note that internal numbering of Availability Zones is specific for each individual Azure account. Therefore, the network latency test must be performed in given account. For additional information please see Azure: Regions and Availability Zones in Azure - Availability Zones.
Last thing to consider is whether desired VM sizes are available inside selected Availability Zones as not all Availability Zones are offering all VM sizes.
Azure is using slightly different fencing mechanism based on Linux distribution that is used:
- Red Hat Enterprise Linux (RHEL) is using IPMI-like Fencing (see Module: High Availability - IPMI-like Fencing for additional details) and SBD (Storage Based Death) Fencing is not available
- SUSE Linux Enterprise Server (SLES) is using combination of IPMI-like Fencing and SBD (Storage Based Death) Fencing
Fencing agent source code is available here: fence_azure_arm. Behind the scenes it is using Azure SDK for Python call azure_fence.set_network_state. This will hard stop Azure VM without gracefully stopping the Operating System.
Prerequisites for fencing mechanism to work properly are documented here:
- Azure: Setting up Pacemaker on SUSE Linux Enterprise Server in Azure
- Azure: Setting up Pacemaker on Red Hat Enterprise Linux in Azure
Additional Information:
- Azure: High availability of SAP HANA on Azure VMs on SUSE Linux Enterprise Server
- Azure: High availability of SAP HANA on Azure VMs on Red Hat Enterprise Linux
Traditional implementation of Cluster IP (not applicable to Azure) is covered in section Module: High Availability - Typical Cluster IP Implementation.
Azure Software Defined Networking (SDN) is not supporting ARP cache updates, that are required for traditional implementation of Cluster IP. Therefore, although subnets are stretched across Availability Zones, different mechanism is required. Azure implementation of Cluster IP address is based on Azure Network Load Balancer, that is forwarding all packets sent to Load Balancer IP (third IP address) to the IP address of either primary or secondary server, depending on wherever "health probe port" is active.
This "health probe port" is managed by Pacemaker cluster and is implemented as nc (RHEL) or socat (SLES) command opening dummy port to signal to Load Balancer which VM is currently active (see ClusterLabs / resource-agents / heartbeat / azure-lb). During cluster takeover the Pacemaker cluster will close the "health probe port" on old primary VM and will open it on new primary VM. This will be detected by Azure Load Balancer, which will start forwarding all traffic to new primary VM.
Additional Information:
- YouTube: TUT1134: Microsoft Azure and SUSE HAE When Availability Matters
- PDF: TUT1134: Microsoft Azure and SUSE HAE When Availability Matters
Link to generic content: Module: Disaster Recovery
Disaster Recovery for SAP HANA via SAP HANA System Replication is not infrastructure dependent.
Link to generic content: Module: Data Tiering Options
Following data tiering options are supposed on Azure for traditional Virtual Machines (VMs) and HANA Large Instances (HLI):
| Data Tiering Option | Supported (VMs) | Supported (HLI) |
|---|---|---|
| Persistent Memory (NVRAM) | No | No |
| SAP HANA Native Storage Extensions (NSE) | Yes | Yes |
| SAP HANA Extension Nodes | Yes | Yes |
| SAP HANA Dynamic Tiering (DT) | Yes | No |
Microsoft Azure platform does not offer any instance types having NVRAM that are supported for productive SAP HANA usage.
SAP HANA Native Storage Extensions only need additional disk space compared to regular SAP HANA deployments. Microsoft Azure platform does allow to provision additional disks to both SAP HANA VMs and HANA Large Instances to provide additional capacity into existing filesystems. There is no change to the design of the storage layout.
SAP HANA Extension Node implementation is based on provisioning additional SAP HANA node (with increased storage capacity) to existing SAP HANA system. Result is SAP HANA Scale-Out system where one of the nodes is designated as SAP HANA Extension Node. Storage layout is same as for regular SAP HANA nodes and it is visualized above in section Azure: Storage Setup for SAP HANA Implementation.
Additional Information:
SAP HANA Dynamic Tiering (DT) implementation in Microsoft Azure platform is based on provisioning additional VM for Dynamic Tiering component and connecting it to VM hosting SAP HANA instance, thus forming two-node distributed setup. Storage layout is identical to SAP HANA Scale-out setup as illustrated above in section Azure: Storage Setup for SAP HANA Implementation.
Additional Information:
Link to generic content: Module: SAP XSA
SAP HANA extended application services, advanced model (XSA) component is not infrastructure dependent.