Standard Operating Procedure (SOP)
ADC Load Balancer Operations – West US3, Central US, East US
1. Purpose
This SOP outlines procedures for operational management, monitoring, and maintenance of Application Delivery Controller (ADC) Load Balancers in the West US3, Central US, East US Azure regions, including both internal and external LB types for Epic environments.
2. Scope
Applies to all cloud operations, network, and security teams responsible for ADC Load Balancers.
- if LB contains lbi naming convention --> Internal LB
- if LB contains lbe naming convention --> External LB
Example Code Structure Explanation
3. Resource Group and Diagnostics
| Load Balancer Name | Resource Group | Diagnostics Enabled |
|---|
| ohemr-lbi-west_epic_adc_lb-npd-wus3-001 | ohemr-rg-west_epic_adc_lb-npd-wus3-001 | Yes |
| ohemr-lbe-west_dmz_adc_cs-prd-wus3-001 | ohemr-rg-dmz_netscaler-wus3-001 | Yes |
4. Internal ADC Load Balancer name: ohemr-lbi-west_epic_adc_lb-npd-wus3-001
a. Frontends, Internal LB can contains n number of frontends based on Private IP Addresses
| Name | Subnet | IP Address | Availability Zones |
|---|
| ohemr-lbi-west_epic_adc_lb-npd-wus3-001-fe | ohemr-snet-west_epic_adcmgmt-npd-wus3-001 | 0.0.0.166 | 1,2,3 |
| ohemr-lbi-west_epic_adc_lb-npd-wus3-001-fe-001 to -024 | ohemr-snet-west_epic_adcfe_untrust-npd-wus3-001 | 0.0.0.135–158 | 1,2,3 |
b. Backend Pools
- Pool:
ohemr-lbi-west_epic_adc_lb-npd-wus3-001-be
- Managed by rules; ensure all backend members are healthy.
c. Probes
| Probe Name | Protocol | Port | Interval (s) | Number of Probes | Request Path |
|---|
| ohemr-lbi-west_epic_adc_lb-npd-wus3-001-probe | TCP | 9000 | 15 | 1 | (none) |
d. Load Balancing Rules
- n rules, each mapping a frontend to the backend pool.
- Protocol: TCP/UDP
- Port: 443/3009/53 (frontend/backend)
- Idle Timeout: 4 min (based on requirement)
- Floating IP: Enabled/Disabled
- Load Distribution: Default
- Each rule uses the probe for health checks.
5. External ADC Load Balancer Name: ohemr-lbe-west_dmz_adc_cs-prd-wus3-001
a. Frontends, External LB can contains n number of frontends based on Public IP Addresses
| Name | Public IP Resource | Gateway LB | Gateway RG |
|---|
| ohemr-lbe-west_dmz_adc_cs-prd-wus3-fe-001…004 | ohemr-lbe-west_dmz_adc_cs-prd-wus3-fe-00x | ohemr-wus3-prod-ig-untrust-gwlb | ohemr-rg-core_fw-shared-wus3-001 |
b. Backend Pools
| Name | Type |
|---|
| ohemr-lbe-west_dmz_adc_cs-prd-wus3-be-001 | external |
c. Probes
| Probe Name | Protocol | Port | Interval (s) | Number of Probes | Request Path |
|---|
| ohemr-lbe-west_dmz_adc_cs-prd-wus3-probe-001 | TCP | 9000 | 15 | 1 | (none) |
d. Load Balancing Rules
- rules, each connecting a frontend to the backend pool.
- Protocol: TCP/UDP
- Port: 443/3009/53 (frontend/backend)
- Idle Timeout: 4 min (based on requirement)
- Floating IP: Enabled/Disabled
- Load Distribution: Default
- Type: External
- Each rule uses the probe for health checks.
6. Operational Procedures
a. Monitoring & Diagnostics
- Ensure diagnostics are enabled for all ADC LBs.
- Monitor health probes and verify backend endpoints' health.
- Review probe logs and alerts daily.
b. Change Management
- All modifications must be approved via standard change control.
- Document changes to frontends, backends, rules, or probes.
c. High Availability
- Frontends and backend pools are distributed across Availability Zones 1, 2, and 3 for resilience.
- Regularly test failover scenarios.
d. Security
- For external LBs, confirm public IPs are assigned from approved resource groups.
- Validate floating IP and load distribution settings for compliance.
- Review network security rules for subnets hosting LB frontends.
e. Backup and Restore
- Backup configuration before major changes.
- Document restoration steps for LB objects.
f. Incident Management
- In case of probe failure or backend unavailability, escalate to cloud operations and network team.
- Record all incidents and remediation actions.
7. Audit & Review
- Quarterly review of LB configuration, frontends, backend pools, probes, and rules.
- Annual review and update of SOP.
8. Contacts
| Role | Name/Email (Example) |
|---|
| ADC Owner | [add contact] |
| Operations Contact | [add contact] |
| Network Team | [add contact] |
"ilbs": {
"ohemr-lbi-west_epic_adc_lb-npd-wus3-001": {
"rg": "ohemr-rg-west_epic_adc_lb-npd-wus3-001",
"diagnostics": true,
"frontends": {
"ohemr-lbi-west_epic_adc_lb-npd-wus3-001-fe": {
"subnet": "ohemr-snet-west_epic_adcmgmt-npd-wus3-001",
"ip_address": "10.150.204.166",
"availability_zone": [
"1",
"2",
"3"
]
},
"ohemr-lbi-west_epic_adc_lb-npd-wus3-001-fe-001": {
"subnet": "ohemr-snet-west_epic_adcfe_untrust-npd-wus3-001",
"ip_address": "10.150.204.135",
"availability_zone": [
"1",
"2",
"3"
]
}
},
"backends": {
"ohemr-lbi-west_epic_adc_lb-npd-wus3-001-be": {}
},
"probes": {
"ohemr-lbi-west_epic_adc_lb-npd-wus3-001-probe": {
"probe_protocol": "Tcp",
"probe_port": "9000",
"request_path": "",
"interval_in_seconds": "15",
"number_of_probes": "1"
}
},
"rules": {
"ohemr-lbi-west_epic_adc_lb-npd-wus3-rule-001": {
"probe": "ohemr-lbi-west_epic_adc_lb-npd-wus3-001-probe",
"rule_protocol": "Tcp",
"idle_timeout_in_minutes": "4",
"enable_floating_ip": true,
"frontend_port": "443",
"backend_port": "443",
"load_distribution": "Default",
"frontend": "ohemr-lbi-west_epic_adc_lb-npd-wus3-001-fe",
"backend": "ohemr-lbi-west_epic_adc_lb-npd-wus3-001-be"
},
"ohemr-lbi-west_epic_adc_lb-npd-wus3-rule-002": {
"probe": "ohemr-lbi-west_epic_adc_lb-npd-wus3-001-probe",
"rule_protocol": "Tcp",
"idle_timeout_in_minutes": "4",
"enable_floating_ip": true,
"frontend_port": "443",
"backend_port": "443",
"load_distribution": "Default",
"frontend": "ohemr-lbi-west_epic_adc_lb-npd-wus3-001-fe-001",
"backend": "ohemr-lbi-west_epic_adc_lb-npd-wus3-001-be"
}
}
},
"ohemr-lbe-west_dmz_adc_cs-prd-wus3-001": {
"rg": "ohemr-rg-dmz_netscaler-wus3-001",
"diagnostics": true,
"type": "external",
"frontends": {
"ohemr-lbe-west_dmz_adc_cs-prd-wus3-fe-001": {
"public_ip": "ohemr-lbe-west_dmz_adc_cs-prd-wus3-fe-001",
"gateway_lb": "ohemr-wus3-prod-ig-untrust-gwlb",
"gateway_rg": "ohemr-rg-core_fw-shared-wus3-001"
}
},
"backends": {
"ohemr-lbe-west_dmz_adc_cs-prd-wus3-be-001": {
"type": "external"
}
},
"probes": {
"ohemr-lbe-west_dmz_adc_cs-prd-wus3-probe-001": {
"probe_protocol": "Tcp",
"probe_port": "9000",
"request_path": null,
"interval_in_seconds": "15",
"number_of_probes": "1"
}
},
"rules": {
"ohemr-lbe-west_dmz_adc_cs-prd-wus3-rule-001": {
"probe": "ohemr-lbe-west_dmz_adc_cs-prd-wus3-probe-001",
"type": "external",
"rule_protocol": "Tcp",
"idle_timeout_in_minutes": "4",
"enable_floating_ip": true,
"frontend_port": "443",
"backend_port": "443",
"load_distribution": "Default",
"frontend": "ohemr-lbe-west_dmz_adc_cs-prd-wus3-fe-001",
"backend": "ohemr-lbe-west_dmz_adc_cs-prd-wus3-be-001"
}
}
}
}