When organisations scale applications on Azure, load balancing is often treated as a technical necessity rather than a strategic decision. But the way traffic is distributed has a direct impact on availability, security, performance, and long-term cloud costs.
This was exactly the challenge one of our customers faced.
They were running a microservices-based application on Azure virtual machines, supporting both web-facing and non-web workloads. Their priorities were clear:
- High availability across services
- Predictable performance under load
- A design that could scale without unnecessary cost or complexity
Our task was to design a load-balancing architecture that met these goals without over-engineering the platform or inflating Azure spend.
Why load balancing strategy matters in Azure
From a decision-maker’s perspective, load balancing affects far more than traffic routing:
- Poor designs increase downtime risk
- Overlapping services drive up cost
- Inflexible architectures limit future growth
Azure offers multiple load balancing services, each optimised for different traffic types. Using the wrong one – or forcing a single service to do everything – often results in higher cost and lower reliability.
In this case, a single solution would not have delivered the performance or resilience the customer needed.
Why we use two Azure load balancing services instead of one?
The customer’s platform supported two fundamentally different workload types:
Non-web workloads
- Custom TCP-based services
- High throughput requirements
- No need for HTTP inspection or application-level routing
Web-facing workloads
- HTTP and HTTPS traffic
- Requirement for SSL offloading
- Need for routing rules and web application firewall (WAF) protection
Trying to force both through a single Layer 7 service would have added unnecessary overhead and cost. Instead, we implemented a dual-layer architecture:
- Azure Load Balancer (Layer 4) for TCP-based workloads
- Azure Application Gateway (Layer 7) for HTTP/HTTPS traffic
This ensured each workload type was handled by the most efficient service available.
How Azure Load Balancer improved performance for non-web workloads
For the customer’s TCP-based services, we deployed Azure Load Balancer to provide fast, low-latency traffic distribution.
Key configuration elements:
- Frontend IP configurations to receive inbound TCP connections
- Backend pools built on Azure VM Scale Sets
- Load balancing rules mapping frontend and backend ports
Because Azure Load Balancer operates at Layer 4, traffic is passed directly to backend instances without application-level inspection. This keeps latency low and avoids unnecessary processing costs.
High availability through health probes
We implemented TCP health probes to continuously validate VM availability. If an instance failed a probe, traffic was automatically redirected to healthy instances.
Combined with Azure Metrics monitoring, this gave the customer real-time visibility into service health and confidence that failures would be handled automatically – even outside business hours.
How Azure Application Gateway handled web traffic securely and efficiently
For HTTP and HTTPS workloads, we deployed Azure Application Gateway, giving the customer advanced traffic management and security controls.
Configuration highlights:
- Listeners on ports 80 and 443
- SSL/TLS termination to offload encryption from backend VMs
- Backend HTTP/HTTPS settings for controlled communication
Intelligent routing and health checks
We implemented custom health probes that validated actual application endpoints (for example, /healthcheck) rather than simply checking server responsiveness. This ensured routing decisions were based on application health, not just infrastructure availability.
Path-based routing was also configured:
/imagestraffic routed to one backend pool/apitraffic routed to another
This separation improved performance isolation and reduced the blast radius of potential issues.
Built-in security
Application Gateway enabled the use of Web Application Firewall (WAF) capabilities, protecting public-facing services without introducing additional third-party tooling or operational overhead.
How we decide between Azure Load Balancer and Application Gateway
A common question we hear from cloud leaders is: “Which Azure load balancing service should we use?”
Our decision framework is simple and repeatable:
- Use Azure Load Balancer when:
Traffic is TCP or UDP
Performance and low latency are the priority
No application-level inspection is required
Use Azure Application Gateway when:
Traffic is HTTP or HTTPS
SSL offloading is needed
You require WAF, path-based routing, or application-aware health checks
Applying this framework creates clear operational boundaries, reduces cost overlap, and keeps architectures scalable as platforms evolve.
The outcome: resilient, secure, and future-ready
By combining Azure Load Balancer and Azure Application Gateway, we delivered an architecture that is:
Resilient – traffic is automatically distributed across healthy instances
Secure – web workloads benefit from SSL offload and WAF protection
Scalable – new services and traffic patterns can be added without redesign
Most importantly, the customer now has a load balancing strategy that supports growth without driving unnecessary cloud spend.
Talk with an Azure specialist
Choosing the right Azure load balancing setup isn’t about ticking boxes – it’s about aligning technical design with business priorities like uptime, cost control, and operational simplicity.
Cloud Elemental is an AWS Advanced Tier Partner with experience delivering cloud migration and modernisation programmes across cloud platforms for organisations operating regulated, mission-critical environments. Our approach focuses on creating cloud foundations that are resilient, secure, and optimised to support advanced workloads, including AI, whilst maintaining continuity throughout the project.
If you’re unsure whether your current Azure architecture is optimised for performance and resilience, speak with one of our migration specialists today.
