Why the edge
Application security is usually discussed as a pipeline problem, and it is one. But the last control before the internet is not in the pipeline. It sits in the infrastructure: at the load balancers, in front of every web service and API, across whatever mix of Kubernetes clusters, hybrid cloud and on-premises servers an estate has accumulated. How that control is chosen, tuned and run says more about an organisation's infrastructure security than any hardening checklist.
This is an account from our own estate. We name the product because a vague story teaches nothing: Check Point's open-appsec, a machine-learning web application firewall, deployed on our load balancers and run for a year.
A year at the edge, in one view
Twelve months, 900 million HTTP requests, one control at the load-balancer layer across cloud, containers and on-premises servers.
- Scale for sudden traffic spikes with minimal latency
- Adaptation to new endpoints without manual rules
- Integration at the load balancer with no application change
- Baseline before you block
- Custom APIs still need some hand-written rules
- A control whose logs never reach the SIEM is half a control
- Hardening is a lifecycle: observe, tune, enforce, review
What we needed
Three things, in this order. Scale: large volumes and sudden spikes without measurable latency. Adaptation: new endpoints and new parameters appear every release, and a control that needs a rule written for each one falls behind within a quarter. Integration: it had to sit at the load-balancing layer with minimal or no change to the applications behind it.
The choice of a machine-learning engine followed from the second requirement. Static rule sets cover the known; the estate keeps producing the unknown.
How we rolled it out
We started in a test environment shaped by real traffic, then ran the control in alert-only mode in production for a learning period of weeks to months, depending on request volumes, so the engine could establish what normal looked like. Only then did we tune policies against the logs and phase enforcement in, service by service.
The learning period is the step teams skip, and it is the step that decides whether a WAF blocks attackers or blocks customers.
Baseline before you block.
What it stopped
Out of the box the rule sets covered the OWASP Top 10, and over the year the control consistently blocked SQL injection attempts, cross-site scripting, directory traversal and command injection. The machine-learning layer earned its place on the rest: attack techniques not yet in any static rule set, detected from how the traffic diverged from the baseline.
Running it day to day
Dashboards gave a granular view of threat types, source addresses and blocked requests, and every suspicious request raised an alert into our incident response process. The more valuable decision was forwarding the logs to our SIEM, where WAF events could be correlated with system and network data. During troubleshooting and incident response that correlation, and the forensic detail behind each blocked request, guided root-cause analysis and the next round of rule refinement.
Five lessons
Baseline before you blockAlert-only mode during the learning period prevented the accidental blocking of legitimate requests that gives WAFs a bad name.
Expect a few manual rulesHighly customised APIs and microservices still needed some hand-written rules. Far fewer than a static WAF would demand, but not zero.
False positives are tuning dataThe handful we saw early, mostly on new API endpoints, dropped sharply once the engine had observed more real traffic. Treat each one as a signal, not a failure.
A control whose logs never reach the SIEM is half a controlBlocking is the visible half. Correlation and evidence are the half that improves detection everywhere else.
Hardening is a lifecycleObserve, tune, enforce, review. The same discipline applies to segmentation, host baselines, cloud configuration and every other infrastructure control. None of them is finished at installation.
One year on
After twelve months of continuous, large-scale operation, more than 900 million HTTP requests across all our environments, the control proved scalable, with minimal performance overhead; accurate and adaptive, with false positives falling as the engine learned; and easy to maintain, with automated policy learning doing most of the work our team would otherwise have done by hand.
The wider point for infrastructure security is not the product. It is that a control at the edge or in the estate is only as strong as the operating discipline around it. An installed control is a promise. A run control is evidence.
An installed control is a promise. A run control is evidence.
Krishna works at the intersection of cyber security, infrastructure, AI and enterprise risk, with a focus on engineering trustworthy and resilient technology foundations.
Follow on LinkedInThis perspective expands an article and a post first published by the author on LinkedIn.
