nginx throws away inherited headers

On 15 August I went to put security headers on the six sites running off one VPS. Straightforward job, an hour at most.

The finding I started from was wrong, the fix I was about to write was wrong, and both were wrong in ways the config file will never tell you about. So this is a short account of an easy afternoon that turned out not to be.

The site that already had them

My own sweep reported that alphaitsolutions.uk was serving none of the four headers I was adding. It had been serving all four for months.

Two separate mistakes stacked on top of each other, which is why it looked convincing.

The first was the grep. I searched the nginx config for add_header. The headers were there, in /etc/nginx/ait-security-headers.conf, pulled in with include. A file that exists, is loaded, and works, and does not contain the string I was looking for.

The second is the one worth carrying around. I tested the apex domain, and the apex 301s to www. A 301 response carries no headers. Not the ones you set, anyway. So the check fetched a redirect, found a bare response, and reported a gap that did not exist.

Test the URL that returns 200. Not the one you type.

The other five sites were genuinely bare, so there was still a job to do. It was just a smaller and more embarrassing one than the report said.

The rule that eats your config

Here is the part that would have shipped broken.

The obvious way to do this is one add_header block at the server level and let everything under it inherit. That is how nearly every tutorial writes it. It is also how you end up with headers on your images and none on your HTML.

nginx discards every inherited add_header the moment a child location sets one of its own. Not merges. Discards, all of them, and it does it silently.

Several of these vhosts have a location / that sets Cache-Control. One directive, for a completely unrelated reason, written years earlier by me. That block would have thrown away the entire inherited set for exactly the requests that matter most, the HTML, while every static asset underneath a different location kept them. A curl of a PNG would have looked perfect.

So the include goes into each location that sets any header of its own. Eight of them on alphaitsolutions, four on temporra, two on schoolbooking.

location / {
    include /etc/nginx/security-headers.inc;
    add_header Cache-Control "no-cache";
    ...
}

That is uglier than one block at the top and it is the only version that is true. The standing rule I wrote down for future me: any new location that sets add_header has to include the file too, or it silently opts out.

Configs went to /root/nginx-headers-backup-20260815/ first, and I checked twelve URLs afterwards including every sw.js and robots.txt, because service workers and robots files live in their own locations often enough to be worth the extra fetches.

Two of everything

The first pass produced its own problem. For a few minutes alphaitsolutions.uk served every header twice, with two different max-age values on the HSTS one.

I had added the new include without removing the old ait-security-headers.conf I had failed to find in the first place. add_header does not overwrite; it appends. The browser gets two, and which one wins is not a question you want to be asking about HSTS. The old file is now an empty pointer comment, included nowhere.

The CSP I did not enforce

Content-Security-Policy is where this gets genuinely risky, because a wrong one does not throw an error anybody sees. It just stops something loading.

alphaitsolutions.uk is enforcing. It has no payment flow, no map tiles and nothing I cannot exercise myself from a browser, so I could walk the whole site with the console open and watch it produce zero violations, Three.js scene and contact API included.

The five app SPAs are Content-Security-Policy-Report-Only. They carry live Stripe checkout, an OpenStreetMap tile layer, camera and upload flows. I cannot test a real card payment from outside, and a wrong CSP fails at a customer’s payment screen, at the worst possible moment, with nothing in any log I own.

Report-Only earned its keep in about ten minutes. It flagged connect.facebook.net/en_US/fbevents.js on build-estimate.app, which I had genuinely forgotten was there. Enforcing on the first pass would have killed it dead with no visible symptom at all. It is allowed in all four app policies now, because I found it by reading reports rather than by somebody telling me a thing had stopped working three weeks later.

One catch. Never ship a first CSP as enforcing.

To promote one later, the drill is to walk a real checkout, a map view and a photo upload with the console open, then change only the header name in that site’s csp-*.inc. Same policy, one word different.

The console lies for a while

Worth knowing before it costs you an hour, as it cost me a good chunk of one.

After I updated a policy, the browser kept reporting violations against the old script-src. A fresh tab did not clear it. The HTML was cached with its original headers, and the violation messages were being generated against what that cached response said, not against what the server was now sending.

Confirm from outside the browser with curl -sI, or in the page itself:

fetch(url, {cache: 'no-store'})
  .then(r => r.headers.get('content-security-policy-report-only'))

I spent a while editing a policy that was already correct.

What I left off on purpose

Three omissions, all deliberate, all worth stating because they look like gaps.

HSTS has no includeSubDomains. That directive binds every subdomain for a year with no quick way out, and I could not say with confidence that every subdomain on that box was HTTPS-only. Committing to something for twelve months on a hunch is not a security improvement.

No Permissions-Policy at all. Temporra needs geolocation and camera for GPS clock-in and work photos, KitDesk needs camera for damage photos and signatures. A blanket restrictive policy would have broken real features on two live products to make a header look tidier.

And the client portal’s /api and /login locations get nothing from me. helmet already sets a full set there in the application. Two content security policies do not merge into the looser one, they enforce the intersection, so adding mine on top would have quietly tightened the portal until something in it stopped working. Its static locations get csp-portal.inc and the application keeps its own.

What this is actually about

Every one of these is invisible. The config parses. nginx -t is happy. The site loads. You would never find any of it by looking at whether the change had been made, because the change had been made.

That is most of what a security audit turns up on a system somebody competent already looks after. Not a gaping hole. A thing that is switched on, and one interaction away from doing nothing, like the MongoDB with no authentication behind a firewall rule on the same box.

If you want somebody to go and check what your server is actually sending, rather than what its config says it should be, that is the audit. It is read-only and the evidence comes with it.

Start with one command, on the URL that returns 200 rather than the one you typed:

curl -sI https://example.com/ | grep -i 'strict-transport\|content-security\|x-frame'

If that comes back empty on the HTML and full on a stylesheet, you have the same problem I nearly shipped.

That is also why the audit I sell reads rather than assumes, and why every finding it produces carries the value it was read from.