Solution: TLS Works From Some Clients But Fails From Others¶

Triage¶

Test the TLS handshake and inspect the certificate chain:
```
openssl s_client -connect app.example.com:443 -showcerts </dev/null 2>&1
```
Count the certificates returned. If only one (the leaf), the chain is incomplete.

Check the leaf certificate's issuer:

openssl s_client -connect app.example.com:443 </dev/null 2>&1 | \
  openssl x509 -noout -issuer -subject

The issuer should match the subject of the next certificate in the chain.

Verify with curl (strict client):
```
curl -vI https://app.example.com
```
Look for: "SSL certificate problem: unable to get local issuer certificate"
Test with an external checker: https://www.ssllabs.com/ssltest/ Look for "Chain issues: Incomplete" in the report.

Check the server's certificate configuration file:

# Nginx
cat /etc/nginx/sites-enabled/app.conf | grep ssl_certificate
# Apache
grep SSLCertificateChainFile /etc/apache2/sites-enabled/app.conf

Root Cause¶

During the recent certificate renewal, only the leaf (server) certificate was installed on the web server. The intermediate certificate from the CA was not included in the certificate bundle file.

TLS requires the server to send the full chain (leaf + intermediates) during the handshake so the client can verify the trust path to a root CA. Without the intermediate, clients cannot build the chain.

Desktop browsers work around this because: - They cache intermediate certificates from previous connections to any site signed by the same CA. - They support AIA (Authority Information Access) fetching, where the browser reads the AIA URL from the leaf cert and downloads the intermediate on the fly.

Strict clients like curl, Python requests, and many mobile browsers do NOT support AIA fetching and require the server to provide the complete chain.

Fix¶

Download the correct intermediate certificate from the CA:
```
wget https://crt.sh/?d=INTERMEDIATE_CERT_ID -O intermediate.pem
```
Or obtain it from the CA's documentation.
Create the full chain bundle (leaf first, then intermediate):
```
cat server.crt intermediate.pem > fullchain.pem
```

Update the web server configuration:

# Nginx
ssl_certificate /etc/ssl/certs/fullchain.pem;
ssl_certificate_key /etc/ssl/private/server.key;

# Apache
SSLCertificateFile /etc/ssl/certs/server.crt
SSLCertificateChainFile /etc/ssl/certs/intermediate.pem

Reload the web server:
```
nginx -t && systemctl reload nginx
```

Verify the fix:

openssl s_client -connect app.example.com:443 -showcerts </dev/null 2>&1
curl -vI https://app.example.com

Rollback / Safety¶

Reload (not restart) the web server to avoid dropping active connections.
Test with nginx -t or apachectl configtest before reloading.
Keep the old certificate file as backup.
The fix is non-disruptive; clients will see the full chain on next connection.

Common Traps¶

Including the root CA certificate in the bundle -- this is unnecessary and wastes bandwidth; clients already have roots in their trust store.
Wrong order in the bundle file -- it must be leaf first, then intermediate(s).
Assuming browsers = all clients. Browsers are forgiving; APIs and scripts are not.
Using Let's Encrypt and forgetting to use fullchain.pem instead of cert.pem.
Not verifying after the fix -- always test with a strict client like curl.
Multiple intermediate certificates needed (cross-signed CAs) -- check the full chain path, not just one intermediate.