LDAP & Identity Management - Street-Level Ops

Quick Diagnosis Commands

When users cannot log in and the phone is ringing:

# 1. Is SSSD running?
systemctl status sssd

# 2. Can we resolve the user at all?
id username
getent passwd username

# 3. Check SSSD logs for errors
tail -100 /var/log/sssd/sssd_example.com.log

# 4. Is the LDAP server reachable?
ldapsearch -x -H ldaps://ldap.example.com -b "dc=example,dc=com" -s base

# 5. Can we bind (authenticate) to LDAP?
ldapwhoami -x -H ldaps://ldap.example.com -D "uid=alice,ou=People,dc=example,dc=com" -W

# 6. Check Kerberos ticket status
klist

# 7. Is the PAM stack configured correctly?
cat /etc/pam.d/sshd

# 8. DNS resolution for KDC/LDAP
dig _ldap._tcp.example.com SRV
dig _kerberos._tcp.example.com SRV

---

Pattern: ldapsearch Queries

ldapsearch is the Swiss Army knife for LDAP debugging.

# Basic anonymous search
ldapsearch -x -H ldap://ldap.example.com -b "dc=example,dc=com" "(uid=alice)"

# Authenticated search
ldapsearch -x -H ldaps://ldap.example.com \
    -D "cn=admin,dc=example,dc=com" -W \
    -b "dc=example,dc=com" "(uid=alice)"

# Search for all users (limit to 10)
ldapsearch -x -H ldaps://ldap.example.com \
    -b "ou=People,dc=example,dc=com" \
    -z 10 "(objectClass=posixAccount)" uid cn mail

# Find user by email
ldapsearch -x -H ldaps://ldap.example.com \
    -b "dc=example,dc=com" "(mail=alice@example.com)" dn

# Find groups a user belongs to
ldapsearch -x -H ldaps://ldap.example.com \
    -b "ou=Groups,dc=example,dc=com" \
    "(memberUid=alice)" cn

# Count entries
ldapsearch -x -H ldaps://ldap.example.com \
    -b "dc=example,dc=com" "(objectClass=posixAccount)" dn | grep "^dn:" | wc -l

# Show all attributes for an entry
ldapsearch -x -H ldaps://ldap.example.com \
    -b "dc=example,dc=com" "(uid=alice)" '*' '+'

# Search with a specific scope
ldapsearch -x -s one -b "ou=People,dc=example,dc=com" "(objectClass=*)" dn
# Scopes: base, one, sub (default)

---

Pattern: SSSD Cache Management

SSSD caches identity data aggressively. Stale cache is the most common "user exists but cannot log in" cause.

# Clear ALL caches and restart
systemctl stop sssd
rm -rf /var/lib/sss/db/*
rm -rf /var/lib/sss/mc/*
systemctl start sssd

# Clear cache for a specific user (less disruptive)
sss_cache -u username

# Clear cache for a specific group
sss_cache -g groupname

# Clear everything without deleting files
sss_cache -E

# Check cache status
sss_cache --help  # see all options

# Verify user is resolved after cache clear
getent passwd username

When to Clear Cache

• User was just added to LDAP but id username returns nothing
• User was added to a group but groups username does not show it
• User password was changed but old password still works
• LDAP server was restored from backup

---

Pattern: PAM Debugging

# Enable PAM debug logging
# Add 'debug' to relevant pam module lines:
# auth required pam_sss.so debug

# Check auth log for PAM messages
tail -f /var/log/auth.log        # Debian/Ubuntu
tail -f /var/log/secure          # RHEL/CentOS

# Test PAM authentication without SSH
pamtester sshd username authenticate

# Check which PAM modules are loaded
ldd /usr/lib/x86_64-linux-gnu/security/pam_sss.so

# Verify PAM config syntax (no tool for this — review manually)
cat /etc/pam.d/sshd
cat /etc/pam.d/common-auth        # Debian/Ubuntu
cat /etc/pam.d/system-auth        # RHEL/CentOS

PAM Flow Debugging Checklist

1. Which PAM service file is used? (/etc/pam.d/sshd for SSH)
2. Does it include common files? (@include common-auth)
3. What modules are in the auth stack? (pam_unix, pam_sss)
4. What control flags? (required, sufficient, etc.)
5. Are there any "sufficient" modules that short-circuit?
6. Is pam_deny.so at the end as a safety net?

---

Gotcha: SSSD Log Levels

SSSD default logging shows almost nothing. Crank it up during debugging:

# /etc/sssd/sssd.conf
[domain/example.com]
debug_level = 6

[nss]
debug_level = 6

[pam]
debug_level = 6

# Restart after changing debug level
systemctl restart sssd

# Logs land in /var/log/sssd/
ls -la /var/log/sssd/
# sssd.log             main daemon
# sssd_example.com.log domain-specific (most useful)
# sssd_nss.log         name service switch
# sssd_pam.log         PAM related

# Remember to turn debug back down after troubleshooting
# High debug levels generate massive logs

---

Gotcha: AD Join Failures

Common reasons realm join fails:

# DNS is wrong (most common)
dig _ldap._tcp.ad.example.com SRV
# If this fails, fix DNS first.

# Time is wrong (Kerberos needs <5 min skew)
timedatectl
chronyc sources

# Missing packages
yum install sssd realmd oddjob oddjob-mkhomedir adcli samba-common-tools

# Firewall blocking LDAP/Kerberos ports
# Required ports: 88/tcp+udp (Kerberos), 389/tcp (LDAP), 636/tcp (LDAPS),
# 3268/tcp (Global Catalog), 53/tcp+udp (DNS)
ss -tlnp | grep -E '88|389|636'

# Computer account already exists in AD
# Delete it from AD or use --computer-ou to place it elsewhere
realm join ad.example.com --computer-ou="OU=LinuxServers,DC=ad,DC=example,DC=com"

---

Pattern: Testing Authentication End-to-End

# Step 1: Can we resolve the user?
getent passwd testuser
# Expected: testuser:*:50001:50001:Test User:/home/testuser:/bin/bash

# Step 2: Can we resolve the group?
getent group testgroup
# Expected: testgroup:*:50001:testuser

# Step 3: Can we authenticate? (Kerberos)
kinit testuser@EXAMPLE.COM
klist

# Step 4: Can SSH work?
ssh -v testuser@localhost 2>&1 | grep -i 'auth\|pam\|sssd'

# Step 5: Can sudo work? (if configured via LDAP/IPA)
sudo -l -U testuser

---

Gotcha: Plaintext LDAP Bind Credentials

# WRONG: unencrypted LDAP connection
ldap_uri = ldap://ldap.example.com

# RIGHT: encrypted with LDAPS
ldap_uri = ldaps://ldap.example.com

# RIGHT: encrypted with STARTTLS
ldap_uri = ldap://ldap.example.com
ldap_id_use_start_tls = true

If you use ldap:// (not ldaps://) without STARTTLS, credentials flow in plaintext. Any network sniffer picks them up. This includes the bind DN password in sssd.conf.

---

Pattern: Checking Certificate Issues

TLS certificate problems are a top cause of LDAP connection failures.

# Test TLS connection to LDAP server
openssl s_client -connect ldap.example.com:636 -showcerts

# Check if the CA cert is trusted
openssl verify -CAfile /etc/pki/tls/certs/ca-bundle.crt /path/to/ldap-cert.pem

# In sssd.conf, control certificate verification:
ldap_tls_reqcert = demand    # verify cert (production)
ldap_tls_reqcert = allow     # accept any cert (debugging only!)
ldap_tls_reqcert = never     # skip TLS entirely (NEVER in production)

# If using a custom CA:
ldap_tls_cacert = /etc/pki/tls/certs/custom-ca.pem

---

Gotcha: Home Directory Not Created on First Login

User authenticates but lands in / with a broken shell experience because /home/username does not exist.

# Fix: Enable pam_mkhomedir
# In /etc/pam.d/common-session (Debian) or /etc/pam.d/system-auth (RHEL):
session required pam_mkhomedir.so skel=/etc/skel/ umask=077

# For SSSD, also configure in sssd.conf:
[domain/example.com]
override_homedir = /home/%u
fallback_homedir = /home/%u

# For systems using oddjob-mkhomedir (RHEL):
systemctl enable --now oddjobd
authselect enable-feature with-mkhomedir

Pages that link here

• Anti-Primer: LDAP Identity
• LDAP & Identity Management