Claude Prompts - tailwatchd Segfault Troubleshooting¶

Problem¶

tailwatchd service producing segfaults every 5 minutes:

segfault at 24 ip 00002af7c6994a6f sp 00007ffdf3c085c0 error 4 in libperl.so[2af7c6896000+30f000]

Error 4 = page fault on read from userspace — NULL pointer dereference reading an offset off a NULL struct/object pointer. Fault is inside libperl.so, so this is a Perl interpreter crash.

Diagnosis¶

Every PID is different, exact 5-minute interval¶

tailwatchd was assumed to be crashing on startup and being respawned by chkservd. The identical IP/offset every time (00002af7c6994a6f) suggested it always hits the same code path.

grep -i 'tailwatchd\|segfault' /var/log/messages | tail -30

Check cPanel error log¶

grep -i 'tailwatch\|error\|fatal' /usr/local/cpanel/logs/error_log | tail -30

Key finding: restartsrv_tailwatchd was called by the maintenance script during the daily cPanel update run, then immediately segfaulting — suggesting a partial update left Perl modules in an inconsistent state.

tailwatchd_log was completely empty¶

tailwatchd crashes before writing a single line — dying during Perl interpreter initialization, before any application code runs.

Segfault trigger identified¶

The segfault at 16:11:07 happened at the exact same second as the FTP auth service check (__cpanel__service__auth__ftpd__). This is chkservd doing its service check round, which also spawns tailwatchd. The 5-minute interval matches chkservd's cycle exactly.

# Correct binary location
/usr/local/cpanel/libexec/tailwatchd --status
/usr/local/cpanel/libexec/tailwatchd --version=long

strace Analysis¶

Running strace revealed tailwatchd was actually running fine — processing logs normally (maillog, exim_mainlog). The segfault was happening in a long-running instance, not on startup.

Smoking gun¶

bind(113, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("210.139.2.106")}, 16) = -1 EADDRNOTAVAIL (Cannot assign requested address)

The driver was trying to bind to client source IPs from mail auth log entries as local source addresses. When those IPs aren't assigned to the server, bind fails with EADDRNOTAVAIL. The error handling doesn't null-check the return value → deref of 0x24 (offset into a NULL struct) → segfault.

Parse all failing IPs from strace¶

grep 'EADDRNOTAVAIL' /tmp/tw_strace.txt | grep -oP 'inet_addr\("\K[^"]+' | sort | uniq -c | sort -rn

Root Cause¶

Cpanel::TailWatch::RecentAuthedMailIpTracker driver reads the remote client IP from mail auth log entries and incorrectly passes it to bind() as the local source address. This is a cPanel bug.

Confirm active drivers¶

/usr/local/cpanel/libexec/tailwatchd --status 2>&1

Output showing the offending driver:

Driver (Active: 1) Cpanel::TailWatch::RecentAuthedMailIpTracker

Fix¶

Disable the offending driver:

/usr/local/cpanel/libexec/tailwatchd --disable=Cpanel::TailWatch::RecentAuthedMailIpTracker

Monitor for segfaults:

tail -f /var/log/messages | grep -i 'segfault\|tailwatch'

Report to cPanel support with the strace output as evidence — they need to patch RecentAuthedMailIpTracker.pm to not bind client IPs as local source addresses.

# Include cPanel version in ticket
cat /usr/local/cpanel/version

cPanel NodeJS App Selector — Next.js Breaks SSL DCV (ACME Challenge 404)¶

Cause¶

ProxyPass / in include.conf intercepts all traffic including /.well-known/acme-challenge/, proxying it to the Node port (3000) which returns 404 — blocking HTTP DCV.

Fix¶

Add exclusion before the catch-all in both ssl and non-ssl include.conf:

ProxyPass /.well-known/ !

Full path: /etc/apache2/conf.d/userdata/ssl/2_4/<user>/<domain>/include.conf

/scripts/rebuildhttpdconf && systemctl reload httpd

Then re-run: /usr/local/cpanel/bin/checkallsslcerts --verbose