A Mail for Every SSH Login: A Two-Minute Tripwire with wtmp and systemd

🌐 Also available in: Deutsch · Français · Español

This is a follow-up to yesterday’s long post, Defense in Depth: Nine Layers Between the Internet and My Servers. There, the login alarm got one sentence. Here is how it works, why it is built this way – and what an attacker learns from reading this.

The idea is old and simple: if anyone logs in to one of my servers, I want to know – within minutes, not when I happen to look at a log a week later. Every login mails me. Since I know when I logged in myself, every mail I did not expect is worth looking at immediately. It is a tripwire: it does not stop anyone, it makes sure an intrusion does not go unnoticed.

There were three requirements. It must never get in the way of a login – a security feature that locks me out of my own server is worse than none. It must stay quiet for everything automated, otherwise it trains me to ignore it. And it must work the same way on all three servers, deployed by Ansible like everything else.

🔍 Where the logins come from: wtmp

Linux records every login session that gets a terminal in /var/log/wtmp. The last command reads it, and with ISO timestamps it is easy to parse:

$ last -w -i --time-format iso | head -3
root     pts/0        203.0.113.25   2026-10-14T05:56:45+00:00 - 2026-10-14T05:56:46+00:00  (00:00)
root     pts/0        203.0.113.25   2026-10-13T14:27:37+00:00 - 2026-10-13T16:23:34+00:00  (01:55)
root     pts/0        203.0.113.25   2026-10-13T07:53:55+00:00 - 2026-10-13T07:53:55+00:00  (00:00)

The important detail is what is not in there: sessions without a terminal. When my backup machine pulls the nightly backups, it runs a command over SSH, gets no terminal, and leaves no wtmp entry. That is exactly the filter I want – interactive shells are reported, plain automated commands are not.

🧩 The script

Every two minutes a small script reads last, compares it with the newest login it has already reported, and sends one mail for everything new in that window – several logins become one mail, not five. The state is a single timestamp in /var/lib/ssh-login-alert/last-seen. On the very first run on a host it only stores the current position, so it does not mail me the whole history at once. And the state is only advanced after the mail went out: if sending fails, the next run tries again.

#!/bin/bash
#
# Tripwire: mails every new interactive login since the previous run. Started
# every 2 minutes by ssh-login-alert.timer; all new logins of one window go
# out as ONE mail.
#
# Source is wtmp (read with "last"): only sessions that got a terminal are
# recorded there. The nightly backup pulls run plain commands without a
# terminal and never appear. Ansible requests a terminal for some steps, so a
# playbook run does show up (as short sessions) - you know when that was you.
set -u

MAILTO="you@example.com"
STATE_DIR=/var/lib/ssh-login-alert
STATE="$STATE_DIR/last-seen"
mkdir -p "$STATE_DIR"

# One line per login: start-time (ISO 8601 with the host's UTC offset) TAB
# user TAB tty TAB source IP. Sorted by time; the offset is the same for all
# entries of one host, so string comparison of the first field works.
logins=$(last -w -i --time-format iso 2>/dev/null \
    | awk '$1 != "reboot" && $1 != "wtmp" && NF >= 4 { print $4 "\t" $1 "\t" $2 "\t" $3 }' \
    | sort)

newest=$(printf '%s\n' "$logins" | tail -n 1 | cut -f 1)

# First run on this host: only remember the current position, so the alert
# does not mail the whole history at once.
if [ ! -s "$STATE" ]; then
    printf '%s\n' "${newest:-1970-01-01T00:00:00+00:00}" > "$STATE"
    exit 0
fi

seen=$(cat "$STATE")
new=$(printf '%s\n' "$logins" | awk -F '\t' -v seen="$seen" '$1 > seen')
[ -n "$new" ] || exit 0

count=$(printf '%s\n' "$new" | wc -l)
{
    echo "New interactive login(s) on $(hostname -f):"
    echo
    printf '%s\n' "$new" | awk -F '\t' '{ printf "  %s  %-8s %-7s from %s\n", $1, $2, $3, $4 }'
    echo
    echo "Times carry the host's UTC offset. If none of these were you, check: last -i | head"
} | /usr/bin/mail -s "[$(hostname -f)] SSH login: $count new" "$MAILTO" || exit 1

# Only after the mail went out: a failed send is retried on the next run.
printf '%s\n' "$newest" > "$STATE"

The mail goes out through the same small SMTP relay (msmtp) that already sends the servers’ other reports. It looks like this:

Subject: [www.example.org] SSH login: 1 new

New interactive login(s) on www.example.org:

  2026-10-14T08:01:02+00:00  root     pts/1   from 203.0.113.25

Times carry the host's UTC offset. If none of these were you, check: last -i | head

⏱️ A timer, not a PAM hook

The obvious way to react to a login is a PAM module: pam_exec in the SSH PAM stack runs a script on every session. I decided against it for two reasons.

• Lockout risk. PAM is the login itself. A typo or a hanging script in the SSH PAM stack can make the server refuse every login – exactly the situation where I then need the console of the hosting provider.
• SELinux. A script started by PAM runs in the security context of sshd. Sending mail from there means network connections from a domain that SELinux rightly does not allow to do that.

A systemd timer has neither problem. It runs completely separate from sshd, in its own unit, and the worst thing a bug can do is not send a mail. The price is a delay of up to two minutes – acceptable for a tripwire.

# /etc/systemd/system/ssh-login-alert.service
[Unit]
Description=Mail new interactive SSH logins

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/ssh-login-alert

# /etc/systemd/system/ssh-login-alert.timer
[Unit]
Description=Check for new interactive SSH logins every 2 minutes

[Timer]
OnBootSec=2min
OnUnitActiveSec=2min

[Install]
WantedBy=timers.target

🤖 Keeping Ansible quiet

The first test brought a surprise: every Ansible run showed up in wtmp as a few short sessions. Ansible’s SSH plugin asks for a terminal for some steps by default. None of my playbooks needs one – pipelining is on, and nothing ever prompts for a password – so one line in ansible.cfg turns it off:

[ssh_connection]
pipelining = True
# No terminal for Ansible's own connections - nothing here needs one, and
# without it a playbook run leaves no entry in wtmp.
usetty     = False

After that, a full playbook run left no trace in wtmp, and the alarm only fires for real interactive sessions. In Ansible itself the whole thing is three small tasks – script from a template, two unit files, enable the timer:

- name: SSH | Deploy the login alert script
  ansible.builtin.template:
    src: ssh-login-alert.sh.j2
    dest: /usr/local/sbin/ssh-login-alert
    owner: root
    group: root
    mode: "0750"

- name: SSH | Enable the login alert timer
  ansible.builtin.systemd:
    name: ssh-login-alert.timer
    enabled: true
    state: started
    daemon_reload: "{{ _ssh_alert_units.changed }}"
  # in --check the units were not written yet on a fresh host
  when: not (ansible_check_mode and _ssh_alert_units.changed)

🕵️ Does this help an attacker?

This post describes exactly what triggers the alarm and what does not. So the honest question: does it hand an attacker a recipe for staying invisible?

Partly, yes – and that is fine. Anyone with a stolen key who reads this knows that ssh host 'command' without a terminal does not trigger a mail. But that is not a secret gap, it is how wtmp works, and anyone who knows Linux knows it. More importantly, the alarm is one layer of several. Getting in requires a valid key in the first place – passwords are switched off, SSH only listens to one country’s address ranges, and failed attempts get banned. And whatever someone does once inside without a terminal still has to get past the other tripwires: changes to user accounts, sudo, the SSH configuration, kernel modules or the cluster’s secrets are recorded and reported by other means, and changed system files are found the next night.

Someone who gets root can also switch the timer off. But the interactive login that got them there has already been mailed by then – within two minutes, long before anyone typically starts cleaning up traces. Which brings us back to yesterday’s principle: the design may be public; the protection comes from the keys and from having more than one layer.

✅ Result

• One mail per interactive SSH login, within two minutes, on all three servers.
• Silent for backups and for Ansible.
• No change to sshd or PAM, so it cannot lock anyone out.
• A deliberate limit (commands without a terminal), covered by other layers.

The first real mail arrived during testing, a few minutes after deployment – a test login I had triggered on purpose. That is exactly what it should feel like: boring when it was you, impossible to miss when it was not.