After moving two Nextcloud instances to new servers this week, I asked a simple question: do my three servers – this blog and the two Nextclouds – attract different kinds of attackers? The answer was mostly “no, it’s the same background noise everywhere”. Except for one line that bugged me: SSH. It runs on a non-standard port, accepts keys only, and fail2ban watches it. And yet the blog alone logged around 16 connection attempts per hour to the SSH daemon, from all over the world.
The thing is: I only ever connect from Germany. So why should a server in a German data centre even talk SSH to the rest of the planet? Same workflow as always: I ask the questions and make the calls, my AI co-admin (Claude) digs through the logs, writes the code and double-checks every number before we trust it.
📊 Before: who knocks on port 10022?
Over the last eight days, 3,089 connections from 510 different IP addresses made it all the way to the blog’s SSH daemon (my own connections excluded). None of them got in – there is no password to guess – but every single one costs a handshake, a log line and a fail2ban evaluation. Where they came from:
| Country | Share of attempts |
|---|---|
| China | 30 % |
| United States | 16 % |
| South Korea | 4 % |
| Germany | 4 % |
| India, United Kingdom, Hong Kong, Vietnam, South Africa | 3 % each |
| Everything else | ~31 % |
The usernames they tried are the usual dictionary: admin, ubuntu, user, test, ftpuser, deploy, git, steam. The picture on the two Nextcloud servers was the same: 98 % (blog) and 99 % (Nextcloud) of all failed SSH attempts came from outside Germany. And the few German ones? Almost exclusively rented servers at the big German hosting providers – scanners rent machines here, too.
🤔 “But attackers just use a German VPN!”
Correct, and I want to be honest about it: a country filter is not a security boundary. Anyone who targets me specifically rents a German VPS for a few euros and walks right past it. The actual protection stays exactly where it was: key-only authentication, fail2ban, and patches. So why bother?
- Noise. 98 % less garbage in the logs means the odd entry that matters actually stands out.
- The next zero-day. When a real hole in OpenSSH shows up (remember regreSSHion, CVE-2024-6387, in 2024), the first wave is mass scanning of the entire internet. A server that doesn’t answer 98 % of that internet simply isn’t in most of that wave.
- Cost: next to nothing – see below.
🌍 Where does “Germany” come from?
No commercial GeoIP database needed. The five Regional Internet Registries publish daily which IP blocks they delegated to which country. For Germany that’s RIPE NCC’s delegated-ripencc-extended-latest: about 8,700 IPv4 and 3,100 IPv6 blocks, roughly 126 million IPv4 addresses. Merged into nftables interval sets that’s 6,639 + 3,013 entries.
Does that cost RAM or CPU? We measured instead of guessing:
| What | Measured |
|---|---|
| Kernel memory for both sets | ~1.5 MB |
| Loading the full list | 0.06 s |
| Lookup per new SSH connection | ~14 comparisons (sorted interval tree), nanoseconds |
| Effect on web traffic | none – only new connections to port 10022 are checked |
Keep in mind: RIR data says who an address block was delegated to, not where the device physically sits. For “is this a German ISP or hoster?” that’s exactly the right question.
🧱 The new Ansible feature: one variable
All three servers are built from the same Ansible repository, and the firewall is the shared role common_firewall. The country filter is now part of it – off by default, switched on per host:
# inventory/host_vars/<host>/vars.yml
fw_ssh_geo_countries: [DE]
fw_ssh_geo_extra: [] # optional: CIDRs that are always allowedIn the rendered nftables ruleset, the SSH rule simply gains a set lookup:
ip daddr 203.0.113.10 tcp dport 10022 ct state new ip saddr @ssh_geo4 accept
ip6 daddr 2001:db8::1 tcp dport 10022 ip6 saddr @ssh_geo6 acceptEverything that doesn’t match falls through to the existing port-scan detection and the default drop – exactly as if the port were closed. A small script, ssh-geo-update, downloads the RIPE file, keeps the blocks of the configured countries, merges them and writes /etc/nftables/ssh-geo.nft. A systemd timer refreshes it every Sunday night.
🪤 The interesting part: how not to lock yourself out
A list of allowed addresses has one nasty failure mode: if it’s ever empty, nobody gets in – including me. Most of the work went into making that impossible:
- Firewall reloads. My ruleset recreates its whole table on every reload. We learned that the hard way earlier this week, when a reload silently emptied fail2ban’s ban sets. Here it would be worse: empty sets mean SSH closed for everyone. So the list is included into the ruleset file and refilled in the same atomic transaction. There is never a moment without it.
- The first run. Ansible builds the list before it writes the new ruleset. If the download fails, the playbook stops right there, before a single rule could lock anyone out.
- Bad data. The update keeps the old list if the download fails, if RIPE’s data is older than 14 days, or if the new list suddenly shrinks by more than 20 %. Live updates only touch the two sets; fail2ban’s sets and all other rules stay untouched.
- SELinux. The first draft kept the list in
/var/lib. But on AlmaLinux the nftables unit loads the ruleset in a confined SELinux domain that reads/etc, and an unreadable include would make the entire ruleset fail at boot. So the list lives in/etc/nftables. - Integrity monitoring. AIDE watches
/etcand mails every change. The timer therefore runs Sunday between 02:30 and 03:00, after the nightly AIDE check and before the 04:00 baseline refresh, so the weekly update never triggers a false alarm. - The emergency exit. Abroad, or on a foreign VPN? The provider’s web console still works with the root password, and
fw_ssh_geo_extratakes any address you need.
Before switching it on, Claude rendered the template for all three hosts and compared it with the live rulesets: without the variable, the output is byte-identical to before, so the two Nextcloud servers didn’t even notice the new code. The blog’s new ruleset went through nft -c (check mode) on the real host before the playbook ever ran. Because the blog is the least critical of the three, it went first.
📉 After: silence
To count what the filter rejects, I temporarily added a dedicated log rule for refused SSH attempts (the normal drop log is capped at two lines per minute for everything). The first measurement window:
| Before (8 days) | After (first ~2 hours) | |
|---|---|---|
| Connections reaching the SSH daemon | 16.3 per hour | 0 |
| Rejected by the country filter | – | 41 attempts from 9 IPs (China, USA, Hong Kong, Macau, Mexico, Canada, UK) |
| My own logins from Germany | work | still work |
Two hours is a short window, so take the exact numbers with a grain of salt – but zero versus sixteen per hour is not a rounding error. The temporary log rule is gone again; the filter stays.
👀 Bonus: who actually reads this blog?
While we were in the logs anyway, I wanted to know something else: I’ve been publishing a lot of new posts lately, so are more people finding the blog through search engines? For that, the repo has a small script, scripts/analytics/visitor-stats.py. It runs on the server itself, because visitor IPs are personal data under GDPR and never leave the host; it only prints aggregated, masked numbers. This week it learned to count arrivals from search engines, too.
The sobering part first: of 18,084 requests in the measured window, the overwhelming majority were machines. 81 % of all page views came from self-declared bots (Amazonbot, PetalBot, SemrushBot, PerplexityBot, AhrefsBot, Googlebot, ClaudeBot, …), and a good chunk of the rest were headless browsers in cloud data centres that load a page with all its assets in one second and vanish. What remained after filtering:
| Metric | Value |
|---|---|
| Regular visitors | ~17 per day, ~120 per week |
| Top countries | United States, China (probably still some disguised scrapers), Italy, United Kingdom, Netherlands, Switzerland, Germany, … |
| Arrived via search engine | 13 of 33 visitors (Google 7, DuckDuckGo 4, Bing 2) |
| Most read | the start page, the Samba-on-Debian-13 guide, the iLO heat-history post |
An honest caveat: K3s only keeps a couple of days of container logs, so this is a two-day snapshot, not a trend. Whether search traffic grows with the new posts is something I can only answer once the server keeps a daily, anonymous tally – that’s the next item on the list. And yes: an IP is not a person. These are orders of magnitude, not a headcount.
📦 Grab it
Everything lives in the public repo github.com/aptupgrademe/www_k3s (secrets stay local, of course): the firewall role with the new feature is in roles/common_firewall, the visitor statistics in scripts/analytics. Useful commands once it’s running:
# is this address allowed to reach SSH?
nft get element inet filter ssh_geo4 { 203.0.113.7 }
# when does the next refresh run, and what did the last one do?
systemctl list-timers ssh-geo-update.timer
journalctl -u ssh-geo-update.service🎯 Takeaway
A non-standard port hides SSH from almost nobody, and key-only authentication makes the knocking harmless but not quiet. Limiting SSH to the one country I actually work from removed practically all of the noise for 1.5 MB of RAM. The hard part wasn’t the filter itself; it was making sure an empty list, a firewall reload, SELinux or a bad download can never lock me out. The blog goes first; if it behaves for a while, the two Nextcloud servers follow with a single line of YAML.
