The Ansible Tour, Part 3: The Blog – WordPress, MariaDB and Redis on K3s, Role by Role

🌐 Also available in: Deutsch · Français · Español

Part 3 of the Ansible Tour. Part 1 covered the framework, part 2 the platform underneath – base system and Kubernetes. This part puts the blog on top. As always, it describes release v3.2.0.

This blog is WordPress. Four roles take it from an empty Kubernetes cluster to a running, configured, monitored site, and one update playbook keeps it current:

Before any code, though, it helps to know what is being built and why each piece is there. So this part starts with the architecture.

🏗️ What runs, and why

Follow one request. A reader opens a post; the request arrives on port 443 of the server.

1. The ingress controller (from part 2) terminates TLS – it holds the Let’s Encrypt certificate, decrypts the request and checks a few rules (rate limit on the login page, blocked paths, security headers). It then passes the plain HTTP request inside the cluster to the WordPress Service, a stable internal name for “whichever pod runs WordPress right now”.
2. nginx, inside the WordPress pod, receives it. If a ready-made HTML copy of the page exists in the page cache, nginx sends that file back and is done – no PHP involved. Images, CSS and JavaScript are also served by nginx directly, with long cache headers.
3. Only if the page has to be generated does nginx hand the request to PHP-FPM in the same pod. FPM (“FastCGI Process Manager”) keeps a pool of PHP worker processes ready; one of them runs WordPress’s PHP code for this request.
4. WordPress needs data: the post, the menu, dozens of settings. It first asks Redis, an in-memory store. Whatever is not there it reads from MariaDB, the database, and puts into Redis for next time.

Each of these pieces answers a concrete question:

Two Kubernetes words come up constantly below. A Deployment describes “keep one pod of this kind running” – if the pod dies, Kubernetes starts a new one. Its strategy here is always Recreate: stop the old pod first, then start the new one. Two pods writing to the same host directory at the same time is how databases get corrupted, so a few seconds of downtime during a restart are the price. A ConfigMap is a set of configuration files stored in the cluster and mounted into a pod; a Secret is the same for passwords.

🚀 blog_k3s_deploy: the whole blog as Kubernetes objects

The role turns the architecture above into objects. Every component is a Jinja template in templates/ and a task file in tasks/, applied with the K8s apply pattern from part 2. The main file is a checklist in dependency order:

- import_tasks: dirs.yml               # host directories with the right owners
- import_tasks: namespace_secrets.yml  # namespace, Secret, ConfigMaps
- import_tasks: mariadb.yml            # database (+ wait until ready)
- import_tasks: external_services.yml  # a name for Grafana on the host
- import_tasks: redis.yml              # object cache (+ wait)
- import_tasks: wordpress.yml          # WordPress + nginx + exporter (+ wait)
- import_tasks: ingress.yml            # routing from the internet
- import_tasks: cronjob.yml            # wp-cron every 5 min, nightly updates
- import_tasks: exporters.yml          # metrics ports for Prometheus
- import_tasks: networkpolicy.yml      # who may talk to whom
- import_tasks: image_policy.yml       # which images may run

WordPress needs a running database, so the role waits after MariaDB, Redis and WordPress – with kubectl rollout status or kubectl wait, which block until the new pods report ready. When the role finishes, the blog is up.

A guard before anything else

- name: blog_k3s_deploy | Check that the exporter DB password is present in the vault
  ansible.builtin.assert:
    that:
      - blog_db_exporter_pass is defined
      - blog_db_exporter_pass | length > 0
    fail_msg: >-
      blog_db_exporter_pass is missing from this host's vault.yml. Add a line
      like  blog_db_exporter_pass: "<a fresh random password>" ...

This variable was added later, when the database exporter stopped logging in as root. Older vaults do not have it. Without this check, the run would fail halfway through with “undefined variable” inside a template. With it, it fails at the start, and the message says exactly what to add. assert is the tool for such guards: a list of conditions, all must be true, and a message for when they are not. (The role’s argument_specs list it as optional, so that check alone would not catch it.)

dirs.yml: owners that stay right

Two host directories are created with the file module: the WordPress files, owned by uid 33 (www-data inside the container), and the database files, owned by uid 999 (mysql). The numbers matter. An earlier version created them as root. The pods’ init containers (more on those below) fixed the owner at startup – and the next playbook run set it back to root. Everything kept working until a plugin tried to create a database table and failed with a misleading “Permission denied”. The cause was not the database at all, but this task undoing the init container’s work. Since then the task sets the same owner the containers expect – and since 2026-10-16 the same mode too: the database init container still reset the directory to 0755 while the role set 0750, so it drifted after every pod restart. Now both say 0750, and the two never fight. The same file also places the PGP public key from the imprint into the web root.

namespace_secrets.yml: the frame

A namespace is a named area in the cluster; everything of the blog lives in wordpress. Three details:

• Pod Security labels, warn and audit only. Kubernetes can enforce “Pod Security Standards”. The “baseline” standard forbids host directories in pods – which is exactly how this blog stores its data. Enforcing it would reject every pod, including the cron job every five minutes. So the namespace gets the labels in warn and audit mode: violations are reported, nothing is blocked. This task uses k8s without apply: a plain merge adds the labels to the existing ones instead of replacing the whole namespace object.
• No service-account token. The same patch as in part 2, for the namespace that did not exist yet back then.
• The Secret, with no_log. Database passwords, admin login, the exporter’s credentials. The template is rendered in memory and sent to the API; no_log: true keeps all of it out of the output, even in a dry run with --diff.

Then the ConfigMaps: the nginx site configuration, the PHP settings (upload size, memory, OPcache) and the PHP-FPM pool (twelve workers at most, a status page for the exporter). The nginx part is where much of the blog’s behaviour is defined: which files are never served (wp-config.php, backup suffixes like .bak or .save, .phar, the exports of a migration plugin), how the page cache is served before PHP is ever asked, and how long browsers may cache images and scripts. Every one of the blocked patterns has a comment with the reason, several of them with a date – for example a backup plugin that writes complete site exports, database included, into a web-accessible folder and relies on an .htaccess file that nginx ignores.

A pod restarts when its configuration changes

A trap that is easy to fall into: the configuration files are mounted into the pod with subPath, and such mounts never update in a running pod. A changed ConfigMap only took effect at the next accidental restart. The fix is one annotation in the WordPress pod template:

template:
  metadata:
    annotations:
      checksum/config: "{{ lookup('ansible.builtin.template', 'nginx-cm.yml.j2') | hash('sha256') }}"

The template of the ConfigMaps is rendered once more, here, and reduced to a SHA-256 checksum. If the configuration changes, the checksum changes – and with it the pod template, which makes Kubernetes roll out a new pod. If nothing changed, the checksum is the same and the pod is left alone. A small trick that turns “remember to restart” into something that cannot be forgotten.

mariadb.yml: the database and a least-privilege user

The MariaDB Deployment uses the official image, which creates the database and the WordPress user on first start from environment variables. A few things in the template are worth pointing out:

• Passwords as files, not as environment variables. Environment variables are visible in /proc/<pid>/environ, in kubectl describe and in some error pages. The image supports the convention MARIADB_ROOT_PASSWORD_FILE: it reads the password from a file instead, and the file comes from the Secret, mounted read-only.
• An init container runs before the database starts and sets the owner of the data directory. An init container is a short-lived container that has to finish before the main containers start – the right place for one-time preparation.
• Probes: mysqladmin ping tells Kubernetes when the database is ready to accept connections (readiness) and when it hangs and should be restarted (liveness).
• A locked-down container: all Linux capabilities dropped and only the five back that the entrypoint needs, no privilege escalation, a read-only root file system with two small writable volumes for the socket and temp tables. The temp-table volume lives in RAM since 2026-10-16 (about 2,000 temp tables a day used to go to disk); because a memory volume counts against the container’s memory limit, that limit grew from 768 MiB to 1280 MiB. Part 5 explains these settings in general.

After the rollout the role creates a database user for the metrics exporter. It used to log in as root – full access for a program that only reads counters. Now it has exactly three privileges and read access to the performance schema, nothing on the WordPress tables. The interesting part is how this is done idempotently without a MySQL module that could reach into the pod:

- name: MariaDB | Check the mysqld_exporter database user
  ansible.builtin.command:
    argv:
      - k3s
      - kubectl
      - exec
      - deployment/mariadb
      - --namespace=wordpress
      - --container=mariadb
      - --
      - sh
      - -c
      - >-
        mariadb -N -u root -p"$(cat /etc/secrets/db-root-password)" -e "
        SELECT
          (SELECT COUNT(*) FROM mysql.user WHERE User = 'exporter' ...
             AND authentication_string = PASSWORD('...'))
        + (SELECT COUNT(*) FROM mysql.db WHERE User = 'exporter' ...
             AND Db = 'performance_schema' AND Select_priv = 'Y');"
  register: exporter_user_check
  changed_when: false
  no_log: true

- name: MariaDB | Create least-privilege mysqld_exporter database user
  ansible.builtin.command:
    argv: [ ... CREATE OR REPLACE USER ... GRANT ... ]
  when: exporter_user_check.stdout | default('') | trim != '4'
  no_log: true

Read it as a question with a number as the answer. The SQL counts two things: users named “exporter” with exactly this password (two hosts: localhost and 127.0.0.1), and the read grant on the performance schema for both. Everything correct means 4. Any other number – the user is missing, the password in the vault was changed, a grant was lost – and the second task recreates the user with CREATE OR REPLACE. That also makes password rotation trivial: change it in the vault, run the playbook.

Note argv instead of a command string: each list item is passed as exactly one argument, without a shell in between that would interpret quotes or dollar signs. The SQL travels intact into the pod, where sh -c runs it – and the root password is read inside the pod from the mounted file, so it never appears on a command line on the host.

redis.yml and an entrypoint that never ran

Redis runs with persistence switched off (--save "" --appendonly no), a memory limit of 80 MB and the eviction policy allkeys-lru: when memory is full, the least recently used entry goes. The 80 MB are deliberately well below the container limit of 128 MB, because Redis’s own memory accounting was measured at almost a third of the real usage – Redis should evict by itself long before the kernel kills it. WordPress additionally sets a maximum lifetime of one day per cache entry, so the cache cleans itself.

The template also carries one of the more instructive bugs of this repository. The container is started with its own command: – and in Kubernetes, command replaces the image’s entrypoint. That entrypoint is the script that would have switched from root to the redis user. So until October 2026, Redis ran as root, with two capabilities added for a user switch that never happened. The fix is not to restore the entrypoint, but to start the container directly as the right user:

securityContext:
  runAsUser: 999
  runAsGroup: 1000
  runAsNonRoot: true
  fsGroup: 1000

Started as uid 999, Redis needs no capabilities at all. fsGroup makes Kubernetes give the pod’s volumes to that group, so the (unused, but present) data directory is writable.

wordpress.yml: three containers and an init container

The WordPress pod is the largest template of the role. Its init container has two jobs: set the owner of the web root, and download WP-CLI, the command-line tool that blog_wordpress uses to control WordPress. Both jobs carry a lesson:

• WP-CLI lives outside the web root. It used to be in /var/www/html – and everything there is reachable over HTTP. nginx only refused .php files, not .phar, so anybody could download a 7 MB tool from the blog. Now it goes into an emptyDir (a scratch volume that lives as long as the pod), mounted read-only into the PHP container.
• Pinned and checksummed. The download comes from GitHub with a fixed version and is checked against a SHA-512 from versions.yml. The busybox wget in the init container does not verify TLS certificates, and WP-CLI later runs as root – an unverified download would be remote code execution waiting to happen. If the checksum does not match, the init container fails and the pod never starts.
• sleep 5 before the download. A brand-new pod’s first outgoing connection can fail because Calico has not programmed its network rules yet. That caused a real outage: during a restart the download hung, the old pod was already gone (one replica, Recreate), and the site answered 502 until the init container was retried by hand. Five seconds of waiting are cheaper.

Then the three containers – PHP-FPM, the FPM exporter and nginx – each with the same pattern of hardening: all capabilities dropped and only the necessary ones back, with a comment per capability (“the FPM master starts as root and forks workers as www-data” needs SETUID/SETGID), no privilege escalation, the default seccomp profile, a read-only root file system with small writable volumes for exactly the paths that need them. The exporter runs as user “nobody” and needs no capabilities at all.

WordPress’s own configuration is passed in through an environment variable the image understands, WORDPRESS_CONFIG_EXTRA. It sets the public URL, the Redis connection, the cache lifetime, the page cache – and DISALLOW_FILE_EDIT, which removes the plugin and theme editor from the admin area. Without it, a stolen admin password is one click away from running arbitrary PHP on the server.

Finally the probes, three kinds for a reason:

For nginx the probes deliberately check the TCP port, not an HTTP page: an HTTP check would go through to PHP and could declare nginx dead when only PHP had a problem.

ingress.yml: master, minion and a placeholder

The blog’s routing is three Ingress objects. That is not by choice, it is how F5’s controller handles several Ingresses for one host name: one master holds the TLS certificate and server-wide settings, and minions hold the paths. cert-manager’s temporary challenge Ingress (part 2) joins as another minion. Here the master carries the redirect to HTTPS (except for the ACME challenge path), a few hard denies, and the rate limit on wp-login.php – in its own location, which once meant that the login page went out without the HSTS header, because header rules live in the minion. One minion routes everything to WordPress and adds the security headers; a second one routes /grafana/ to Grafana on the host. Grafana got its own minion because it needs WebSocket headers that WordPress should not get, and because two contradicting frame headers would have been sent otherwise. A fourth Ingress redirects the bare domain without “www” to the full one.

After the apply, a small piece of logic handles the very first deployment on a new server:

- name: Ingress | Check whether TLS secrets already exist
  kubernetes.core.k8s_info:
    kind: Secret
    namespace: wordpress
    name: "{{ item.secret }}"
  loop:
    - { secret: wordpress-tls, host: "{{ blog_hostname }}" }
    - { secret: apt-upgrade-me-apex-tls, host: "{{ blog_hostname | regex_replace('^www\\.', '') }}" }
  register: wp_tls_secret_check
  no_log: true

- name: Ingress | Seed self-signed placeholder for any TLS secret cert-manager hasn't issued yet
  ansible.builtin.shell: |
    openssl req -x509 -nodes -newkey rsa:2048 -days 7 ... -subj "/CN={{ item.item.host }}"
    k3s kubectl create secret tls {{ item.item.secret }} ...
  loop: "{{ wp_tls_secret_check.results }}"
  when: item.resources | length == 0

For a host name whose certificate does not exist yet, this controller rejects the TLS handshake – the site looks dead until Let’s Encrypt has answered. So for each expected certificate Secret the role asks whether it exists, and creates a seven-day self-signed placeholder if not; cert-manager overwrites it as soon as the real certificate arrives. The loop shows a pattern worth knowing: the second task loops over the results of the first. Each result contains the original loop item as item.item (host name and Secret name) and the answer as item.resources (empty list = not found). The regex_replace filter derives the bare domain from the www name. And no_log is there because k8s_info returns a Secret’s data – private keys.

cronjob.yml: two scheduled jobs

The first CronJob runs wp-cron.php every five minutes, with concurrencyPolicy: Forbid (never two at once). It had two bugs worth telling. Its pods used to carry the label app: wordpress – the same label the WordPress Service selects on. So for their ten seconds of life, the cron pods were part of the Service, and the ingress sent real visitors to a pod without a web server: a 502 every now and then, every five minutes. Now they are labelled wordpress-cron, and the NetworkPolicies allow both labels. The second bug: the cron container never had the WORDPRESS_CONFIG_EXTRA block, so it did not know where Redis was, tried 127.0.0.1, and every scheduled run died on “Error establishing a Redis connection” – silently, because WordPress exits with 0 in that case. The block is now identical in both templates, with a comment saying it must stay that way, and the container also waits five seconds for Calico, like the init container above.

The second CronJob is the nightly update. The cron pods mount the web root read-only, so WordPress’s own background updates silently gave up – every plugin update waited for a click in the admin area. Instead of making the cron pods writable, a separate nightly job does it properly: an init container downloads WP-CLI (pinned and checksummed again), a second one dumps the database into a backup directory and keeps 14 days, and only if the dump succeeded does the main container run core, plugin, theme and translation updates as www-data and check the home page afterwards. The update playbook further down reuses exactly this job.

exporters.yml, networkpolicy.yml, image_policy.yml

The exporters listen inside their pods. Prometheus runs on the host, so two Services of type NodePort make them reachable on fixed ports of the server – and the firewall from part 5 drops those ports for anything but local connections.

The NetworkPolicies start with a threat model in the comment, which I find the best part of the file. PHP-FPM parses untrusted input – requests, uploads, plugin code – and is by far the most likely place for an attacker to get code execution. It also legitimately needs the database and Redis. So the goal is not to cut it off, but (1) to make sure only the WordPress pods can reach MariaDB and Redis, and (2) to cap what a compromised WordPress pod can reach. The policies allow exactly: the ingress (via the host’s address, because the controller runs in the host network), DNS, MariaDB, Redis, the exporter port – and outgoing HTTP/HTTPS to the internet. The comment is honest about that last one: updates come from wordpress.org and GitHub behind changing CDN addresses, and standard NetworkPolicies cannot match domain names. What they do prevent is movement inside the cluster.

The image policy is a ValidatingAdmissionPolicy: a rule the API server checks before it accepts a pod. Its condition is written in CEL, a small expression language:

validations:
- expression: >
    object.spec.containers.all(c, c.image.split(':')[0] in variables.allowedRepos)
    && (!has(object.spec.initContainers) ||
        object.spec.initContainers.all(c, c.image.split(':')[0] in variables.allowedRepos))

“Every container’s image, without the tag, is in the list of allowed repositories – and the same for init containers, if there are any.” Matching on the repository and not the tag means a version bump never needs a change here; what is blocked is a different image. One entry in the list has a story: cert-manager’s challenge solver runs its pod in the blog’s namespace, not in its own. When the bare-domain certificate was added, the solver was rejected, and no certificate could be issued until its image was allowed.

🔧 blog_wordpress: configuring a running site from the outside

WordPress is now running, but empty or – on every later run – possibly drifted. This role configures it. There is no Ansible module for WordPress; everything goes through WP-CLI inside the pod:

k3s kubectl exec deployment/wordpress --namespace wordpress --container wordpress-fpm --
  php /usr/local/lib/wp-cli/wp-cli.phar <command> --path=/var/www/html ...

That makes the whole role one long application of the “ask, then act” pattern from part 2. The role says so in its header and lists the order: installation, a hardening plugin, plugins, appearance, the AI notice and three small helper plugins, settings, updates.

Root or www-data?

kubectl exec runs as root inside the container. That is a problem for a PHP application: files created by root cannot be changed by PHP-FPM, which runs as www-data. The role follows a rule, written down at the top of main.yml: commands that install or update code run as root with --allow-root (WP-CLI refuses root otherwise) and are always followed by a chown back to www-data. Read-only queries and everything that writes without a later chown run as www-data via su -s /bin/sh www-data -c "...". That rule exists because the other way round once left a cache directory owned by root, and the caching plugin could no longer write to it.

install.yml: only once

wp core is-installed exits with 0 when the WordPress tables exist and with 1 when not. Registered with failed_when: false, the exit code becomes a decision: when: wp_installed.rc != 0 runs wp core install on a fresh database and never again. The install task has no_log: true, because the admin password is part of its command line. The site title goes through the quote filter, which wraps it safely for the shell.

security_muplugin.yml: a file that only changes when it changes

A must-use plugin is a PHP file in wp-content/mu-plugins/. WordPress loads every file there on every request, before normal plugins, and it cannot be deactivated in the admin area – the right place for rules that must always apply. This one hides version numbers, removes discovery links, and blocks the REST endpoint that lists user names to anonymous visitors.

How it is written is the interesting part, because the same pattern is used for every PHP file the role deploys:

- name: WordPress | Deploy security hardening mu-plugin
  ansible.builtin.command:
    argv:
      - k3s
      - kubectl
      - exec
      - deployment/wordpress
      - ...
      - sh
      - -c
      - |
        F=/var/www/html/wp-content/mu-plugins/security-hardening.php
        cat > "$F.new" << 'PHPEOF'
        <?php
        ...
        PHPEOF
        if [ -f "$F" ] && [ "$(md5sum < "$F.new")" = "$(md5sum < "$F")" ]; then
          rm -f "$F.new"
        else
          mv "$F.new" "$F" && chown www-data:www-data "$F" && echo CHANGED
        fi
  register: wp_mu_hardening
  changed_when: "'CHANGED' in wp_mu_hardening.stdout"
  notify: Restart wordpress deployment (OPcache picks up changed PHP files)

1. The file is written as security-hardening.php.new – a name WordPress does not load, because it does not end in .php.
2. The quoted heredoc (<< 'PHPEOF') keeps the PHP code literal: no $ is expanded by the shell. argv makes sure Ansible does not touch it either.
3. If the new file has the same checksum as the existing one, it is thrown away. Otherwise it replaces the old one and the script prints CHANGED.
4. changed_when looks for that word. Only then is the task “changed” – and only then is the handler notified.

The handler restarts WordPress, because PHP caches compiled code in OPcache and only re-checks files once a minute; a restart makes a changed file live at once. Before this pattern, the file was simply overwritten with changed_when: true – and every playbook run restarted WordPress, with thirty to sixty seconds of 502 each time. The handler file holds a single entry, notified by four tasks, and runs at most once at the end of the play.

plugins.yml: block, always, and a list difference

The plugin list lives in the host’s inventory (blog_wordpress_plugins). The role reads the installed plugins once, as JSON, and computes the rest:

- name: WordPress | Install and activate missing plugins
  ansible.builtin.command: >
    ... wp-cli.phar plugin install {{ item }} --activate --allow-root
  loop: "{{ blog_wordpress_plugins | default([]) | difference(_wp_plugins_installed) }}"
  register: wp_plugin_result
  until: wp_plugin_result.rc == 0
  retries: 3
  delay: 10
  vars:
    _wp_plugins_installed: "{{ wp_plugin_state.stdout | from_json | map(attribute='name') | list }}"

- name: WordPress | Activate installed but inactive plugins
  ...
  when: _wp_plugins_inactive | length > 0
  vars:
    _wp_plugins_inactive: >-
      {{ wp_plugin_state.stdout | from_json
         | selectattr('name', 'in', blog_wordpress_plugins | default([]))
         | selectattr('status', 'equalto', 'inactive')
         | map(attribute='name') | list }}

from_json turns WP-CLI’s output into a list of dictionaries. difference is list subtraction: wanted plugins minus installed ones – only those are installed. selectattr filters a list of dictionaries by a field: of all installed plugins keep those that are wanted, of those keep the inactive ones. Before this, every run reinstalled every plugin with --force, reported all of them as changed – and could quietly pull a newer release without anyone noticing. New releases now come only from the update step.

until with retries works per loop item: if one download times out (WP-CLI’s timeout is a fixed ten seconds), only that plugin is retried. And all of this sits in a block with an always section:

- name: WordPress | Install plugins and fix ownership
  block:
    - ...read state, install, activate, Redis...
  always:
    - name: WordPress | Fix wp-content ownership after plugin installation
      ansible.builtin.command: >
        k3s kubectl exec ... -- chown -R www-data:www-data /var/www/html/wp-content

Tasks under always run even when a task in the block fails. Why that matters: WP-CLI as root writes plugin files as root. Once, a download timed out halfway, the play stopped, and the chown never ran – a backup plugin was left unable to create its own storage folder until someone fixed it by hand. Now the ownership is restored no matter what happens above it.

The Redis plugin gets its own two tasks, because wp redis enable places a drop-in file that turns the plugin on. Its changed_when is a nice example of reading a tool’s output carefully: WP-CLI prints “Activating …” even for a plugin that is already active, followed by “already active” on stderr. So only a run that does not report both “already installed” and “already active” counts as a change.

appearance.yml, ai_disclosure.yml, settings.yml

Three smaller files, each with one idea:

• Appearance runs small PHP snippets through wp eval to set the custom CSS and the menu, both stored in the database. The comment admits a limitation honestly: the CSS is written on every run, but only its first creation counts as “changed” – an edited CSS block is applied, yet the recap shows “ok”.
• The AI notice creates the disclosure page and its link from the About page – compared first, written only on a difference, so a manual edit in the admin area is put back – and deploys three more must-use plugins with the same .new-and-checksum pattern: the footer notice, performance hints (font preloading and an eagerly loaded first image, after a mobile Lighthouse run showed both problems) and the “Also available in” line for the translations.
• Settings closes public self-registration (it was being farmed by spam bots: eighteen fake accounts in one clean-up) and switches off WordPress’s visitor-triggered cron, because the CronJob does that job. Two settings, two ways to stay idempotent: wp option update says “Value updated” only when something really changed, so changed_when reads that; the cron constant is read first and written only when it is not already true.

updates.yml: updates, cleanup and an honest restart

The last file updates core, plugins, themes and translations on every run, inside another block/always pair with the same ownership fix. Two techniques make it work:

• failed_when that reads the output. wp plugin update --all exits with 1 when there is nothing to update – the normal case. So the task fails only if the exit code is not 0 and the output does not say “No plugins updated”. The same for themes, and for deleting plugins that are already gone.
• Two snapshots. At the start of plugins.yml the role records all core, plugin and theme versions; here it records them again. WordPress is restarted only if something reported an update or the two snapshots differ. A run without updates causes no downtime.

Between the update steps the role also deletes unused plugins and themes, because a deactivated plugin is still on disk and still reachable – a vulnerability in it is still exploitable. Inactive themes are computed, not listed: theme list --status=inactive can never return the active theme, so the site’s own theme is structurally safe, and one default theme is kept as WordPress’s fallback. The deletion runs after the core update on purpose: a core update can reinstall the bundled default themes.

One limitation is stated in the role header: command tasks are skipped in check mode. A dry run can therefore only run the read-only queries here; what WP-CLI would change cannot be shown in advance.

📊 blog_visitor_stats: counting readers without a tracker

The idea has its own post: no tracking script, no cookies, no third party – count visitors from the ingress logs, on the server, and keep only the numbers. K3s keeps container logs for only a few days, so a nightly snapshot is needed to see a trend over months. The role is short, but uses several features that appear nowhere else:

• copy src: "{{ playbook_dir }}/scripts/analytics/visitor-stats.py" – a file from outside the role. playbook_dir is the directory of the running playbook; a plain src: name would look in the role’s files/. The script lives in scripts/ because it is also a tool to run by hand.
• Four systemd units from templates, in one loop, with notify to a handler that reloads systemd – and meta: flush_handlers right after, because the timers must start with the new units, not at the end of the play.
• The daily service runs with systemd’s sandbox options: the whole file system read-only except its own data directory, no home directories, a private /tmp, no privilege escalation. A statistics script that reads logs needs nothing more.
• A one-time backfill: the logs still on disk cover the last few days, so the role records them once, with a loop over 1 to 5 and a date computed in Jinja – '%Y-%m-%d' | strftime(epoch - item * 86400), “today minus N days”. A {% for %} inside the command line adds an --exclude argument for each network to leave out (my own connections, which would otherwise dominate a small blog’s numbers). creates: on a marker file makes the whole backfill run exactly once.

✅ blog_verify and common_verify: did it work?

Part 1 described the idea: collect all results, judge them, report, and only then fail. Here is how it is built. blog_verify has three files: main.yml sets the colour codes for the report and imports the other two, checks.yml runs the checks, report.yml judges them.

Part of the checks is not written in this role at all. common_verify is a library role: it never appears in a playbook and has no tasks/main.yml, only three task files – host checks, Kubernetes checks, HTTP checks. Both verify roles pull them in:

- name: Verify | Shared host checks (common_verify)
  ansible.builtin.import_role:
    name: common_verify
    tasks_from: host

tasks_from picks a file other than main.yml. The host file checks nftables (service, table, default-drop policy, ban list), fail2ban, K3s, auditd, SELinux and TCP BBR; the Kubernetes file checks the ingress and cert-manager pods and looks for pods in an error state anywhere; the HTTP file checks the site from outside, its headers and a blocked path. The blog adds its own: the login page reachable, xmlrpc.php and readme.html blocked, no user names via the REST API, OPcache on, MariaDB reachable from the WordPress pod. Several of them are exactly the things I tried to break once.

Every check task has the same four lines: register: _v_something, failed_when: false, changed_when: false, check_mode: false. Nothing fails, nothing changes, everything runs even in a dry run. report.yml then computes a true/false per check with set_fact, counts them, prints a coloured table and sends it by mail – before the final assert, so the mail also goes out when something critical is broken. Only the assert can fail the play. An optional last step runs a self-test of the Monit watchdog, which sends real alarm mails – off by default, switched on per run with -e monit_selftest=true.

🔄 blog-update.yml: the routine update

The full playbook can update WordPress, but it touches the whole server. The update playbook only touches the application, and it does so the same way the nightly job does – it runs that job. The strategy behind it is older than this code; here is what the code does today:

1. Guards. Refuse to run without --limit. Refuse if WordPress has no available pod – updating a site that is already down would mix two problems.
2. Versions before, for the report and the restart decision.
3. Run the update job now. kubectl create job --from=cronjob/wordpress-autoupdate starts a one-off copy of the nightly job: database dump, then the updates. The playbook then polls the job’s status with until instead of kubectl wait --for=condition=complete – the latter would sit until its timeout if the job fails instead of completing. The job log is printed. The whole step is skipped in check mode.
4. Rebuilt images. common_image_refresh runs a small Python script on the server (with Ansible’s script module, which copies it, runs it and removes it). For every Deployment it pulls the image tag again and compares the digest with the one the running pod started from. Docker Hub rebuilds unchanged tags for security fixes in the base image, but K3s never pulls again on its own. The script returns JSON: which Deployments run an outdated build.
5. Restart only what changed, in a fixed order: database first, WordPress last – WordPress reconnects to a restarted database, but a restarted WordPress facing a database that is just going down would serve errors. The loop expression keeps that order and appends anything not in the list.
6. Home page check with the uri module: HTTP 200 or the run fails.

All of that is inside a block. If anything fails, the rescue section shows pods and recent events, the newest database dump of the update job (the restore point), and the state at the moment of the failure – and then stops. No automatic rollback: a restore is a decision, not a reflex.

🛠️ Running it: the blog day to day

Everything above happens automatically. This section is what you do by hand – the commands, the routine and what to do when something is wrong. All commands run as root on the server; k3s kubectl is the Kubernetes client that comes with K3s.

Looking at it

# Are all pods running? (mariadb, redis, wordpress; cron pods come and go)
k3s kubectl get pods -n wordpress

# Did the scheduled jobs succeed? (wordpress-cron every 5 min, wordpress-autoupdate nightly)
k3s kubectl get jobs -n wordpress

# Logs of a container (add --previous after a crash)
k3s kubectl logs deployment/wordpress -c wordpress-fpm -n wordpress --tail=50

# Is the certificate valid, and when is it renewed?
k3s kubectl get certificate -n wordpress \
  -o custom-columns=NAME:.metadata.name,READY:.status.conditions[0].status,RENEWAL:.status.renewalTime

Talking to WordPress

WP-CLI is in the pod. Run it as www-data, never as root, unless you install code and fix the owner afterwards – root-owned files are the most common way to break plugins and caches here:

k3s kubectl exec -n wordpress deployment/wordpress -c wordpress-fpm -- \
  su -s /bin/sh www-data -c 'php /usr/local/lib/wp-cli/wp-cli.phar --path=/var/www/html plugin list'

The routine

When something goes wrong

• 502 Bad Gateway: nginx is up, PHP-FPM is not answering. Look at the FPM logs (above); a restart is k3s kubectl rollout restart deployment/wordpress -n wordpress – about half a minute of downtime because of the Recreate strategy.
• A pod does not start: k3s kubectl describe pod -l app=wordpress -n wordpress shows events at the bottom. The usual causes are a database that is not running, a password in the Secret that does not match the database user (run the playbook again), or wrong owners on the host directories (uid 33 for the web root, 999 for the database).
• A plugin update broke the site: the nightly job dumped the database first, into the update backup directory on the host (14 days). Restore the dump if the update touched the database, and pin the plugin to the last good release: wp plugin install <slug> --version=<x> --force. The update playbook’s rescue output names the newest dump.
• The whole server is gone: set up a new one with the full playbook, then restore files and database from the external backup with the restore script, then run the playbook once more. The backup is pulled from outside every day – files with rsync, the database as a verified dump, versioned for 30 days – and the watchdog alerts when the last successful backup is older than three days. The operations manual in the repository has every step.

🧭 What comes next

Part 4 is Nextcloud: the same building blocks, but with the database and Redis on the host, an office suite next to it, a push service, and a configuration role that sets every value with “get, compare, set”. If you only run a blog, you can skip it and go straight to part 5, the security roles – SSH, the firewall, hardening, SELinux and the nightly scanners.