📡 Shipping Logs Like a Pro: Fluent Bit Meets Ansible in My Homelab

By now, my little homelab has grown into a small but mighty fleet of virtual machines — almost a dozen of them are humming along nicely. But with more services come more logs… and more chaos. 🌀

To bring order to that chaos, I’m using the open-source log shipper Fluent Bit to collect logs from all my VMs and forward them to a centralized OpenSearch instance for analysis and visualization.

The goal? Collect everything — from disk space warnings, failed login attempts, and systemd noise to juicy kernel messages. All logs across all my machines should end up neatly indexed and searchable in a single OpenSearch dashboard.

(I’ll walk you through the OpenSearch setup in a separate post soon.)

Now, instead of manually logging into every VM and configuring Fluent Bit by hand (no thanks 🙃), I’ve built a small, tidy Ansible Playbook that automates the entire setup.

Here’s what the playbook does:

• Installs Fluent Bit
• Uploads a custom config file from the local files/ directory
• Ensures the service is started and enabled

Once the playbook is ready, I can deploy Fluent Bit to all VMs at once, in parallel — no manual work required. Just one command, and Ansible takes care of the rest.

One of the best things about Ansible is its idempotence: it smartly checks the state of each target system and only runs the tasks that are actually needed. You can run the playbook multiple times without breaking things — and that’s pure DevOps magic. ✨

To make this work, there’s one more file involved: a hosts.ini inventory file, which lists the IP addresses of all my VMs and specifies how Ansible should SSH into them.

--- # Ansible Playbook to install Fluent Bit
- name: Install packages and configure Fluent Bit on Debian 12
  hosts: homelab
  become: true

  vars:
    fluentbit_custom_config: true  # Set to true to upload your own config file from "files/"

  tasks:

    - name: Ensure gpg is installed
      ansible.builtin.apt:
        name: gnupg
        state: present
        update_cache: yes

    - name: Import Fluent Bit GPG key
      ansible.builtin.apt_key:
        url: https://packages.fluentbit.io/fluentbit.key
        keyring: /usr/share/keyrings/fluentbit-archive-keyring.gpg
        state: present

    - name: Add Fluent Bit repository
      ansible.builtin.apt_repository:
        repo: deb [signed-by=/usr/share/keyrings/fluentbit-archive-keyring.gpg] https://packages.fluentbit.io/debian/bookworm bookworm main
        filename: fluentbit
        state: present

    - name: Update apt cache
      ansible.builtin.apt:
        update_cache: yes

    - name: Install Fluent Bit
      ansible.builtin.apt:
        name: fluent-bit
        state: present

    - name: Optional - Deploy custom fluent-bit.conf from files/
      ansible.builtin.copy:
        src: fluent-bit.conf
        dest: /etc/fluent-bit/fluent-bit.conf
        owner: root
        group: root
        mode: '0644'
      notify: Restart fluent-bit
      when: fluentbit_custom_config | default(false)

    - name: Enable and start Fluent Bit service
      ansible.builtin.systemd:
        name: fluent-bit
        enabled: yes
        state: started

  handlers:
    - name: Restart fluent-bit
      ansible.builtin.systemd:
        name: fluent-bit
        state: restarted

Here’s a sample output after running the playbook. As you can see, everything worked smoothly — except for pihole, where gpg was missing at first, causing the initial install to partially fail. After installing gpg, the playbook ran through just fine:

root@seedVM:~/ansible/fluent-bit# ansible-playbook install_and_configure.yml -i hosts.ini

PLAY [Install packages and configure Fluent Bit on Debian 12] ************************************************

TASK [Gathering Facts] ***************************************************************************************
ok: [pihole]
ok: [gitlab]
...

TASK [Install Fluent Bit] ************************************************************************************
ok: [gitlab]
ok: [grafana]
changed: [pihole]
...

TASK [Enable and start Fluent Bit service] *******************************************************************
ok: [nextcloud]
changed: [pihole]
...

RUNNING HANDLER [Restart fluent-bit] *************************************************************************
changed: [pihole]

PLAY RECAP ****************************************************************************************************
blog     : ok=8  changed=0  unreachable=0  failed=0
pihole   : ok=9  changed=7  unreachable=0  failed=0
seedVM   : ok=8  changed=0  unreachable=0  failed=0