Ansible Concatenate Files: Merge Multiple Files in Order (Guide)

By Luca Berton · Published 2024-01-01 · Category: installation

How to concatenate multiple files in a specific order with Ansible. Use template, assemble, and shell modules to merge configs and files with examples.

Ansible Concatenate Files: Merge Multiple Files in Order (Guide)

How to use Concatenate multiple files in a specific order using Ansible?

This is extremely useful for service configuration files, reports, and so much more use cases. I personally use this code for markdown documents for Pandoc.

Ansible Concatenate multiple files in a specific order

ansible.builtin.template
Template a file out to a target host
ansible_managed, template_host, template_uid, template_path, template_fullpath, template_destpath, and template_run_date

Let's talk about the Ansible module template. The full name is ansible.builtin.template, it's part of ansible-core and is included in all Ansible installations. It templates a file out to a target host. Templates are processed by the Jinja2 templating language. Also you could use also some special variables in your templates: ansible_managed, template_host, template_uid, template_path, template_fullpath, template_destpath, and template_run_date. It supports a large variety of Operating Systems. For basic text formatting, use the Ansible ansible.builtin.copy module or for empty file Ansible ansible.builtin.file module. For Windows, use the ansible.windows.win_template module instead.

Parameters

src _path_ - template ("templates/" dir)
dest _path_ - target location
validate _string_ - validation command before ("%s")
backup _boolean_ - no/yes
mode/owner/group - permission
setype/seuser/selevel - SELinux

Let me highlight the most useful parameters for the template module. The only required parameters are "src" and "dest". The "src" parameter specifies the template file name. Templates usually are stored under "templates" directories with the ".j2" file extension. The "dest" parameter specifies the path where to render the template on the remote machine. The "validate" parameters allow you to specify the validation command to run before copying it into place. It's very useful with configuration files for services. Please note that the special escape sequence "%s" is going to be expanded by Ansible with the destination path. If the "backup" parameter is enabled Ansible creates a backup file including the timestamp information before copying it to the destination. Let me also highlight that we could also specify the permissions and SELinux properties.

## Playbook

How to concatenate multiple files in a specific order with the Ansible module template and YAML.

code

a.txt

A content

b.txt

B content

includes.yaml

input-files:
  - concatenate/b.txt
  - concatenate/a.txt

concatenate.yml

---
- name: concatenate Playbook
  hosts: "{{ HOSTS }}"
  become: false
  gather_facts: true
  vars:
    myinput: "concatenate/includes.yaml"
    myoutput: "concatenate/output.txt"
  tasks:
    - name: include file list
      include_vars:
        file: "{{ myinput }}"
        name: files

    - name: concatenate
      ansible.builtin.template:
        src: templates/concatenate.j2
        dest: "{{ myoutput }}"

concatenate.j2

{% for i in files["input-files"] %}
{{ lookup('file', i) }}
{% endfor %}

execution

ansible-pilot $ ansible-playbook -e "HOSTS=localhost" concatenate.yml 
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit
localhost does not match 'all'
PLAY [concatenate Playbook] ***************************************************************************
TASK [Gathering Facts] ****************************************************************************
ok: [localhost]
TASK [include file list] **************************************************************************
ok: [localhost]
TASK [concatenate] ********************************************************************************
changed: [localhost]
PLAY RECAP ****************************************************************************************
localhost                  : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
ansible-pilot $

idempotency

ansible-pilot $ ansible-playbook -e "HOSTS=localhost" concatenate.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit
localhost does not match 'all'
PLAY [concatenate Playbook] ***************************************************************************
TASK [Gathering Facts] ****************************************************************************
ok: [localhost]
TASK [include file list] **************************************************************************
ok: [localhost]
TASK [concatenate] ********************************************************************************
ok: [localhost]
PLAY RECAP ****************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
ansible-pilot $

before execution

ansible-pilot $ ls -al concatenate
total 24
drwxr-xr-x  5 lberton  staff  160 Feb  1 15:40 .
drwxr-xr-x  9 lberton  staff  288 Feb  1 15:08 ..
-rw-r--r--  1 lberton  staff    9 Feb  1 15:18 a.txt
-rw-r--r--  1 lberton  staff    9 Feb  1 15:18 b.txt
-rw-r--r--  1 lberton  staff   56 Feb  1 15:48 includes.yaml
ansible-pilot $ cat concatenate/a.txt
A content%
ansible-pilot $ cat concatenate/b.txt 
B content%
ansible-pilot $ cat concatenate/includes.yaml 
input-files:
  - concatenate/b.txt
  - concatenate/a.txt
ansible-pilot $

after execution

ansible-pilot $ ls -al concatenate
total 32
drwxr-xr-x  6 lberton  staff  192 Feb  1 15:50 .
drwxr-xr-x  9 lberton  staff  288 Feb  1 15:08 ..
-rw-r--r--  1 lberton  staff    9 Feb  1 15:18 a.txt
-rw-r--r--  1 lberton  staff    9 Feb  1 15:18 b.txt
-rw-r--r--  1 lberton  staff   56 Feb  1 15:48 includes.yaml
-rw-r--r--  1 lberton  staff   20 Feb  1 15:49 output.txt
ansible-pilot $ cat concatenate/output.txt 
B content
A content
ansible-pilot $

code with ❤️ in GitHub

Conclusion

Now you know how to concatenate multiple files in a specific order with the Ansible module template and YAML.

Using assemble Module

- name: Concatenate config fragments
  ansible.builtin.assemble:
    src: /etc/myapp/conf.d/
    dest: /etc/myapp/config.conf
    delimiter: "\n"
  become: true

Files are assembled in alphabetical order:

conf.d/
├── 00-header.conf
├── 10-database.conf
├── 20-cache.conf
└── 99-footer.conf

Using Template with Loop

{# combined.conf.j2 #}
# Auto-generated by Ansible - DO NOT EDIT
{% for file in config_files %}
# === {{ file | basename }} ===
{{ lookup('file', file) }}

{% endfor %}

- template:
    src: combined.conf.j2
    dest: /etc/myapp/config.conf
  vars:
    config_files:
      - files/header.conf
      - files/database.conf
      - files/cache.conf
      - files/logging.conf

Using shell/command

- name: Concatenate in specific order
  shell: cat header.sql schema.sql data.sql triggers.sql > database.sql
  args:
    chdir: /opt/migrations/
  become: true

Deploy Config Fragments Then Assemble

- name: Deploy individual configs
  copy:
    src: "{{ item }}"
    dest: "/etc/myapp/conf.d/{{ item | basename }}"
  loop:
    - files/00-global.conf
    - files/10-database.conf
    - files/20-cache.conf
  become: true

- name: Assemble into single config
  assemble:
    src: /etc/myapp/conf.d/
    dest: /etc/myapp/app.conf
    validate: "/opt/myapp/validate-config %s"
  become: true
  notify: restart myapp

Dynamic Content with Template

- vars:
    vhosts:
      - { name: site1.com, root: /var/www/site1 }
      - { name: site2.com, root: /var/www/site2 }
  template:
    src: nginx-sites.conf.j2
    dest: /etc/nginx/sites-enabled/all-sites.conf

{# nginx-sites.conf.j2 #}
{% for vhost in vhosts %}
server {
    listen 80;
    server_name {{ vhost.name }};
    root {{ vhost.root }};
}
{% endfor %}

Merge with Separators

- assemble:
    src: /etc/myapp/rules.d/
    dest: /etc/myapp/all-rules.conf
    delimiter: "\n---\n"  # YAML document separator
  become: true

Read and Concatenate Remote Files

- find:
    paths: /var/log/myapp
    patterns: "*.log"
  register: log_files

- slurp: { src: "{{ item.path }}" }
  loop: "{{ log_files.files | sort(attribute='path') }}"
  register: log_contents

- copy:
    content: "{{ log_contents.results | map(attribute='content') | map('b64decode') | join('\n') }}"
    dest: /tmp/combined-logs.txt
  delegate_to: localhost

assemble vs template vs shell

Method	Best For
`assemble`	Directory of fragments → single file
`template`	Dynamic content with Jinja2 logic
`shell: cat`	Quick one-off concatenation
`blockinfile`	Insert block into existing file

FAQ

How does assemble order files?

Alphabetically by filename. Use numeric prefixes (00-, 10-, 20-) to control order.

Can I filter which files to include?

- assemble:
    src: /etc/myapp/conf.d/
    dest: /etc/myapp/config.conf
    regexp: "^.*\\.conf$"  # Only .conf files

How do I add content between fragments?

Use delimiter parameter or use template with lookup('file') for full control.

Category: installation

Watch the video: Ansible Concatenate Files: Merge Multiple Files in Order (Guide) — Video Tutorial

Browse all Ansible tutorials · AnsiblePilot Home

AnsiblePilot — Master Ansible Automation

Popular Topics

About Luca Berton