Transforming JSON Data with Ansible and Jinja2

Introduction

In the world of IT automation, JSON data manipulation is a common task. Whether it’s reformatting data for API consumption or simplifying configurations, efficient handling of JSON is crucial. This article delves into leveraging Ansible and Jinja2 templates to transform JSON data dynamically.

We’ll tackle a specific example: renaming a JSON key while preserving the nested structure. By the end of this guide, you’ll be equipped to handle similar transformations in your automation workflows.

The Problem Statement

Here’s the source JSON we want to transform:

{
  "primary_network": {
    "aa:bb:cc": "xyz"
  }
}

The goal is to transform it into:

{
  "network": {
    "aa:bb:cc": "xyz"
  }
}

This involves renaming the top-level key from primary_network to network, keeping the data structure intact.

The Solution: Using Ansible and Jinja2

Ansible, combined with Jinja2 templates, provides a clean and reusable approach to solving this problem.

Ansible Playbook

Below is an example playbook that defines the input JSON and applies a Jinja2 template to transform it.

- name: Transform JSON with Jinja2
  hosts: localhost
  gather_facts: no
  vars:
    source_json:
      primary_network:
        "aa:bb:cc": "xyz"
  tasks:
    - name: Transform JSON
      template:
        src: transform_template.j2
        dest: transformed_output.json

    - name: Display the transformed JSON
      debug:
        msg: "{{ lookup('file', 'transformed_output.json') }}"

Jinja2 Template

Create a Jinja2 template named transform_template.j2 to perform the transformation.

{
  "network": {
    {% for key, value in source_json.primary_network.items() %}
    "{{ key }}": "{{ value }}"
    {% if not loop.last %}, {% endif %}
    {% endfor %}
  }
}

Explanation of the Process

1. Access the Source JSON:

   • The vars section in the playbook defines the source JSON under the variable source_json.

2. Jinja2 Transformation:

   • The template accesses primary_network and iterates over its key-value pairs using items().
   • It dynamically constructs the network JSON object while ensuring there are no trailing commas.

3. Output:

   • The transformed JSON is written to transformed_output.json.

Running the Playbook

To execute the playbook:

ansible-playbook transform_json.yml

Once executed, the playbook generates the transformed JSON in the specified file.

Troubleshooting and Tips

Common Errors

• Too Many Values to Unpack:

  • Ensure that primary_network is a dictionary. If it’s a list or another data type, adjust the template accordingly.

• Trailing Commas:

  • JSON doesn’t allow trailing commas. Use {% if not loop.last %} to handle this issue.

Final Output

The resulting file, transformed_output.json, will contain:

{
  "network": {
    "aa:bb:cc": "xyz"
  }
}

Why Use Ansible and Jinja2?

• Automation: Reduces manual effort and ensures consistency.
• Flexibility: Handles complex data transformations efficiently.
• Integration: Seamlessly integrates with Ansible workflows for broader automation.

Conclusion

Transforming JSON data with Ansible and Jinja2 is a robust approach for automating data manipulation. By mastering these tools, you can simplify configuration management, streamline data workflows, and enhance your DevOps practices.

Try it out and take your automation skills to the next level!

Academy

Learn the Ansible automation technology with some real-life examples in my Udemy 300+ Lessons Video Course.

My book Ansible By Examples: 200+ Automation Examples For Linux and Windows System Administrator and DevOps

Donate

Want to keep this project going? Please donate

Patreon Buy me a Pizza