My previous post was about creating Ansible modules for NSX-v with Powershell. Today I will show how to skip Powershell and speed up the process significantly. I will do this by using the Ansible uri module and NSX-v RestAPI.

Background

Making your own Ansible modules with Powershell is pretty neat and cool. The big downside I experienced though is deployment time. Using a Powershell jumphost really slowed things down for me. Also it requires quite a bit extra configuration to make it work. I got two questions. Can this be done easier and can you make it go any faster? So as it turns out, no for easier and yes for faster. And I do have to say, the ‘no’ for easier totally depends on perception and experience. What I want to do is to leverage the NSX-v RestAPI into Ansible with the uri module. This cuts away the need for Powershell and extra configuration. Also talking directly to the NSX-v RestAPI should make deployment a lot faster.

Goal

I want the script to create firewall rules from a file. The firewall rule contains one securitygroup as source and destination. This is because I am creating micro-segmentations in a zero-trust firewall. The rule is added in a disabled state. The section where the rule is created is fixed and this is done to avoid freedom and errors in generating sections.

pre-requisites

Actually this list is not too long, but you should be aware that some config is needed.

• NSX for vSphere 6.4.5 or higher (tested)
• Ansible controlhost. Tested 2.9.1
• NSX for vSphere CLI user with web access – zie deze link
• Existing Section to create rules in
• Empty dummy rule (some bug i can’t work around)
• Securitygroups should be present

Little note on the dummy rule, but for some reason the script fails when the section is empty. I certainly would appreciate some help !

Ofcourse it is possible to automate securitygroups as well. For today’s example I won’t go into that though.

Road to success

Well, diving into the NSX for vSphere API turned out to be quite a deep-dive. There were some resources I could find, but not many. Especially when it comes to integrating the API into Ansible. I have to mention the RAML tool as wel. This tools templates the Rest-API to be used in Python. Which makes it possible to make your own Python modules. For reasons I have not gone that path, but it’s quite a nice tool if you are interested.

After lots of testing and sending commands to NSX-v with Postman I felt ready to leverage commands to the Uri module. There is one really icky thing though about this module. It likes JSON, but XML not so much. So I had quite a challenge in using and reading the information from the API.

The other thing that made life a little less easy was the Etag (Entity tag). Every iteration that causes a resource change also causes the Etag number to change. You can only make changes against resources with the current (running script) Etag id. This means the script needs to search for the Etag every iteration.

It was a lot of work, but I did not give up! Needed to dig deep on some issues and sometimes it was pretty frustrating and infuriating. In the end though I got it all working. The script you see today is just one (edited) part of a larger deployment script. In the future I might write more on how to glue everything together.

Notes on the code

Before pasting code there are a few things to mention. The code is pretty raw and sometimes quite complex. The code has comments on what is happening so you will know what is going on. Also I am very open to suggestions. When you feel things can be done better or more efficient I would love to hear from you.

Structure of the code

The code exists of three scripts.
1. nsx-firewallrule-vars
2. nsx-firewallrule-playbook
3. nsx-firewallrule-create
This scripts can reside in the same directory, but you can adjust this to your needs. The first script contains the variables in a dict. The second script is the playbook that includes the variables and runs the create script. It contains a loop to make sure that only one dict item gets passed to the create script each iteration. This is necessary because of dependencies within the create script.

In the create script you need to manually add the NSX Manager plus credentials. These are not variables in this example.

Code used for the script

The vars file

fwrules:
  'rule_cc_01':
    section_name: Micro Segmentation Codecrusaders
    secgroup_name: sg_codecrusaders_01
  'rule_cc_02':
    section_name: Micro Segmentation Codecrusaders
    secgroup_name: sg_codecrusaders_02
  'rule_cc_03':
    section_name: Micro Segmentation Codecrusaders
    secgroup_name: sg_codecrusaders_03

The above is a dictionary. Define rules to your liking. Be aware that for this example the script does not create security groups or sections. They need to exist. In an empty section please add a (temporary) dummy rule.

The playbook file

---
- name: playbook for adding rules to Sections
  hosts: localhost

  pre_tasks:
    - name: include vars from dict
      include_vars: nsx-firewallrule-vars.yml

  tasks:
    - name: create some firewall rules
      include_tasks: nsx-firewallrule-create.yml
      loop: "{{ query('dict',fwrules) }}"

This file states to include the vars created in the previous file and it also it includes the create file with a loop. This is to make sure one rule at a time will be processed.

The create file

# nsx-firewallrule-create.yml
---
- name: retrieve securitygroup information
  uri:  
    url: https://<your-nsx-manager>/api/2.0/services/securitygroup/scope/globalroot-0
    method: GET
    return_content: yes
    user: <nsx_api_web_user>
    password: <nsx_api_web_user_password>
    headers:
      Content-Type: "application/xml"
    force_basic_auth: yes
    validate_certs: no
    status_code: 200
  register: output_secgroup

- name: get secgroup-attribute from xml content
  xml:
    xmlstring:  "{{ output_secgroup.content }}"
    xpath: /list/securitygroup[name='{{ item.value.secgroup_name }}']/objectId
    content: 'text'
  register: secgroup_attributes

- name: set securitygroup-id as fact
  set_fact: 
    secgroup_id: "{{ secgroup_attributes.matches.0.objectId }}"
  register: secgroup_results

- name: make urls machine readable
  set_fact: section="{{ item.value.section_name | replace(' ','%20') }}"

- name: check for dfw etag and section ID
  uri:
    url: https://<your-nsx-manager>/api/4.0/firewall/globalroot-0/config/layer3sections?name={{ section }}
    method: GET
    return_content: yes
    user: <nsx_api_web_user>
    password: <nsx_api_web_user_password>
    body_format: raw
    headers:
      Content-Type: "application/xml"
    force_basic_auth: yes
    validate_certs: no
    status_code: 200
  register: l3section 

- name: get section_id xml content
  xml:
    xmlstring: "{{ l3section.content }}"
    xpath: /sections/section
    content: attribute
  register: section_attributes

- name: set the section id needed for use in API
  set_fact: 
    section_id: "{{ section_attributes.matches.0.section.id|default(['post']) }}"

- name: set the Etag id
  set_fact:
    etag_id: "{{section_attributes.matches.0.section.generationNumber|default(['post']) }}"
    # etag_id: "{{ l3section.etag }}" -> not working since 6.4.5 because of double-double quotes issue

# below debugs are just informational to show you the section id and the etag id
- debug:
    msg: "{{ section_id }}"

- debug:
    msg: "{{ etag_id }}"

- name: get rule_name xml content
  xml:
    xmlstring: "{{ l3section.content }}"
    xpath: /sections/section/rule[sectionId={{ section_id }}]/name
    content: text
  register: section_attributes_rule

- name: search for existing rules
  set_fact: 
    rule_dict: "{{ section_attributes_rule['matches'] | selectattr('name','search', item.key|string ) | list | default([]) }}"
  register: rules

- name: set rule name if found - optional task
  set_fact:
    rule_name: "{{ rule_dict[0].name }}"
  when: rule_dict != []

- name: create rule (POST)
  uri:
    url: https://<your-nsx-manager>/api/4.0/firewall/globalroot-0/config/layer3sections/{{ section_id }}/rules
    method: POST
    return_content: yes
    user: <nsx_api_web_user>
    password: <nsx_api_web_user_password>
    body_format: raw
    headers:
      Content-Type: "application/xml"
      If-Match: "{{ etag_id }}"
    body: 
      <rule disabled="true" logged="false">
        <name>{{ item.key }}</name>
        <action>allow</action>
        <sectionId>{{ section_id }}</sectionId>
        <sources excluded="false">
          <source>
            <value>{{ secgroup_id }}</value>
            <type>SecurityGroup</type>
          </source>
        </sources>
        <destinations excluded="false">
          <destination>
            <value>{{ secgroup_id }}</value>
            <type>SecurityGroup</type>
          </destination>
        </destinations>
      </rule>
    force_basic_auth: yes
    validate_certs: no
    status_code: 201
  register: l3section_rule
  when: rule_dict == []
  changed_when: l3section_rule.status == 201

This code actually creates the firewall rules and does so one by one. The code is based on the Ansible uri module and the NSX-v RestAPI. The dummy issue probably happens because of a coding error somewhere in getting xml content. I will look into it, but feel free to support.

Result

When run this code it is fast and idempotent. With the ‘old’ Powershell modules it could take up minutes per rule to deploy. This new code brings that back to a few seconds. When automating large numbers of rules this is a big win !

Conclusion

It is quite a bit of work to sort the API out and get it to work. Sometimes frustrating and even infuriating when bumping into the next error. That being said, it is and was totally worth the effort. It is always a good feeling when you go back from minutes to seconds, especially in automation land.
Thanks for reading!

If you want to read more about Ansible and Powershel modules, please read:
https://www.codecrusaders.nl/vmware/powershell-to-write-ansible-modules-for-nsx-v/

Code is also on github here:
https://github.com/davyvandelaar/devops/tree/master/playbooks/nsxv_api