nxos_bfd_global / NxosCmdRef initial commit (#56317)

* nxos_bfd_global: initial commit

This is an initial POC with just a few commands included.  The code has been written somewhat generically so that it can act as a best practices template for re-use in future modules. The implementation follows the yaml cmd_ref style to define each command's getter/setter/type/default. It supports platform-specific defaults.

The basic logic is to collect all relevant data in a `cmd_ref` dict and pass that around to various methods.

In the BFD case the devices don't provide JSON output so we have to screen-scrape with show runs.

BFD does not support present/absent states so there is no state param.

BFD has three different property types to handle. We can add add'l types as needed:

- int
- int_list (list of ints)
- str (needs support for 'no' keyword)

* Use get_capabilities to find platform type

* PR comment fixes, round 1

* Minor cleanups

* nxos_bfd_global: create NxosCmdRef in module_utils

This commit just takes the latest bfd global code and moves the bulk
of the code into new `class NxosCmdRef` in `module_utils/nxos/nxos.py`.

The only remaining code in `nxos_bfd_global.py` are the calls from `main()`.

* Add remaining command properties and documentation

* update argument_spec

* Add check for _exclude; add sanity test

* Add targets files for bfd

* Context and state absent updates

* Add dict support to cmd_ref

* Changed remaining list commands to dict usage

* Add idempotence check for dict

* Fix existing overwrite bug

* Move pattern matching logic into its own method

* add support for 'command: absent'

* Add `get_platform_shortname`; update BFD platform-specific settings

* /absent/deleted/

* /sh/show/ in prepare_nxos_tests

* add dict check to get_platform_shortname

* Add normalize_defaults()

* UTs for bfd_global

* support yaml for both py2/py3

* update cmd_ref doc header

* Fix python2.6 incompatibility with dict comprehensions

* Fix bfd_global doc header (yaml syntax fail)

* more shippable fixes

* yet more shippable fixes

* shippable: remove r' ' wrappers

* docfix - remove ':'

* escape regex ctl chars in yaml table

* remove extra blank lines

* Fix str(None) issue

* Command context updates

* import PY2,PY3 instead of import sys

* fix ordereddict import & parent_context

* try/except for yaml import

* fix import issue for ordereddict

* remove epdb

* nxosCmdRef_import_check() workaround for shippable

* fix PEP ws errors

test/integration/targets/nxos_bfd_global/defaults/main.yaml

@@ -0,0 +1,2 @@
+---
+testcase: "*"

test/integration/targets/nxos_bfd_global/meta/main.yml

@@ -0,0 +1,2 @@
+dependencies:
+  - prepare_nxos_tests

test/integration/targets/nxos_bfd_global/tasks/cli.yaml

@@ -0,0 +1,27 @@
+---
+- name: collect common test cases
+  find:
+    paths: "{{ role_path }}/tests/common"
+    patterns: "{{ testcase }}.yaml"
+  connection: local
+  register: test_cases
+
+- name: collect cli test cases
+  find:
+    paths: "{{ role_path }}/tests/cli"
+    patterns: "{{ testcase }}.yaml"
+  connection: local
+  register: cli_cases
+
+- set_fact:
+    test_cases:
+      files: "{{ test_cases.files }} + {{ cli_cases.files }}"
+
+- name: set test_items
+  set_fact: test_items="{{ test_cases.files | map(attribute='path') | list }}"
+
+- name: run test cases (connection=network_cli)
+  include: "{{ test_case_to_run }} ansible_connection=network_cli connection={{ cli }}"
+  with_items: "{{ test_items }}"
+  loop_control:
+    loop_var: test_case_to_run

test/integration/targets/nxos_bfd_global/tasks/main.yaml

@@ -0,0 +1,3 @@
+---
+- { include: cli.yaml, tags: ['cli'] }
+- { include: nxapi.yaml, tags: ['nxapi'] }

test/integration/targets/nxos_bfd_global/tasks/nxapi.yaml

@@ -0,0 +1,33 @@
+---
+- name: collect common test cases
+  find:
+    paths: "{{ role_path }}/tests/common"
+    patterns: "{{ testcase }}.yaml"
+  connection: local
+  register: test_cases
+
+- name: collect nxapi test cases
+  find:
+    paths: "{{ role_path }}/tests/nxapi"
+    patterns: "{{ testcase }}.yaml"
+  connection: local
+  register: nxapi_cases
+
+- set_fact:
+    test_cases:
+      files: "{{ test_cases.files }} + {{ nxapi_cases.files }}"
+
+- name: set test_items
+  set_fact: test_items="{{ test_cases.files | map(attribute='path') | list }}"
+
+- name: run test cases (connection=httpapi)
+  include: "{{ test_case_to_run }} ansible_connection=httpapi connection={{ nxapi }}"
+  with_items: "{{ test_items }}"
+  loop_control:
+    loop_var: test_case_to_run
+
+- name: run test cases (connection=local)
+  include: "{{ test_case_to_run }} ansible_connection=local connection={{ nxapi }}"
+  with_items: "{{ test_items }}"
+  loop_control:
+    loop_var: test_case_to_run

test/integration/targets/nxos_bfd_global/tests/common/sanity.yaml

@@ -0,0 +1,152 @@
+---
+- debug: msg="START connection={{ ansible_connection }} nxos_bfd_global sanity test"
+- debug: msg="Using provider={{ connection.transport }}"
+  when: ansible_connection == "local"
+
+- name: set facts common
+  # nd_* vars are "non-default" values
+  set_fact:
+    echo: deleted
+    nd_echo: loopback1
+    interval: &def_interval
+      tx: 50
+      min_rx: 50
+      multiplier: 3
+    nd_interval:
+      tx: 51
+      min_rx: 52
+      multiplier: 4
+    slow: 2000
+    nd_slow: 2001
+
+- name: set facts (exclude 5K/6K)
+  set_fact:
+    echo_rx: 50
+    nd_echo_rx: 51
+    ipv4_echo_rx: 50
+    nd_ipv4_echo_rx: 54
+    ipv4_interval: *def_interval
+    nd_ipv4_interval: &nd_afi_interval
+      tx: 54
+      min_rx: 56
+      multiplier: 8
+    ipv4_slow: 2000
+    nd_ipv4_slow: 2044
+  when: platform is not search('N5K|N6K')
+
+- name: set facts (exclude 35/5K/6K)
+  set_fact:
+    ipv6_echo_rx: 50
+    nd_ipv6_echo_rx: 56
+    ipv6_interval: *def_interval
+    nd_ipv6_interval: *nd_afi_interval
+    ipv6_slow: 2000
+    nd_ipv6_slow: 2046
+  when: platform is not search('N35|N5K|N6K')
+
+- name: set facts (exclude 5K/6K/7K)
+  set_fact:
+    startup: 5
+    nd_startup: 6
+  when: platform is not search('N35|N5K|N6K|N7K')
+
+- name: set facts 3k defaults (resets some values above)
+  set_fact:
+    echo_rx: 250
+    interval: &n3k_def_interval
+      tx: 250
+      min_rx: 250
+      multiplier: 3
+    ipv4_echo_rx: 250
+    ipv6_echo_rx: 250
+    ipv4_interval: *n3k_def_interval
+    ipv6_interval: *n3k_def_interval
+    ipv4_slow: 2000
+    ipv6_slow: 2000
+  when: platform is search('N3K')
+
+- name: set facts fabricpath
+  set_fact:
+    fab_interval: *def_interval
+    nd_fab_interval:
+      tx: 57
+      min_rx: 57
+      multiplier: 7
+    fab_slow_timer: 2000
+    nd_fab_slow_timer: 2007
+    fab_vlan: 1
+    nd_fab_vlan: 47
+  when: platform is not search('N35|N3K|N9K')
+
+- name: Setup
+  nxos_feature: &setup_teardown
+    feature: bfd
+    provider: "{{ connection }}"
+    state: disabled
+  ignore_errors: yes
+
+- block:
+  - name: BFD non defaults
+    nxos_bfd_global: &bfd_non_def
+      echo_interface:        "{{ nd_echo }}"
+      echo_rx_interval:      "{{ nd_echo_rx | default(omit) }}"
+      interval:              "{{ nd_interval }}"
+      slow_timer:            "{{ nd_slow }}"
+      startup_timer:         "{{ nd_startup | default(omit) }}"
+      ipv4_echo_rx_interval: "{{ nd_ipv4_echo_rx | default(omit) }}"
+      ipv6_echo_rx_interval: "{{ nd_ipv6_echo_rx | default(omit) }}"
+      ipv4_interval:         "{{ nd_ipv4_interval | default(omit) }}"
+      ipv6_interval:         "{{ nd_ipv6_interval | default(omit) }}"
+      ipv4_slow_timer:       "{{ nd_ipv4_slow | default(omit) }}"
+      ipv6_slow_timer:       "{{ nd_ipv6_slow | default(omit) }}"
+      fabricpath_interval:   "{{ nd_fab_interval | default(omit) }}"
+      fabricpath_slow_timer: "{{ nd_fab_slow | default(omit) }}"
+      fabricpath_vlan:       "{{ nd_fab_vlan | default(omit) }}"
+      provider: "{{ connection }}"
+    register: result
+
+  - assert: &true
+      that:
+        - "result.changed == true"
+
+  - name: bfd_non_def idempotence
+    nxos_bfd_global: *bfd_non_def
+    register: result
+
+  - assert: &false
+      that:
+        - "result.changed == false"
+
+  - name: BFD defaults
+    nxos_bfd_global: &bfd_def
+      echo_interface:        "{{ echo }}"
+      echo_rx_interval:      "{{ echo_rx | default(omit) }}"
+      interval:              "{{ interval }}"
+      slow_timer:            "{{ slow }}"
+      startup_timer:         "{{ startup | default(omit) }}"
+      ipv4_echo_rx_interval: "{{ ipv4_echo_rx | default(omit) }}"
+      ipv6_echo_rx_interval: "{{ ipv6_echo_rx | default(omit) }}"
+      ipv4_interval:         "{{ ipv4_interval | default(omit) }}"
+      ipv6_interval:         "{{ ipv6_interval | default(omit) }}"
+      ipv4_slow_timer:       "{{ ipv4_slow | default(omit) }}"
+      ipv6_slow_timer:       "{{ ipv6_slow | default(omit) }}"
+      fabricpath_interval:   "{{ fab_interval | default(omit) }}"
+      fabricpath_slow_timer: "{{ fab_slow | default(omit) }}"
+      fabricpath_vlan:       "{{ fab_vlan | default(omit) }}"
+      provider: "{{ connection }}"
+    register: result
+
+  - assert: *true
+
+  - name: bfd_def idempotence
+    nxos_bfd_global: *bfd_def
+    register: result
+
+  - assert: *false
+
+  always:
+  - name: Teardown
+    nxos_feature: *setup_teardown
+    ignore_errors: yes
+
+- debug: msg="END connection={{ ansible_connection }} nxos_bfd_global sanity test"

test/integration/targets/prepare_nxos_tests/tasks/main.yml

@@ -43,7 +43,7 @@
 # Get image version information for this device
 - name: "Gather image version info"
   nxos_command:
-    commands: ['sh version | json']
+    commands: ['show version | json']
   connection: network_cli
   register: nxos_version_output
 
@@ -58,7 +58,7 @@
 #
 - name: "Gather platform info"
   nxos_command:
-    commands: ['sh inventory | json']
+    commands: ['show inventory | json']
   connection: network_cli
   register: nxos_inventory_output
 

test/units/modules/network/nxos/fixtures/nxos_bfd_global/N7K.cfg

@@ -0,0 +1,16 @@
+feature bfd
+
+bfd echo-interface loopback2
+bfd echo-rx-interval 56
+bfd interval 51 min_rx 52 multiplier 4
+bfd slow-timer 2001
+bfd startup-timer 6
+bfd ipv4 echo-rx-interval 54
+bfd ipv4 interval 54 min_rx 54 multiplier 4
+bfd ipv4 slow-timer 2004
+bfd ipv6 echo-rx-interval 56
+bfd ipv6 interval 56 min_rx 56 multiplier 6
+bfd ipv6 slow-timer 2006
+bfd fabricpath slow-timer 2008
+bfd fabricpath interval 58 min_rx 58 multiplier 8
+bfd fabricpath vlan 2

test/units/modules/network/nxos/fixtures/nxos_bfd_global/N9K.cfg

@@ -0,0 +1,13 @@
+feature bfd
+
+bfd echo-interface loopback2
+bfd echo-rx-interval 56
+bfd interval 51 min_rx 52 multiplier 4
+bfd slow-timer 2001
+bfd startup-timer 6
+bfd ipv4 echo-rx-interval 54
+bfd ipv4 interval 54 min_rx 54 multiplier 4
+bfd ipv4 slow-timer 2004
+bfd ipv6 echo-rx-interval 56
+bfd ipv6 interval 56 min_rx 56 multiplier 6
+bfd ipv6 slow-timer 2006

test/units/modules/network/nxos/test_nxos_bfd_global.py

@@ -0,0 +1,251 @@
+# (c) 2019 Red Hat Inc.
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+# Make coding more python3-ish
+from __future__ import (absolute_import, division, print_function)
+__metaclass__ = type
+
+from units.compat.mock import patch
+from ansible.modules.network.nxos import nxos_bfd_global
+from ansible.module_utils.network.nxos.nxos import NxosCmdRef
+from .nxos_module import TestNxosModule, load_fixture, set_module_args
+
+# TBD: These imports / import checks are only needed as a workaround for
+# shippable, which fails this test due to import yaml & import ordereddict.
+import pytest
+from ansible.module_utils.network.nxos.nxos import nxosCmdRef_import_check
+msg = nxosCmdRef_import_check()
+@pytest.mark.skipif(len(msg), reason=msg)
+class TestNxosBfdGlobalModule(TestNxosModule):
+
+    module = nxos_bfd_global
+
+    def setUp(self):
+        super(TestNxosBfdGlobalModule, self).setUp()
+
+        self.mock_load_config = patch('ansible.modules.network.nxos.nxos_bfd_global.load_config')
+        self.load_config = self.mock_load_config.start()
+
+        self.mock_execute_show_command = patch('ansible.module_utils.network.nxos.nxos.NxosCmdRef.execute_show_command')
+        self.execute_show_command = self.mock_execute_show_command.start()
+
+        self.mock_get_platform_shortname = patch('ansible.module_utils.network.nxos.nxos.NxosCmdRef.get_platform_shortname')
+        self.get_platform_shortname = self.mock_get_platform_shortname.start()
+
+    def tearDown(self):
+        super(TestNxosBfdGlobalModule, self).tearDown()
+        self.mock_load_config.stop()
+        self.execute_show_command.stop()
+        self.get_platform_shortname.stop()
+
+    def load_fixtures(self, commands=None, device=''):
+        self.load_config.return_value = None
+
+    def test_bfd_defaults_n9k(self):
+        # feature bfd is enabled, no non-defaults are set.
+        self.execute_show_command.return_value = "feature bfd"
+        self.get_platform_shortname.return_value = 'N9K'
+        set_module_args(dict(
+            echo_interface='deleted',
+            echo_rx_interval=50,
+            interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            slow_timer=2000,
+            startup_timer=5,
+            ipv4_echo_rx_interval=50,
+            ipv4_interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            ipv4_slow_timer=2000,
+            ipv6_echo_rx_interval=50,
+            ipv6_interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            ipv6_slow_timer=2000
+        ))
+        self.execute_module(changed=False)
+
+    def test_bfd_defaults_n3k(self):
+        # feature bfd is enabled, no non-defaults are set.
+        self.execute_show_command.return_value = "feature bfd"
+        self.get_platform_shortname.return_value = 'N3K'
+        set_module_args(dict(
+            echo_interface='deleted',
+            echo_rx_interval=250,
+            interval={'tx': 250, 'min_rx': 250, 'multiplier': 3},
+            slow_timer=2000,
+            startup_timer=5,
+            ipv4_echo_rx_interval=250,
+            ipv4_interval={'tx': 250, 'min_rx': 250, 'multiplier': 3},
+            ipv4_slow_timer=2000,
+            ipv6_echo_rx_interval=250,
+            ipv6_interval={'tx': 250, 'min_rx': 250, 'multiplier': 3},
+            ipv6_slow_timer=2000
+        ))
+        self.execute_module(changed=False)
+
+    def test_bfd_defaults_n35(self):
+        # feature bfd is enabled, no non-defaults are set.
+        self.execute_show_command.return_value = "feature bfd"
+        self.get_platform_shortname.return_value = 'N35'
+        set_module_args(dict(
+            echo_interface='deleted',
+            echo_rx_interval=50,
+            interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            slow_timer=2000,
+            startup_timer=5,
+            ipv4_echo_rx_interval=50,
+            ipv4_interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            ipv4_slow_timer=2000,
+        ))
+        self.execute_module(changed=False)
+
+    def test_bfd_defaults_n6k(self):
+        # feature bfd is enabled, no non-defaults are set.
+        self.execute_show_command.return_value = "feature bfd"
+        self.get_platform_shortname.return_value = 'N6K'
+        set_module_args(dict(
+            echo_interface='deleted',
+            interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            slow_timer=2000,
+            fabricpath_interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            fabricpath_slow_timer=2000,
+            fabricpath_vlan=1
+        ))
+        self.execute_module(changed=False)
+
+    def test_bfd_defaults_n7k(self):
+        # feature bfd is enabled, no non-defaults are set.
+        self.execute_show_command.return_value = "feature bfd"
+        self.get_platform_shortname.return_value = 'N7K'
+        set_module_args(dict(
+            echo_interface='deleted',
+            echo_rx_interval=50,
+            interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            slow_timer=2000,
+            ipv4_echo_rx_interval=50,
+            ipv4_interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            ipv4_slow_timer=2000,
+            ipv6_echo_rx_interval=50,
+            ipv6_interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            ipv6_slow_timer=2000,
+            fabricpath_interval={'tx': 50, 'min_rx': 50, 'multiplier': 3},
+            fabricpath_slow_timer=2000,
+            fabricpath_vlan=1
+        ))
+        self.execute_module(changed=False)
+
+    def test_bfd_existing_n9k(self):
+        module_name = self.module.__name__.rsplit('.', 1)[1]
+        self.execute_show_command.return_value = load_fixture(module_name, 'N9K.cfg')
+        self.get_platform_shortname.return_value = 'N9K'
+        set_module_args(dict(
+            echo_interface='deleted',
+            echo_rx_interval=51,
+            interval={'tx': 51, 'min_rx': 51, 'multiplier': 3},
+            slow_timer=2000,
+            startup_timer=5,
+            ipv4_echo_rx_interval=50,
+            ipv4_interval={'tx': 51, 'min_rx': 51, 'multiplier': 3},
+            ipv4_slow_timer=2000,
+            ipv6_echo_rx_interval=50,
+            ipv6_interval={'tx': 51, 'min_rx': 51, 'multiplier': 3},
+            ipv6_slow_timer=2000
+        ))
+        self.execute_module(changed=True, commands=[
+            'no bfd echo-interface loopback2',
+            'bfd echo-rx-interval 51',
+            'bfd interval 51 min_rx 51 multiplier 3',
+            'bfd slow-timer 2000',
+            'bfd startup-timer 5',
+            'bfd ipv4 echo-rx-interval 50',
+            'bfd ipv4 interval 51 min_rx 51 multiplier 3',
+            'bfd ipv4 slow-timer 2000',
+            'bfd ipv6 echo-rx-interval 50',
+            'bfd ipv6 interval 51 min_rx 51 multiplier 3',
+            'bfd ipv6 slow-timer 2000',
+        ])
+
+    def test_bfd_idempotence_n9k(self):
+        module_name = self.module.__name__.rsplit('.', 1)[1]
+        self.execute_show_command.return_value = load_fixture(module_name, 'N9K.cfg')
+        self.get_platform_shortname.return_value = 'N9K'
+        set_module_args(dict(
+            echo_interface='loopback2',
+            echo_rx_interval=56,
+            interval={'tx': 51, 'min_rx': 52, 'multiplier': 4},
+            slow_timer=2001,
+            startup_timer=6,
+            ipv4_echo_rx_interval=54,
+            ipv4_interval={'tx': 54, 'min_rx': 54, 'multiplier': 4},
+            ipv4_slow_timer=2004,
+            ipv6_echo_rx_interval=56,
+            ipv6_interval={'tx': 56, 'min_rx': 56, 'multiplier': 6},
+            ipv6_slow_timer=2006
+        ))
+        self.execute_module(changed=False)
+
+    def test_bfd_existing_n7k(self):
+        module_name = self.module.__name__.rsplit('.', 1)[1]
+        self.execute_show_command.return_value = load_fixture(module_name, 'N7K.cfg')
+        self.get_platform_shortname.return_value = 'N7K'
+        set_module_args(dict(
+            echo_interface='deleted',
+            echo_rx_interval=51,
+            interval={'tx': 51, 'min_rx': 51, 'multiplier': 3},
+            slow_timer=2002,
+            ipv4_echo_rx_interval=51,
+            ipv4_interval={'tx': 51, 'min_rx': 51, 'multiplier': 3},
+            ipv4_slow_timer=2002,
+            ipv6_echo_rx_interval=51,
+            ipv6_interval={'tx': 51, 'min_rx': 51, 'multiplier': 3},
+            ipv6_slow_timer=2002,
+            fabricpath_interval={'tx': 51, 'min_rx': 51, 'multiplier': 3},
+            fabricpath_slow_timer=2003,
+            fabricpath_vlan=3,
+        ))
+        self.execute_module(changed=True, commands=[
+            'no bfd echo-interface loopback2',
+            'bfd echo-rx-interval 51',
+            'bfd interval 51 min_rx 51 multiplier 3',
+            'bfd slow-timer 2002',
+            'bfd ipv4 echo-rx-interval 51',
+            'bfd ipv4 interval 51 min_rx 51 multiplier 3',
+            'bfd ipv4 slow-timer 2002',
+            'bfd ipv6 echo-rx-interval 51',
+            'bfd ipv6 interval 51 min_rx 51 multiplier 3',
+            'bfd ipv6 slow-timer 2002',
+            'bfd fabricpath interval 51 min_rx 51 multiplier 3',
+            'bfd fabricpath slow-timer 2003',
+            'bfd fabricpath vlan 3',
+        ])
+
+    def test_bfd_idempotence_n7k(self):
+        module_name = self.module.__name__.rsplit('.', 1)[1]
+        self.execute_show_command.return_value = load_fixture(module_name, 'N7K.cfg')
+        self.get_platform_shortname.return_value = 'N7K'
+        set_module_args(dict(
+            echo_interface='loopback2',
+            echo_rx_interval=56,
+            interval={'tx': 51, 'min_rx': 52, 'multiplier': 4},
+            slow_timer=2001,
+            ipv4_echo_rx_interval=54,
+            ipv4_interval={'tx': 54, 'min_rx': 54, 'multiplier': 4},
+            ipv4_slow_timer=2004,
+            ipv6_echo_rx_interval=56,
+            ipv6_interval={'tx': 56, 'min_rx': 56, 'multiplier': 6},
+            ipv6_slow_timer=2006,
+            fabricpath_interval={'tx': 58, 'min_rx': 58, 'multiplier': 8},
+            fabricpath_slow_timer=2008,
+            fabricpath_vlan=2,
+        ))
+        self.execute_module(changed=False)