Transition inventory into plugins (#23001)

* draft new inventory plugin arch, yaml sample

 - split classes, moved out of init
 - extra debug statements
 - allow mulitple invenotry files
 - dont add hosts more than once
 - simplified host vars
 - since now we can have multiple, inventory_dir/file needs to be per host
 - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
 - centralized localhost handling
 - added plugin docs
 - leaner meaner inventory (split to data + manager)
 - moved noop vars plugin
 - added 'postprocessing' inventory plugins
 - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
 - grouphost_vars loading as inventory plugin (postprocessing)
 - playbook_dir allways full path
 - use bytes for file operations
 - better handling of empty/null sources
 - added test target that skips networking modules
 - now var manager loads play group/host_vars independant from inventory
 - centralized play setup repeat code
 - updated changelog with inv features
 - asperioribus verbis spatium album
 - fixed dataloader to new sig
 - made yaml plugin more resistant to bad data
 - nicer error msgs
 - fixed undeclared group detection
 - fixed 'ungrouping'
 - docs updated s/INI/file/ as its not only format
 - made behaviour of var merge a toggle
 - made 'source over group' path follow existing rule for var precedence
 - updated add_host/group from strategy
 - made host_list a plugin and added it to defaults
 - added advanced_host_list as example variation
 - refactored 'display' to be availbe by default in class inheritance
 - optimized implicit handling as per @pilou's feedback
 - removed unused code and tests
 - added inventory cache and vbox plugin now uses it
 - added _compose method for variable expressions in plugins
 - vbox plugin now uses 'compose'
 - require yaml extension for yaml
 - fix for plugin loader to always add original_path, even when not using all()
 - fix py3 issues
 - added --inventory as clearer option
 - return name when stringifying host objects
 - ajdust checks to code moving

* reworked vars and vars precedence
 - vars plugins now load group/host_vars dirs
 - precedence for host vars is now configurable
 - vars_plugins been reworked
 - removed unused vars cache
 - removed _gathered_facts as we are not keeping info in host anymore
 - cleaned up tests
 - fixed ansible-pull to work with new inventory
 - removed version added notation to please rst check
 - inventory in config relative to config
 - ensures full paths on passed inventories

* implicit localhost connection local

docs/docsite/rst/intro_inventory.rst

@@ -5,22 +5,22 @@ Inventory
 
 .. contents:: Topics
 
-Ansible works against multiple systems in your infrastructure at the
-same time.  It does this by selecting portions of systems listed in
-Ansible's inventory file, which defaults to being saved in
-the location ``/etc/ansible/hosts``. You can specify a different inventory file using the
-``-i <path>`` option on the command line.
+Ansible works against multiple systems in your infrastructure at the same time.
+It does this by selecting portions of systems listed in Ansible's inventory,
+which defaults to being saved in the location ``/etc/ansible/hosts``.
+You can specify a different inventory file using the ``-i <path>`` option on the command line.
 
-Not only is this inventory configurable, but you can also use
-multiple inventory files at the same time (explained below) and also
+Not only is this inventory configurable, but you can also use multiple inventory files at the same time and
 pull inventory from dynamic or cloud sources, as described in :doc:`intro_dynamic_inventory`.
+Introduced in version 2.4, Ansible has inventory plugins to make this flexible and customizable.
 
 .. _inventoryformat:
 
 Hosts and Groups
 ++++++++++++++++
 
-The format for ``/etc/ansible/hosts`` is an INI-like format and looks like this:
+The inventory file can be in one of many formats, depending on the inventory plugins you have.
+For this example, the format for ``/etc/ansible/hosts`` is an INI-like (one of Ansible's defaults) and looks like this::
 
 .. code-block:: ini
 
@@ -118,6 +118,8 @@ Variables can also be applied to an entire group at once::
    ntp_server=ntp.atlanta.example.com
    proxy=proxy.atlanta.example.com
 
+Be aware that this is only a convenient way to apply variables to multiple hosts at once; even though you can target hosts by group, variables are always flattened to the host level before a play is executed.
+
 .. _subgroups:
 
 Groups of Groups, and Group Variables
@@ -149,8 +151,11 @@ It is also possible to make groups of groups using the ``:children`` suffix. Jus
    southwest
    northwest
 
-If you need to store lists or hash data, or prefer to keep host and group specific variables
-separate from the inventory file, see the next section.
+If you need to store lists or hash data, or prefer to keep host and group specific variables separate from the inventory file, see the next section.
+Child groups have a couple of properties to note:
+
+ - First, any host that is member of a child group is automatically a member of the parent group.
+ - Second, a child group's variables will have higher precedence (override) a parent group's variables.
 
 .. _default_groups:
 
@@ -228,7 +233,7 @@ ansible_connection
 
 .. include:: ../rst_common/ansible_ssh_changes_note.rst
 
-SSH connection:
+General for all connections:
 
 ansible_host
     The name of the host to connect to, if different from the alias you wish to give to it.
@@ -236,6 +241,10 @@ ansible_port
     The ssh port number, if not 22
 ansible_user
     The default ssh user name to use.
+
+
+Specific to the SSH connection:
+
 ansible_ssh_pass
     The ssh password to use (never store this variable in plain text; always use a vault. See :ref:`best_practices_for_variables_and_vaults`)
 ansible_ssh_private_key_file
@@ -252,10 +261,7 @@ ansible_ssh_extra_args
     This setting is always appended to the default :command:`ssh` command line.
 ansible_ssh_pipelining
     Determines whether or not to use SSH pipelining. This can override the ``pipelining`` setting in :file:`ansible.cfg`.
-
-.. versionadded:: 2.2
-
-ansible_ssh_executable
+ansible_ssh_executable (added in version 2.2)
     This setting overrides the default behavior to use the system :command:`ssh`. This can override the ``ssh_executable`` setting in :file:`ansible.cfg`.
 
 
@@ -295,7 +301,7 @@ ansible_shell_executable
     to use :command:`/bin/sh` (i.e. :command:`/bin/sh` is not installed on the target
     machine or cannot be run from sudo.).
 
-Examples from a host file::
+Examples from an Ansible-INI host file::
 
   some_host         ansible_port=2222     ansible_user=manager
   aws_host          ansible_ssh_private_key_file=/home/example/.ssh/aws.pem
@@ -360,3 +366,4 @@ Here is an example of how to instantly deploy to created containers::
        Questions? Help? Ideas?  Stop by the list on Google Groups
    `irc.freenode.net <http://irc.freenode.net>`_
        #ansible IRC chat channel
+

docs/docsite/rst/playbooks_variables.rst

@@ -835,12 +835,12 @@ In 1.x, the precedence is as follows (with the last listed variables winning pri
 In 2.x, we have made the order of precedence more specific (with the last listed variables winning prioritization):
 
   * role defaults [1]_
-  * inventory INI or script group vars [2]_
+  * inventory file or script group vars [2]_
   * inventory group_vars/all
   * playbook group_vars/all
   * inventory group_vars/*
   * playbook group_vars/*
-  * inventory INI or script host vars [2]_
+  * inventory file or script host vars [2]_
   * inventory host_vars/*
   * playbook host_vars/*
   * host facts