Commit Graph

218 Commits

Author SHA1 Message Date
James Cammarata
f4237b2151 Rework the way params are assigned to TaskIncludes when they're dynamic
Copying the TaskInclude task (which is the parent) before loading the blocks
makes the code much more simple and clean, and fixes a bug introduced during
the performance improvement changes (and specifically the change which moved
things to a single-parent model).

Fixes #17064
2016-08-16 15:45:24 -05:00
James Cammarata
f722d41eab Allow notifies to be sent to the top level includes when they were static
Since we introduced static includes in 2.1, this broke the functionality
where a notify could be sent to a named include statement, triggering all
handlers contained within the include. This patch fixes that by adding a
search through the parents of a handler for any TaskIncludes which match.

Fixes #15915
2016-08-15 14:22:35 -05:00
James Cammarata
c669a381d1 Don't immediately return failed for any_errors_fatal tasks
Instead of immediately returning a failed code (indicating a break in
the play execution), we internally 'or' that failure code with the result
(now an integer flag instead of a boolean) so that we can properly handle
the rescue/always portions of blocks and still remember that the break
condition was hit.

Fixes #16937
2016-08-12 14:18:50 -05:00
James Cammarata
925b0ff9e9 cleaning up some debug statements in _queue_task() 2016-08-11 14:26:28 -05:00
James Cammarata
1c7e0c73c9 Several fixes for includes
* when including statically, make sure that all parents were also included
  statically (issue #16990)
* properly resolve nested static include paths
* print a message when a file is statically included

Fixes #16990
2016-08-11 14:07:49 -05:00
James Cammarata
68d44e48ba Remove sleeps and rely on regular queue options to process results 2016-08-11 09:11:32 -05:00
James Cammarata
06d4f4ad0e Move tasks/blocks to a single parent model 2016-08-08 15:58:46 -05:00
James Cammarata
d2b3b2c03e Performance improvements 2016-08-08 15:58:46 -05:00
James Cammarata
1714279b5e Tweak the way the debug strategy imports the linear strategy parent
Due to the way we load plugins, internally to Python there can be issues when
the debug strategy is loaded after the linear strategy. To work around this,
we're changing the import line for the linear strategy to avoid the problem.

Related to #16825
2016-08-05 10:06:50 -05:00
jctanner
238c6461f6 Add a function to check for killed processes in all strategies (#16684)
* Add a function to check for killed processes so that if any
threads are sigkilled or sigtermed, the entire playbook execution is aborted.
2016-07-14 16:37:35 -04:00
James Cammarata
245ce9461d Fix unreachable host/any_errors_fatal bug in linear strategy
2e003adb added the ability for tasks using any_errors_fatal to fail
when there were unreachable hosts. However that patch used the running
unreachable hosts data rather than the results from the current task,
which causes failures when any run_once or BYPASS_HOST_LOOP task is hit
after an unreachable host causes a failure. This patch corrects that by
using the current set of results to determine if any hosts were
unreachable during the last task only.

Fixes ansible/ansible-modules-core#4160
2016-07-12 03:01:47 -05:00
James Cammarata
930d090507 Fix the way handlers are compiled and found/notified
* Instead of rebuilding the handler list all over the place, we now
  compile the handlers at the point the play is post-validated so that
  the view of the play in the PlayIterator contains the definitive list
* Assign the dep_chain to the handlers as they're compiling, just as we
  do for regular tasks
* Clean up the logic used to find a given handler, which is greatly
  simplified by the above changes

Fixes #15418
2016-07-06 14:57:38 -05:00
James Cammarata
17447ff035 Fixing typo in base strategy code from handler listen feature merge 2016-06-20 09:47:43 -05:00
James Cammarata
ca6ee4c789 FEATURE: handler listeners
Fixes ansible/proposals#8
2016-06-20 09:30:20 -05:00
James Cammarata
8218591fec Track notified handlers by object rather than simply their name
Due to the fact that roles may be instantiated with different sets of
params (multiple inclusions of the same role or via role dependencies),
simply tracking notified handlers by name does not work. This patch
changes the way we track handler notifications by using the handler
object itself instead of just the name, allowing for multiple internal
instances. Normally this would be bad, but we also modify the way we
search for handlers by first looking at the notifying tasks dependency
chain (ensuring that roles find their own handlers first) and then at
the main list of handlers, using the first match it finds.

This patch also modifies the way we setup the internal list of handlers,
which should allow us to correctly identify if a notified handler exists
more easily.

Fixes #15084
2016-06-20 09:30:20 -05:00
Remi Ferrand
9c3447c619 Properly template task names in free strategy
* Fixes #16295
2016-06-15 09:49:04 +02:00
James Cammarata
fbec2d9692 Expand return code values returned by TQM and strategies
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.

Fixes #15523
2016-06-08 10:21:57 -05:00
James Cammarata
1f18f5c92d Tweak the poll time to 0.005 to improve perf with large playbooks 2016-06-06 11:24:32 -05:00
James Cammarata
b9e743b478 Merge branch 'devel' of https://github.com/andreyshestakov/ansible into andreyshestakov-devel 2016-06-06 11:03:28 -05:00
James Cammarata
560532da7d Make sure we add host/group vars files when parsing inventory
Also fixes a bug where add_host was not adding the vars files

Fixes #16117
2016-06-06 10:52:39 -05:00
Andrey Shestakov
295c7ce1c8 Fix sleep intervals (14219) 2016-05-31 09:33:58 +03:00
James Cammarata
74a252d0a5 More playbook executor compatibility fixes
With some earlier changes, continuing to forward failed hosts on
to the iterator with each TQM run() call was causing plays with
max_fail_pct set to fail, as hosts which failed in previous plays
were counting those old failures against the % calculation.

Also changed the linear strategy's calculation to use the internal
failed list, rather than the iterator, as this now represents the
hosts failed during the current run only.
2016-05-18 14:09:21 -04:00
James Cammarata
0f659d699e Re-remove checking for failed state on hosts when building list of hosts
This was re-added by 63471cd (and modified by me to use iterator again),
it simply needs to be removed.

Fixes #15395
2016-05-18 08:17:39 -04:00
Vincent Roy
59d3ae6136 Don't let max_fail_percentage affect future plays. 2016-05-18 08:17:39 -04:00
Vincent Roy
994e57fa2b Handle max_fail_percentage per task. 2016-05-18 08:17:39 -04:00
Vincent Roy
ba89b2a095 Backward compatibility execution failures with 1.9 2016-05-18 08:17:39 -04:00
James Cammarata
d2bade6daf Make sure setting facts with run_once makes copies of the data
When using run_once, there is only one dict of facts so passing that
to the VariableManager results in the fact cache containing the same
dictionary reference for all hosts in inventory. This patch fixes that
by making sure we pass a copy of the facts dict to VariableManager.

Fixes #14279
2016-05-13 10:05:19 -04:00
James Cammarata
d391c53b4f Clear blocked hosts when a role duplicate task is found in free strategy
In the free strategy, we mark a host as blocked when it has work to do
(the PlayIterator returns a task) to prevent multiple tasks from being sent
to the host. However, we check for role duplicates after setting the blocked
flag, but were not clearing that when the task was skipped leading to an
infinite loop. This patch corrects that by clearing the blocked flag when
the task is skipped.

Fixes #15681
2016-05-12 10:38:53 -04:00
Brian Coca
47d05e9b58 fix bad assignment, method modifies by ref already
fixes #15694
2016-05-10 12:08:03 -04:00
James Cammarata
09c90f7f2f Fixing bugs in strategies
* Don't filter hosts remaining based on their failed state. Instead rely
  on the PlayIterator to return None/ITERATING_COMPLETE when the host is
  failed.
* In the free strategy, make sure we wait outside the host loop for all
  pending results to be processed.
* Use the internal _set_failed_state() instead of manually setting things
  when a failed child state is hit

Fixes #15623
2016-05-06 09:22:11 -04:00
James Cammarata
1fc44e4103 Don't fail hosts when using any_errors_fatal when ignoring errors
Fixes #15431
2016-05-01 12:40:09 -04:00
Toshio Kuratomi
98feafb411 Fix the mapping of module_name to Locks
This was reinitialized every time we forked before so we weren't sharing
the same Locks.  It also was not accounting for modules which were
directly invoked by an action plugin instead of going through the
strategy plguins.
2016-04-29 20:47:51 -07:00
James Cammarata
1211a0fa12 Fixing the way we iterate over child states for tasks
Previously we were first checking the fail/run state of the child
state for tasks/rescue/always portions of the block. Instead we are now
always recursively iterating over the child state and then evaluating
whether the child state is failed or complete before changing the failed/
run state within the current block.

Fixes #14324
2016-04-25 11:13:44 -04:00
James Cammarata
0e2f1b423d Merge pull request #15455 from chemikadze/devel
Fix duplicate host with different uuid created after add_host
2016-04-19 02:50:27 -04:00
James Cammarata
6eefc11c39 Make the loop variable (item by default) settable per task
Required for include+with* tasks which may include files that also
have tasks containing a with* loop.

Fixes #12736
2016-04-19 01:42:40 -04:00
Nikolay Sokolov
d8867d7ce3 Fix duplicate host with different uuid created after add_host
If add_host is performed with hostname existing in inventory, but
not yet accessed and put in inventory cache, additional host with
same hostname and different uuid is created, causing patterns to
misbehave.
2016-04-18 00:10:09 -07:00
Toshio Kuratomi
dcc5dfdf81 Controller-side module caching.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.

Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data.  Should devise a separate test for the module and caching code
2016-04-12 08:01:07 -07:00
Toshio Kuratomi
54acdd7ead Remove workaround for fixed bug. (#15340)
* Remove workaround for fixed bug.

The bug where PluginLoader required objects to directly inherit from
base_classes has been fixed.  Remove workaround from this strategy
plugin   Also switched to using super so that we don't have to modify
all of hte code anytime something like that happens.

* These should be to_uniocde because they're being sent to display()
2016-04-10 09:24:54 -07:00
Kishin Yagami
e4a6106ea5 Add debug strategy plugin (#15125)
* Add debug strategy plugin

* Fix Python 2-3 compatiblity issue

* Add document for debug strategy
2016-04-08 14:39:08 -04:00
James Cammarata
e5c4f88b71 Fix free strategy, which was incorrectly using _wait_on_pending_results
Broken in 210cf06d
2016-04-08 06:49:16 -04:00
James Cammarata
0f2b1244d2 Make sure VariableManager has a view of HostVars
Fixes #15261
2016-04-05 11:32:28 -04:00
James Cammarata
0656d1466a Adds two variables to facts when a task fails and a rescue block starts
`ansible_failed_task`:
   Contains the task data, essentially a serialized view of the Task() object.
`ansible_failed_result`:
   Contains the result of the task failure, which is identical in function
   to registering the result. Doing so automatically like this saves the user
   from having to register every result in a block and then trying to figure
   out which result had the failure

Similar to the way try/except/finally work, these variables will not be
available in the `always` portion of a block unless there is a corresponding
`rescue` first.

Fixes #12341
2016-03-31 15:29:33 -04:00
James Cammarata
a8dbd3dee9 Pass the variable manager into load_list_of_blocks for include files
Fixes #15194
2016-03-31 00:44:41 -04:00
Brian Coca
5dfc013e18 Merge pull request #14971 from bcoca/clear_facts
meta: clear_facts
2016-03-30 19:09:38 -07:00
James Cammarata
13f3cbaf3b Renaming per-item and retry callbacks 2016-03-18 15:17:43 -04:00
qqo
a7f89cac63 fix TypeError: coercing to Unicode: need string or buffer, AnsibleParserError found 2016-03-17 00:56:47 +03:00
Brian Coca
09699d0cd0 meta: clear_facts
draft implementation
2016-03-14 21:24:26 -07:00
Brian Coca
49bb4803b5 clarify --step prompt
now shows full words and indicates default
fixes #7433
2016-03-08 10:57:54 -05:00
Brian Coca
66ea464ebd Merge pull request #14793 from xiaket/devel
remove main_q for simplicity.
2016-03-07 12:51:38 -05:00
Brian Coca
25e9b5788b add per item diff handling
fixes #14843
2016-03-07 09:49:18 -05:00