Skip to content
Snippets Groups Projects
  • Mark Goddard's avatar
    6c54ce4d
    Introduce max fail percentage to playbooks · 6c54ce4d
    Mark Goddard authored
    This allows us to continue execution until a certain proportion of hosts
    fail. This can be useful at scale, where failures are common, and
    restarting a deployment is time-consuming.
    
    The default max failure percentage is 100, keeping the default
    behaviour. A global max failure percentage may be set via
    kayobe_max_fail_percentage, and individual playbooks may define a max
    failure percentage via <playbook>_max_fail_percentage.
    
    Related Kolla Ansible patch:
    https://review.opendev.org/c/openstack/kolla-ansible/+/805598
    
    Change-Id: Ib81c72b63be5765cca664c38141ffc769640cf07
    6c54ce4d
    History
    Introduce max fail percentage to playbooks
    Mark Goddard authored
    This allows us to continue execution until a certain proportion of hosts
    fail. This can be useful at scale, where failures are common, and
    restarting a deployment is time-consuming.
    
    The default max failure percentage is 100, keeping the default
    behaviour. A global max failure percentage may be set via
    kayobe_max_fail_percentage, and individual playbooks may define a max
    failure percentage via <playbook>_max_fail_percentage.
    
    Related Kolla Ansible patch:
    https://review.opendev.org/c/openstack/kolla-ansible/+/805598
    
    Change-Id: Ib81c72b63be5765cca664c38141ffc769640cf07
network-connectivity.yml 2.95 KiB
---
- name: Check network connectivity between hosts
  hosts: seed:seed-hypervisor:overcloud:infra-vms
  max_fail_percentage: >-
    {{ network_connectivity_max_fail_percentage |
       default(kayobe_max_fail_percentage) |
       default(100) }}
  vars:
    # Set this to an external IP address to check.
    nc_external_ip: 8.8.8.8
    # Set this to an external hostname to check.
    nc_external_hostname: google.com
    # Number of bytes to subtract from MTU to allow for ICMP (8 bytes) and IP
    # (20 bytes) headers.
    icmp_overhead_bytes: 28
  tasks:
    - name: Display next action
      debug:
        msg: >
          Checking whether hosts have access to an external IP address,
          {{ nc_external_ip }}.
      run_once: True

    - name: Ensure an external IP is reachable
      command: ping -c1 {{ nc_external_ip }}
      changed_when: False

    - name: Display next action
      debug:
        msg: >
          Checking whether hosts have access to an external hostname,
          {{ nc_external_hostname }}.
      run_once: True

    - name: Ensure an external host is reachable
      command: ping -c1 {{ nc_external_hostname }}
      changed_when: False

    - name: Display next action
      debug:
        msg: >
          Checking whether hosts have access to any configured gateways.
      run_once: True

    - name: Ensure the gateway is reachable
      command: >
        ping {{ item | net_gateway }} -c1 -M do {% if mtu %} -s {{ mtu | int - icmp_overhead_bytes }}{% endif %}
      with_items: "{{ network_interfaces }}"
      when:
        - item | net_ip
        - item | net_gateway
      changed_when: False
      vars:
        mtu: "{{ item | net_mtu }}"

    # For each network on this host, pick a random remote host also on the
    # network and try to ping it. Set the packet size according to the
    # network's MTU.

    - name: Display next action
      debug:
        msg: >
          Checking whether hosts have access to other hosts on the same
          network.
      run_once: True

    - name: Ensure hosts on the same network are reachable
      command: >
        ping {{ remote_ip }} -c1 -M do {% if mtu %} -s {{ mtu | int - icmp_overhead_bytes }}{% endif %}
      with_items: "{{ network_interfaces }}"