Skip bare metal testing in CI

When first added, the bare metal deployment tests using Tenks were
fairly reliable (although not 100%). Recently though, jobs seem to be
failing at this step quite frequently, making it difficult to merge
patches. Looking at the historical zuul jobs, the failure rate per job
seems to be about 1/3:
http://zuul.openstack.org/builds?job_name=kayobe-overcloud-centos. With
two overcloud jobs and check and gate required to pass, the merge
success rate is around 20%.

Let's skip this testing for now, until we can make it more reliable.

Change-Id: Id0a05f8c5d227f8cd9424c7a5b138eddee93f827
Story: 2005316
Task: 30224

playbooks/kayobe-overcloud-base/run.yml

@@ -26,7 +26,10 @@
         cmd: dev/overcloud-test-vm.sh > {{ logs_dir }}/ansible/overcloud-test-vm
         chdir: "{{ kayobe_src_dir }}"
 
-    - name: Perform testing of the baremetal machines
-      shell:
-        cmd: dev/overcloud-test-baremetal.sh > {{ logs_dir }}/ansible/overcloud-test-baremetal
-        chdir: "{{ kayobe_src_dir }}"
+    # NOTE(mgoddard): Bare metal testing in CI is currently very unreliable,
+    # making it difficult to merge patches. Skip this part of the testing until
+    # this can be improved.
+    # - name: Perform testing of the baremetal machines
+    #   shell:
+    #     cmd: dev/overcloud-test-baremetal.sh > {{ logs_dir }}/ansible/overcloud-test-baremetal
+    #     chdir: "{{ kayobe_src_dir }}"

playbooks/kayobe-overcloud-upgrade-base/run.yml

@@ -74,10 +74,13 @@
             cmd: dev/overcloud-test.sh > {{ logs_dir }}/ansible/overcloud-test-vm-pre-upgrade
             chdir: "{{ previous_kayobe_src_dir }}"
 
-        - name: Perform testing of the baremetal machines in the overcloud prior to upgrade
-          shell:
-            cmd: dev/overcloud-test-baremetal.sh > {{ logs_dir }}/ansible/overcloud-test-bm-pre-upgrade
-            chdir: "{{ kayobe_src_dir }}"
+        # NOTE(mgoddard): Bare metal testing in CI is currently very
+        # unreliable, making it difficult to merge patches. Skip this part of
+        # the testing until # this can be improved.
+        # - name: Perform testing of the baremetal machines in the overcloud prior to upgrade
+        #   shell:
+        #     cmd: dev/overcloud-test-baremetal.sh > {{ logs_dir }}/ansible/overcloud-test-bm-pre-upgrade
+        #     chdir: "{{ kayobe_src_dir }}"
 
         # Upgrade Kayobe, and use it to perform an upgrade of the control plane.
 
@@ -100,10 +103,13 @@
             cmd: dev/overcloud-test-vm.sh > {{ logs_dir }}/ansible/overcloud-test-vm-post-upgrade
             chdir: "{{ kayobe_src_dir }}"
 
-        - name: Perform testing of the baremetal machines in the upgraded overcloud
-          shell:
-            cmd: dev/overcloud-test-baremetal.sh > {{ logs_dir }}/ansible/overcloud-test-bm-post-upgrade
-            chdir: "{{ kayobe_src_dir }}"
+        # NOTE(mgoddard): Bare metal testing in CI is currently very
+        # unreliable, making it difficult to merge patches. Skip this part of
+        # the testing until # this can be improved.
+        # - name: Perform testing of the baremetal machines in the upgraded overcloud
+        #   shell:
+        #     cmd: dev/overcloud-test-baremetal.sh > {{ logs_dir }}/ansible/overcloud-test-bm-post-upgrade
+        #     chdir: "{{ kayobe_src_dir }}"
 
       environment:
         KAYOBE_CONFIG_SOURCE_PATH: "{{ kayobe_config_src_dir }}"