Operational Guide: Epic Test Harness Sample Playbooks

This document explains how to operate, extend, and productionize the sample test harness playbooks under playbooks/epic-on-azure/test-harness/. It assumes familiarity with the core framework documented in the Epic Test Harness Role README. Rather than repeating that content, this guide links to the relevant sections and focuses on:

• How the sample playbooks are structured
• What each sample demonstrates ("art of the possible")
• How to adapt them with real environment data
• Patterns for scaling, securing, and integrating results
• Extensibility and operational best practices

Important: Suite and case names must be unique. See the role README note on uniqueness.

---

1. Relationship Between This Repository and the Test Harness Role

• This repository (playbooks) supplies example orchestration playbooks that import the role: ohemr-ansible-role-epic-tests.
• The role repository encapsulates:
  • Execution engine (dependency-aware ordering, result consolidation)
  • Individual test type task implementations (e.g., iris_check, port_check, sqlserver_privs_check)
  • Output normalization logic

Key role README sections to reference while authoring or modifying playbooks:

• Test suite and case structure: Test Suite Structure
• Dependencies: depends_on and Topological Sort
• Common test types: Common Test Types Table
• Result semantics: Test Case Result Types
• Extending the framework: Extending the Harness

---

2. Inventory of Current Sample Playbooks

All samples:

• Invoke task entry via tasks_from: setup/main.yml (which internally handles test case iteration, dependency sorting, and output).
• Tag with test:healthchecks (can be leveraged in AWX/Tower for filtering and scheduling).
• Supply a test_suites structure inline (override pattern).

---

3. Core Execution Flow (What Happens Under the Hood)

1. Playbook imports role with tasks_from: setup/main.yml.
2. test_suites variable (defined in the playbook or inventory/group_vars) is read.
3. Role builds a lookup of test cases and performs a topological sort (see tasks/setup/topological_sort.yml).
4. Each test case is executed:
   • If dependencies failed test marked as marked skipped.
   • Otherwise includes the corresponding test type task from tasks/<type>/main.yml.
5. Test case result objects are appended to test_results.
6. Suite-level aggregation is computed and pretty tables are emitted (see tasks/output/main.yml).

---

4. Output Handling & Post-Processing

The harness produces:

• Case-level table
• Suite summary table

To further operationalize:

• Capture raw test_results and test_suite_results facts after the role runs:

- name: Persist Test Results as JSON
  copy:
    dest: "artifacts/{{ inventory_hostname }}_test_results.json"
    content: "{{ test_results | to_nice_json }}"
  when: test_results is defined

- name: Persist Suite Summary
  copy:
    dest: "artifacts/test_suite_summary.json"
    content: "{{ test_suite_results | to_nice_json }}"
  run_once: true

You can then:

• Publish artifacts in AWX
• Feed into downstream reporting (Splunk, ELK, PowerBI)
• Gate CI/CD stages by parsing JSON to ensure critical tests pass

For machine-readable gating:

jq -e '.[] | select(.result=="failed")' artifacts/*_test_results.json && echo "Failures present" && exit 1

---

5. Creating Composite / Aggregated Playbooks

Instead of running one playbook per test category, you can aggregate:

---
- name: Full NonProd Readiness Suite
  hosts: all
  gather_facts: false
  vars_files:
    - group_vars/nonprod/sql.yml
    - group_vars/nonprod/iris.yml
  pre_tasks:
    - import_role:
        name: utilities
        tasks_from: get_vault_secrets.yml
  tasks:
    - import_role:
        name: ohemr-ansible-role-epic-tests
        tasks_from: setup/main.yml
      vars:
        test_suites:
          - name: IRIS Availability
            test_cases:
              - { name: IRIS OLD, type: iris_check, instance_name: OLDTST }
              - { name: IRIS POC, type: iris_check, instance_name: POC }
          - name: Network Edge
            test_cases:
              - {
                  name: Artifactory Reachable,
                  type: port_check,
                  host: repo1.uhc.com,
                  ports: [443],
                }
              - {
                  name: LDAP Secure,
                  type: port_check,
                  host: ad-ldap-ad001-centralus.uhc.com,
                  ports: [636],
                }
          - name: SQL Governance
            test_cases:
              - name: Server Privileges Conform
                type: sqlserver_privs_check
                sql_server: "{{ sql_server_primary }}"
                username: "{{ sql_admin_username }}"
                password: "{{ sql_sa_password }}"
                db_name: Kuiper
                expected_rows:
                  - {
                      name: "MS\\sharedepicsql",
                      type: WINDOWS_LOGIN,
                      permission: "CONNECT SQL",
                      state: GRANT,
                    }
                  - {
                      name: "sa",
                      type: SQL_LOGIN,
                      permission: "CONNECT SQL",
                      state: GRANT,
                    }

---

6. Enabling Verbose Diagnostic Output

Set:

vars:
  test_output:
    case_detail_truncation:
      test_title: 40
      optional_detail: 80

This leverages logic in tasks/output/main.yml to truncate table fields while still providing expanded debug messages where tasks include them.

---

7. Failure Handling & Exit Codes

Currently each test internal task uses failed_when: false, so Ansible run completes even with failing tests. To enforce pipeline gating:

Post-task:

- name: Fail build if any test failed
  fail:
    msg: "One or more test cases failed"
  when: gate_failures and (test_results | selectattr('result', 'equalto', 'failed') | list | length > 0)

You can parameterize gating with -e gate_failures=true.

---

8. Roadmap Ideas (Optional Enhancements)

---

9. Summary

Use these playbooks as reference implementations:

• Start small: clone an existing playbook and substitute environment-specific values.
• Layer in dependency chains where domain readiness must precede deeper tests.
• Export and gate on structured results to drive automated quality and reliability.

For deeper architectural or extension questions, refer to (or contribute to) the role’s README and docs set.

---