Capability Acquisition Management:
Modifying the Acquisition Process to Emphasize I&I
________________________________________________________
By Joel Washington
Maintaining alignment of the myriad of program acquisition activities spanning multiple program offices, and often across different Program Executive Offices (PEOs), to realize a complete mission thread of warfighting capability, is a daunting endeavor. Success means managing and maintaining alignment of specific interdependent system attributes of the mission thread during system development through what has traditionally been an asynchronous set of technical and programmatic activities. This is one of the four critical core areas the Naval Surface Warfare Center, Dahlgren Division (NSWCDD), Integration and Interoperability (I&I) Mission Engineering team, sponsored by the Deputy Assistant Secretary of the Navy Research, Development, Test and Evaluation has had to address:
| 1 |
Integrated Capability Framework
Overarching I&I task structure |
| 2 |
Capability Solution Management
Finding solutions to gaps |
| 3 |
Capability Acquisition Management
Managing the production of solutions |
| 4 |
Alignment of I&I Processes
Identifying stakeholders and best practices |
This article describes the Capability Acquisition Management (CAM) Integrated Product Team (IPT) tasks, which are aimed at fusing force-level I&I considerations throughout the system acquisition decision process by modifying Systems Engineering Technical Review (SETR), Gate Review, and Probability of Program Success (PoPS) criteria.
Background
I&I represents synchronization of a series of deliberate steps to provide an end-to-end capability view from an “effects chain” perspective. Where possible, the I&I approach leverages existing Department of Navy processes (e.g., Joint Capabilities Integration Development System, Capability-Based Acquisition, Office of the Chief of Naval Operation’s (OPNAV) N81 Warfighting Capability Plan, and the acquisition analysis of alternatives (AoA). I&I also provides a disciplined Doctrine, Organization, Training, Materiel, Leadership and Education, Personnel, Facilities and Interoperability (DOTMLPF-I) analysis to develop issues based on empirical data, facilitate solution recommendations to achieve effects chain wholeness, and align I&I recommendations to Navy leadership’s informed budget and acquisition/investment decisions. The overarching goal is to deliver strategic readiness by ensuring the Fleet is capable of successfully executing the most important effects chains that, together, form the Navy’s core contribution to Combatant Commander Operation Plans.
I&I contributors consist of system commands (SYSCOMs), warfare centers, OPNAV, Commander Operational Test and Evaluation Force (COTF), and the Fleet. The Fleet provides mission area advocacy from establishing the I&I demand signal through processing approved solutions. The COTF provides the framework and focal point for expertise to accomplish selected effects chain assessments. SYSCOMs provide the technical rigor to accomplish I&I materiel issue assessments. OPNAV and the Assistant Secretary of the Navy (Research, Development, and Acquisition) [ASN(RDA)] provide the expertise and options to leverage the full range of processing and implementation options for I&I solutions as illustrated in Figure 1.
The CAM IPT was specifically tasked to recommend policy changes to support mission engineering, incorporate rigor into mission engineering aspects of the acquisition process, and to mitigate I&I issues. Candidate policies for change were found in the Technical Review Manual, Naval Sea Systems (NAVSEA) instructions, Naval Systems Engineering Technical Review Handbook, and OPNAV instructions. Emphasis was placed on the criteria required to support mission thread and kill chain execution.
In looking at potential policy changes, the CAM IPT addressed the full end-to-end traceability of mission capability requirements in specific domains by examining:
- Data
- Interfaces
- Decomposition/allocation of end-to-end technical requirements
- Dependencies
- Commonality
Any proposed policy changes would be vetted against and help ensure compliance with overarching I&I acquisition policy statements in SECNAVISNT 5000.2E, “Department of the Navy Implementation and Operation of the Defense Acquisition System and the Joint Capabilities Integration and Development System,” 1 September 2011:
PMs shall ensure the (I&I) of all operations, functions, system interfaces, data, software based services, distributed decision-making systems, human processing capabilities, situational awareness systems, and other systems to reflect the requirement for all system elements: hardware, software, facilities, sustainment infrastructure, personnel and data. Interoperability shall be addressed by including system of systems (SoS) or federation of systems (FoS) considerations ....(6.1.6 Interoperability and Integration)
The CAM IPT settled on modifying three interrelated target areas that would then be codified through proposed policy changes: specifically, the SETR process for technical monitoring capability development; the 2-Pass, 6-Gate acquisition management schema for decision control measures; and finally the PoPS dashboard for monitoring program health assessment and reporting. We will explore the CAP IPTs efforts in each of these three areas in this article.
Systems Engineering Technical Review
The first area the CAM IPT focused on was the SETR process. The SETR process is an integral element of the acquisition process and life cycle management, as traditionally depicted in Figure 2. Technical reviews coincide with and support key acquisition milestone decisions and gate reviews in the acquisition process, and provide an independent assessment of emerging designs against plans, processes, standards and specifications, and key knowledge points in the development process. The SETR process is integral to naval systems engineering and is consistent with existing and emerging commercial standards.
Although the SETR process consists of several technical reviews, depicted as red triangles in Figure 2, the CAM IPT assessed I&I-related criteria and targeted three specific reviews: System Requirements Review (SRR); System Functional Review (SFR); and Preliminary Design Review (PDR). Each review is described below.
|
SRR
|
The objective of the SRR is to ensure that operational requirements have been successfully represented by system requirements; the system design has the potential of meeting these system requirements when acceptably balanced with cost, schedule, and technical risk; and the plans for moving ahead with system development are reasonable and of acceptable risk.
|
|
SFR
|
The objective of the SFR is to review and approve the system's technical description, including its system requirements and architecture, and to establish the system's functional base.
|
|
PDR
|
The primary objective of a system or subsystem PDR is to review and approve the system architecture as defined by the allocation of requirements to the subsystems and configuration items, thereby creating the allocated baseline.
|
Mission I&I SETR Event Tier Development
The CAM IPT started with a list of criteria that had been previously developed by an earlier SYSCOM I&I team. A tier structure was used to help analyze and determine I&I entry and exit criteria, leading to the traceability and evaluation of the system of systems throughout the life cycle. The SETR evaluation criteria was developed and documented for inclusion into the SETR handbook. The tier structure for organizing SETR criteria consisted of five tiers:
|
Tier 1
|
SETR event
|
|
Tier 2
|
Categories: mission engineering, systems engineering, test and evaluation, IA, open architecture
|
|
Tier 3
|
Subcategories of Tier 2, such as capability gaps analysis
|
|
Tier 4
|
Evaluation criteria
|
|
Tier 5
|
Detailed criteria designed to help evaluate Tier 4 criteria
|
The tier structure numbers are used to identify the categories and the actual I&I evaluation criteria.
2-Pass, 6-Gate Process
The second area the CAM IPT addressed was acquisition milestone management and milestone decision-making policies and processes. The Navy has mandated that all programs use the 2-pass, 6-gate process for program evaluations. The goal is to ensure programs are adequately assessed during the system engineering development process. As depicted in Table 1, the gate review process ensures alignment between service-generated capability requirements and acquisition, and improves senior leadership decision- making through better understanding of risks and costs throughout a program’s entire development.
|
FIRST PASS
|
SECOND GATE
|
|
OPNAV Chaired
|
ASN(RDA) Chaired
|
|
Gate 1
|
Initial Capabilities Document Approval |
Gate 4
|
System Design Specification Approval
|
|
Gate 2
|
AoA Approval |
Gate 5
|
Request for Proposal Release
|
|
Gate 3
|
Capabilities Development Document/
Concept of Operations Approval |
Gate 6
|
In-Progress Review Capabilities
Production Document Approval
|
|
|
Table 1. 2-Pass, 6-Gate
|
Specifically, the CAM IPT reviewed Gate 3 and Gate 6 templates for inclusion of I&I activities and actions. This included rewriting Gates 3 and 6 entry and exit criteria, based on the understanding of SoS engineering. For Gate 3, the CAM IPT recommended providing test and evaluation strategies for developing end-to-end test plans. Early operational assessments, prior to developmental testing, were also recommended to reduce integration risk and SoS testing. For Gate 6, the CAM IPT recommended the use of mission threads for improving system and platform operational effectiveness, including how mission threads can be used to assess DOTMLPF material solutions.
Incorporating these changes to Gates 3 and 6 will provide the rigor necessary to ensure the program under review is fully reporting I&I characteristics and is capable of transitioning to the next phase with minimal risk.
Probability of Program Success (PoPs)
The Program Health Assessment Guidance, PoPS, assesses the health of an acquisition decision and the likelihood of implementation and associated impacts to capabilities by measuring a program’s real ability to accomplish its objectives and effectively illustrate that status to decision makers. For Gates 3 and 6, the CAM IPT reviewed the associated PoPS criteria for inclusion of I&I acquisition requirements. When holistically implemented, PoPS produces an interactive dashboard (see Figure 3) that displays a program’s current “health status,” and enables leadership to identify issues, determine mitigation strategies, and allocate resources accordingly across the program.
The PoPS family tree is intended to provide a holistic look at a program and provide leadership an easily interpreted (red-yellow-green) consistent assessment of current program state. Of particular interest to the CAM IPT was the linkage, shown in Figure 4, among the SETR technical reviews, the acquisition 2-Pass, 6-Gate events, and the continual PoPS measures and program status. The proposed SETR I&I criteria for SRRs, SFRs, and PDRs supports the I&I requirements at acquisition Gates 3 and 6. This information is reported through PoPS, which provides a holistic response to realizing capability acquisition management for mission-level I&I.
If criteria questions reveal issues during any of the targeted reviews, the program manager or senior leader can assess the program’s health based on program requirements, resources, program planning and execution, and external influencers.
Recommended Policy Changes to Support I&I
The CAM IPT identified four specific policy-rich documents at various levels of the Navy’s acquisition and development organizational structure to achieve coverage of mission-level I&I requirements at organization depth:
- OPNAVINST 9070.2a, “Signature Control Policy for Ships and Crafts of the U.S. Navy”
- CFFCINST/CPFINST 4720.3b, “C5ISR Modernization Policy”
- NAVSEAINST 5000.9/MARCORSYSCOM Order 5400.5/SPAWARINST 5000.1/ NAVAIRINST 5000.24, Naval Systems Engineering Technical Review Handbook
- Technical Review Manual Version 2.0, PEO IWS and NAVSEA 05H
The recommended update to these instructions ensures that integration and interoperability are considered across Navy systems of systems, and provide evaluation criteria for SETRs and design reviews.
Summary
The goal of the CAM IPT was to address I&I issues and reduce the programmatic risk to enabling mission-level requirements. To do this, the CAM IPT developed new SETR criteria questions that addressed I&I across mission and platform systems of systems, proposed entry and exit criteria for the 2-Pass, 6-Gate review process, and recognized the value of having I&I issues visible in the PoPS dashboard. Codification of the new SETR criteria was accomplished by providing recommended additions to current policy documents.