Top left banner Top right banner
Bottom left banner Bottom right banner
Home : Home : Warfare Centers : NSWC Dahlgren : Resources : Leading Edge : I&I Leading Edge : Clawson
Warfare Capability Baseline:
Assessing Operational Gaps 
_______________________________________________________________

By David A. Clawson, Sara E. Wallace, Gregory D. Little, and Keith Wheeler

Mission engineering, including the desire to predict and assess the warfighting capability of the Navy, is nothing new. Analysts in the private and public sectors have supported warfighting capability assessment for years. Most studies have relied on subject-matter experts and the use of modeling and simulation (M&S) to assess system performance. While this approach has proven to be invaluable to the acquisition and engineering communities, experience has shown it does not necessarily capture integration and interoperability (I&I) issues that could impact the performance of complex systems. Manually assessing performance using technical expertise alone is not feasible for most cases and, although M&S allows analysts to model very complex system behavior, models may be wrong, may not accurately represent interfaces or important (possibly unknown) interactions between systems, and may not accurately reflect how the systems are used once deployed. Clearly, the identification of gaps in our warfighting capabilities requires that these and other issues to be considered in the system assessment process. This article provides an overview of the Warfare Capability Baseline (WCB) assessment process, which identifies current capability gaps using an approach primarily based on operational test data.

Background

Investigation into a possible fleet issue necessitated the assessment of current fleet capabilities grounded with data collected from fleet exercises and system operational and developmental tests. WCB began as a pilot study led by Commander, Operational Test and Evaluation Force (COMOPTEVFOR). Its mission is to provide senior Navy decision makers with rigorous, fact-based reports of current baseline systems-of-systems warfighting capabilities “that are not available anywhere else in the Navy.”1

Since the pilot study, the WCB assessment process has been executed three times (referred to as “increments”). Each successive increment has expanded the scope of the assessment to include more threats and more warfare areas. As of this writing, approximately 150 assessments have been performed across six warfare areas: Ballistic Missile Defense (BMD), Air Warfare (AW), Surface Warfare (SUW), Anti-Submarine Warfare (ASW), Mine Warfare (MIW) and Electronic Warfare (EW). The WCB assessment process is derived from the Naval Air Systems Command (NAVAIR)’s Horizontal Integration and Capability Assessment Process (HICAP). Some modification of the HICAP scoring criteria was necessary to address the expanded scope of the WCB assessment. Specifically, Threat Evaluation and Weapon Assignment is a critical, complex activity not explicitly identified in the HICAP functions.

Assessment Team

The WCB team comprises a diverse set of naval agencies including COMOPTEVFOR; NAVAIR; Naval Surface Warfare Center, Dahlgren Division (NSWCDD); Naval Surface Warfare Center, Panama City Division (NSWCPCD); Naval Undersea Warfare Center (NUWC); Naval Air Weapons Center (NAWS) China Lake; and the Space and Naval Warfare Command (SPAWAR). An organizational chart depicting the team hierarchy of the WCB assessment team is displayed in Figure 1.

COMOPTEVFOR leads the overall WCB assessment. The assessment team is divided into five groups: surface; mine; undersea; air; and command, control, and communication (C3), each led by a different warfare center.

The lead at each warfare center is referred to as a “Mission Engineer” (ME) and is responsible for working with MEs from other warfare centers to develop and review scoring criteria, provide the rationale behind the assessment, and provide support for the integrated assessment of all kill chains, including the creation of the final report. Each ME also forms a working group of “Principal Investigators” (PIs), who are responsible for scoring systems in their particular area of expertise. (e.g., the surface sensor PI will score the performance of surface-based sensors such as the SPY-1B). PIs also work with warfare center subject- matter experts and keep program offices informed as the assessment proceeds.

Each group is assigned a “Mission Integrator” (MI), who is an active military COMOPTEVFOR representative. MIs serve as liaisons between WCB technical personnel and the warfighter community. MIs’ roles include working with the fleet to develop tactical situations (TACSITs) used in the assessment, working with warfare centers to ensure current Navy doctrine and tactics are represented, and supporting the integrated assessment of kill chains.

Several other agencies played a role in the WCB. These agencies include Program Executive Offices that provided access to platform, sensor, weapon, and network test data; and Commander U.S. Fleet Forces Command (CFFC), who provided prioritized weapon target pairs for assessment as well as the mission context and criteria for each kill chain. The Warfare Centers of Excellence also provided inputs into developing TACSITs and provided feedback on WCB findings.

Assessment Process

The WCB assessment process consists of several steps:

  • Defining and prioritizing weapon-target pairs of interest
  • Developing kill chains and TACSITs
  • Developing scoring criteria
  • Scoring kill chains
  • Performing an integrated kill-chain assessment
  • Reporting assessment findings

A brief overview of each step is provided.

Defining and Prioritizing Weapon-Target Pairs

The WCB assessment process begins with a fleet-prioritized list of weapon-target pairs (WTPs) that identify a specific weapon for use against a specific target. CFFC requests each fleet to provide a prioritized list of WTPs that are of interest to them. The CFFC and COMOPTEVFOR use these submissions to determine the WTPs to examine in the next WCB increment. For each WTP, the WCB team attempts to answer the question: “Can this weapon be effectively employed to achieve the desired effect against this target under the circumstances described in this TACSIT?”

Developing Kill Chains and TACSITs

Scored kill chains are the basic tool of WCB assessments, are used to answer the above question, and are the foundation for WCB products. A kill chain consists of mission tasks or functions required to successfully employ a specific weapon against a specific threat and the platforms that could provide the required functionality (e.g., target detection could be done by an aircraft or a surface ship). In addition, the kill chain includes the major decision nodes (e.g., the decision to commit an aircraft to visually identify a tracked object) as well as the communication links required to transmit information between and within units. The specific mission tasks and C3 nodes represented in a kill chain are dependent on the mission area being examined and are subject to change from one increment to the next. In increment 3, the AW kill chains consisted of eight mission tasks plus three subtasks and six C3 nodes, as illustrated in Figure 2.

Given the number of combinations of mission tasks and platforms and/or performers that could carry out those tasks, there could be many “paths” through a kill chain. Many of these paths will have “broken” links, some will have “weak” but not broken links, and (hopefully) some will be comprised of all “strong” links. Since assessing all possible paths through a kill chain is not generally possible, WCB focuses on assessing the primary path, which represents the Navy’s preferred path, based on current doctrine, training, etc., and possibly one alternative path that may circumvent weak or broken links in the preferred path. In Figure 2, the primary path is indicated by the solid black line connecting the scoring nodes for Platform 1, and an alternative sensor path is indicated by the dashed black lines connecting the scoring nodes for Platform 2.

Kill chains are inherently tied to TACSITs, which are vetted through an annual TACSIT development effort with participation from fleet warfighters. These TACSITs set the baseline year and provide warfighting scenarios for kill chain assessments. The TACSIT contains several important factors that must be considered in the assessment, including the threat, the participating platforms, the operational environments, and expected methods of employment showing associated geometries and timelines throughout mission execution. In short, TACSITs provide the context within which kill chains will be scored.

Developing Scoring Criteria

Each kill chain function must be scored in the context of and constraints defined by its TACSIT. Scoring criteria define what the systems must do. In general, Green, Yellow, Red or White scores, as shown in Table 1, are applied for each scored platform as follows:

  • Green: The platform provides the full level of performance required by the TACSIT.
  • Yellow: The platform provides a partial or degraded level of performance in the constraints defined by the TACSIT. For instance, a Yellow score would be applied to performance that occurred between a desired and a minimum threshold, if both were defined. In some circumstances, the desired and minimum threshold are the same, and a Yellow score does not exist, i.e., performance is either Green or Red.
  • Red: The platform fails to provide the minimum level of performance required by the TACSIT.
  • White: Test data does not exist or is insufficient to accurately score system performance. The PI responsible for scoring a given system or function makes the decision as to whether there is sufficient data to score system performance. In some cases, accredited M&S is used to supplement test results. If the M&S supports the trend observed in the test data, then the function will be scored; otherwise, the White score remains in effect.
Critical
Measures
Notional Scoring
Criteria
Notional Score
and Rationale
 
Overall
Task
Score
 
 Launch_1  Is the correct weapon selected?  GREEN: No issues observed in
 testing
 YELLOW
 Launch_2  Is all prelaunch data provided to
 the weapon?
 YELLOW: Hardware
 interface issues prevented launch
 in x of y attempts
 Launch_3  Is the launcher able to fire all 
 requested rounds for the 
 engagement?
 GREEN: No issues observed in 
 testing

Table 1. Notional Critical Measures, Scores and Justifications for the Launch Subtask

       

Although, as explained above, the decision to apply a White score is left to the discretion of the responsible PI, the Green, Yellow and Red scores must be justified and traceable to specific scoring criteria developed for each mission area. For the AW mission area, each mission task is broken down into a set of critical measures. These critical measures are tied to factors impacting mission success in a given TACSIT. The ME and PIs develop the scoring criteria used to score each of a task’s critical measures. Table 1 provides a notional set of critical measures and scoring criteria for the Launch subtask.

Scoring the Kill Chain

After receiving access to the test data, PIs must sift through the data to determine which tests are applicable to the mission task they are scoring, given the conditions specified in the TACSIT. For example, a PI may reject a test because the target drone is not sufficiently representative of the threat. The PI then uses the applicable test data and the appropriate scoring criteria to score each of a task’s critical measures. The lowest score assigned to any of a task’s critical measures determines the task’s overall score for a given platform. Notional scores and rationale are provided in column 3 of Table 1.

Note: This data is for illustrative purposes only and is not consistent with the actual WCB assessment. The overall task scores (e.g., Green, Yellow, Red or White) are applied to the scoring nodes and data links shown in Figure 2 to provide a top-level summary of the kill chain’s health.

It is important to understand that each mission task in a kill chain is scored independently of the tasks that precede it. For example, the Launch task will be scored as if all tasks “upstream” were scored Green, so it is possible to have a Green Launch task even though the Track task is scored Red. This is done to avoid masking the overall strength or weakness of a “downstream” link, so that multiple issues affecting the successful completion of a kill chain can be identified.

Performing an Integrated Kill Chain Assessment

In this step of the assessment process, all of the individual platform and system scores and scoring rationale are discussed in a working group consisting of the entire WCB team. The national team critiques the scores and justifications for every critical measure in every kill chain to ensure they properly reflect the scenario laid out in the TACSIT and accurately reflect current capability in the TACSIT context. The weeklong national team meeting allows the WCB team to work through issues in real time and finalize all scoring with the full consensus of the team across all mission areas.

Reporting Assessment Findings

The primary purpose of a single scored kill chain is to highlight broken and weak links or failed interoperability and integration that would prevent success in delivering a particular weapon against a particular target with a specific mission employment in a given environment. WCB findings are the delivered product for highlighting such capability gaps in individual kill chains. As the number of scored kill chains grows in a particular mission area, including functional, degraded, and broken kill chains, a broader picture of warfighting capability becomes clear, and major warfighting capability gaps identified.

In WCB Increments 1 and 2, a formal report documented the major assumptions, scores, justifications and findings for every assessed kill chain. In Increment 3, annotated electronic presentations served as the final report. In all three increments, scoring and justifications were placed into the WCB database maintained by COMOPTEVFOR.

Interpreting Results

The current WCB assessment process is not perfect, and care must be taken when interpreting the results. In particular:

  • Because overall platform scores are based on a rollup of subsystem performance as well as tactics, doctrine, etc., a score of RED does not necessarily indicate the platform vice the doctrine or tactic is at fault. PIs will document the cause of the failure.
  • Each AW kill chain to be assessed considers only one weapon and one shooter. The cumulative effect of multiple shooters or additional weapons is not included in the assessment. One implication of this decision is that a “broken/failed” kill chain does not necessarily imply that the targeted platform is at risk.
  • Similarly, since the kill chain is assessed against a single-target raid, GREEN scores do not necessarily indicate the kill chain will perform well against a multi-target raid.
  • Finally, in order to score a complete kill chain, it is likely that data will be pieced together from separate test events that occurred under different conditions. Consequently, the availability, applicability, and consistency of test data must be considered during the scoring process, when interpreting the results, and when drawing any conclusions.

Conclusion

The WCB process provides a valid method to assess current warfare capability based on actual system performance data from live-fire tests. The integration of warfighters’ experience with analysts’ technical expertise ensures that tactical needs will be addressed at the technical level. Findings from previous increments have established areas of focus for the acquisition and science and technology communities that align with the fleet’s highest priorities and concerns.

 

Article Images



Figure 1. Warfare Capability Baseline Organization



Figure 2. Notional Unscored Kill Chain
       

Article Acknowledgements

The authors express their gratitude for the significant assistance provided by the WCB team, specifically: Charles Tatum, NSWCDD; CAPT Scott Guimond, COMOPTEVFOR; James Rogers, COMOPTEVFOR; and CDR Mark Carlton, COMOPTEVFOR. Their guidance and expertise greatly contributed to the development of this article.


Article Endnotes

  1. It is important to note the weapons and threats identified in the WTPs are currently deployed systems. WCB does not attempt to assess capability of non-deployed weapon systems or of current weapon systems against future threats.


  2. “Platform” is used generically to refer to the entity performing the kill chain function. This entity may be an aircraft, ship, vehicle, person or weapon, and includes all of the sensors, systems, equipment, and software used by that entity to carry out the given kill chain function.

         

Article References

  1. “Warfare Centers of Excellence,” Rhumb Line, 28 May 2009.