Mission Level Test and Experimentation:
Drive Mission Level Performance During Early Development
________________________________________________________
By Neil Baron, Darren Barnes, and Dr. James D. Moreland, Jr.
Since the first test shot was fired over the Potomac
River Test Range (PRTR) in 1918, the Naval Surface Warfare
Center, Dahlgren Division (NSWCDD), has answered
the nation’s call to support the warfighter. In today’s
environment, naval systems are more complex than
ever with the need to integrate many systems in order
to provide full mission capability. How these systems
work and communicate with each other is critical to
understanding warfare systems performance. Instead
of waiting for shipboard integration and testing to
assess integrated capability, NSWCDD has developed
the USS Dahlgren capability.
USS Dahlgren connects the PRTR, which provides
real target data in a littoral environment to
our plan-detect-control-engage-assess labs and systems
for end-to-end testing at NSWCDD. Using USS
Dahlgren, scientists and engineers are able to plan,
detect, identify, track, engage and assess sensorweapons
capabilities in a real world environment prior
to shipboard integration and delivery to the Fleet. USS
Dahlgren also allows us to evaluate new capabilities
and assess the benefit to the warfighting system prior
to full scale acquisition and development.
Background
Mission Level Test and Experimentation (MLT&E)
is a system-of-systems (SoS) approach for validating
the development of individual systems by evaluating
these systems in a representative operational environment
with the other systems that they must operate
with to achieve an integrated warfighting capability.
The intent of this approach is to gain technical insights
on integration and interoperability (I&I) challenges
as well as determine the derived requirements for
individual systems based on mission success criteria
during early development. Individual systems do not
produce warfighting capabilities unto themselves but
rely on the interaction of multiple systems to execute
an effects/kill chain effectively and safely. By addressing
I&I issues earlier in the acquisition cycle, we have
a better chance at reducing expensive rework and
expediting fielding capabilities to the warfighter without
delay. MLT&E plays a critical role in determining
the right requirements, assessing I&I wholeness for
integrated capabilities, and solving issues early, on
the left side of the Systems Engineering “V” model, to
reduce cost, thus making capabilities more affordable.
The technical insights gained are also used to develop
the Mission Technical Baseline (MTB) and Integrated
Capability Technical Baseline (ICTB) architecture
products, which serve as the technical reference documents
to drive acquisition development of systems.
In today’s environment, the majority of test and
experimentation activities occur for single systems
to exercise that system’s ability to meet requirements
irrespective of the mission success criteria. This focus
on single system testing has driven us down a path of
sub-optimizing individual systems and losing focus
on the operational context in which the systems will
operate when transitioned to the Fleet. As individual
systems are matured independently, the integration
of these systems within the required SoS has fallen
on the Fleet at delivery. In addition, the assessment
of Fleet-prioritized effects/kill chains indicates a
compelling need to address I&I problems from an
end-to-end mission thread perspective to consider
all interfaces and relationships across many systems.
Figure 1 illustrates a notional effects/kill chain with
linkages to architectural products used in the acquisition
community for developmental purposes.
NSWCDD leadership developed a forum, Integrated
Lab Council (ILC), to provide technical
guidance across all departments in order to link
the right labs together to make the USS Dahlgren
concept a reality. The ILC provided a cross-departmental
vision and a strategic plan for integrating
NSWCDD Research, Development, Test and Evaluation
(RDT&E) resources into a cohesive, net-centric
engineering environment. A key aspect of this
capability includes connectivity to the PRTR assets
to leverage and incorporate real-time, live data from
outdoor experiments. This end-to-end integrated
capability will foster collaborative RDT&E demonstrations,
experiments, integration events, and engineering
development tests to accelerate and improve
NSWCDD core technical capabilities and products
for warfighter use.
USS Dahlgren Definition
In 2011, the Naval Surface Warfare Center, Dahlgren
Division (NSWCDD), embarked on establishing
the USS Dahlgren construct. In response to the
driving need to evaluate the Naval fire-control-loop
through RDT&E that spans all aspects of the kill
chain in a land-based test environment, NSWCDD
has established a virtual ship, the USS Dahlgren. As
our naval systems continue to increase in complexity
and the fire-control-loop continues to expand within
and between naval platforms, the need to integrate
and conduct MLT&E in a laboratory environment is
fast becoming a necessity. Additionally, an emphasis
is shifting Test and Evaluation (T&E) to the left in
the development cycle to identify design flaws and
preferred design alternatives prior to full-scale development
or shipboard integration. This shift results
in cost effectively meeting performance goals and
eliminating corrections after fielding. Although not
an uncommon approach in some development cycles,
it has proven to be a less common approach when
looking across the kill chains and mission threads
of larger scale distributed weapon system platforms.
The USS Dahlgren envelopes not just one ship combat
system variant but all ship combat/weapon system
variants that can be assembled utilizing the current
distributed laboratory infrastructure. The entire firecontrol-
loop must be developmentally exercised in the
distributed laboratory environment and not for the
first time onboard the intended platform(s). USS Dahlgren
is the integration of NSWCDD RDT&E resources
into a cohesive, real-time, deterministic, net-centric
engineering environment, capable of replicating many
of the fire-control (FC) systems including, sensor,
command and control (C2), and weapons systems
found onboard naval platforms. The unique aspect of
USS Dahlgren is the ability to integrate and test new
systems or their modification to existing systems early
in the development cycle interoperating side-by-side
with their already deployed counterparts. USS Dahlgren
provides the infrastructure for early verification
that fire-control systems perform in standalone and
distributed environments, supporting identification
of integration issues such as timing, data latency,
and throughput early in the development cycle, in a
high-fidelity environment. This environment provides
for continuous testing across the acquisition life cycle,
off-site connectivity to investigate ship, battle force,
and joint force integration via Joint Mission Environment
Test Capability (JMETC); and the ability
to replicate the fleet environment to address issues
observed at sea.
Recent focus has been on the interconnectivity
of the existing labs as a means to exercise an overall
federated warfighting capability capable of
supporting multiple mission areas of interest. The USS
Dahlgren capability spans the six technical departments
at NSWCDD as well as corporate networks,
and includes remote connections via the JMETC
to include other warfare/system center laboratories
as well as our sister service laboratories for joint
experimentation and testing.
USS Dahlgren Components
The USS Dahlgren is composed of the many laboratories
that house the developmental and in-service
systems utilized in the effects/kill chain as found
on deployed naval platforms (see Figure 2). Integration
of the laboratories is facilitated through local
and global secure network connectivity, modeling
and simulation environments to generate and exercise
the synchronized warfighting scenarios across
the disparate locations, and representative architectural
lay-downs of the effects/kill chain within (and
between where necessary) platforms that house the
warfighting capability.
Results from exercising the USS Dahlgren on a
specific mission thread scenario can then be evaluated
with other model-driven, experimental, and at-sea
test data to evaluate the overall performance of the
integrated mission capability. In this way, warfighter
capability can be assessed earlier in the development
cycle, and warfighter expectations can be demonstrated
with high precision while still in development
(where the cost of modification is minimized) and
before taking final products to sea.
Mission Level T&E
Utilizing The USS Dahlgren
There is an urgent need in the naval warfare centers
and acquisition communities to develop techniques
and procedures to map mission operational demands
with system and SoS hardware and software capabilities.
This has become increasingly relevant as emerging
SoS are required to create, consume, and fuse a
vast amount of data with the intent of informing a
future decision maker about an operational scenario
that has not been fully defined. The MLT&E environment
through USS Dahlgren is providing that
mechanism to postulate how to map an operational
architecture to the system architectures to investigate
I&I problems.
MLT&E provides the scientists/engineers a venue
to obtain technical insights on I&I issues while
executing representative operational mission threads
in a system-of-systems environment. This critical
information is used to drive system requirements,
specifications, and architectures in the acquisition
community which thereby increases confidence in
early development decisions. In today’s environment,
systems are tested individually to determine Technology
Readiness Levels (TRLs) that focus on the
maturity of individual systems. This level of evaluation
is necessary but not sufficient to field integrated
warfighting capabilities involving dependencies across
many systems. USS Dahlgren provides the ability
to link all effects/kill chain systems to evaluate and
demonstrate performance across an end-to-end mission
thread.
The operational test community evolved the test
environment from an individual system focus to
mission-based test. To prepare the
newly developed systems for operational
tests in this mission-focused
domain, the systems need to be thoroughly
tested while being developed
under those complex mission conditions.
This requires good use of
experimentation resources on the left
side of the Systems Engineering “V”
model. Rather than waiting until late
in the acquisition life cycle to perform
operational test for the integration
of SoS, many I&I issues can be eliminated
during system development
to make mission-based test a much
easier milestone for the transition of
capabilities to the Fleet.
Lastly, the operational community
can leverage the USS Dahlgren environment to
investigate the performance of an SoS under different
situations for the development of Tactics, Techniques,
and Procedures (TTPs) as well as doctrine. These
non-material drivers provide the details on how to
operate systems within an operational context while
working across legacy and new systems. At the same
time, the operational warfighters get a chance to
train on new systems under development from the
beginning so that human systems integration can
be handled as human systems engineering thereby
ensuring that the operators’ needs are part of the
overall design rather than an added piece.
USS Dahlgren Applications
Utilizing a crawl, walk, run strategy, NSWCDD
began the USS Dahlgren initiative in September of
2012 with an initial experiment utilizing the secure
connection of an Aegis Combat System baseline
located in one building with the Mk 160 Gun Fire
Control System located in another building and the
Mk 45 Mod 4 Naval gun on the gun line. The combat
system generated a firing order, which was passed to
the fire-control system, and the gun was fired at a test
target at a range of over 8,000 yards on the Potomac
River Test Range. As seen in Figure 3, this test was a
success yielding multiple hits on the target. This first
experiment exercised command and control, weapons
control, gun control, river range control and multilevel
security through a Cross Domain Solution (CDS)
between the laboratories.
In 2013, the USS Dahlgren experimentation campaign
continued to advance with the inclusion of an
Unmanned Air Vehicle (UAV) in the mission thread
providing target information messages and video
utilizing cursor-on-target technology. The information
passed a target object to the Naval Fire Control
System in one laboratory, which then, through voice
command, sent a firing order to the electromagnetic
Railgun in another facility to engage the target. The
UAV, still on station, was then utilized to stream video
for battle damage assessment to a third assessment
laboratory. A high-level operational view of this land
attack mission thread is shown in Figure 4.
Currently, NSWCDD, sponsored by the Office of
Naval Research, is investigating the Railgun as one
of the next generation electric weapon system development
programs. The Railgun uses electromagnetic
power to achieve kinetic kill capabilities with a significant
increase in speed and range and reduced
cost per shot. The U.S. Navy has committed to testing
the Railgun aboard a Joint Homeland Security
Vessel in 2016; however, before testing commences,
a significant amount of RDT&E will be performed at
NSWCDD, to include testing of the Railgun on the
PRTR. Advancement of the USS Dahlgren laboratory
integration construct will include early development
of systems such as the Railgun to assure other critical
system elements of the mission thread are integrated.
In addition to the Railgun, the technological
advancement of lasers as another electric weapon
system capability is also continuing. The Laser
Weapon System Quick Reaction Capability (LaWS
QRC) was tested onboard the USS Ponce (AFSB (I)
15) in late 2014. This testing serves as the first of what
is likely to be the eventual wide spread introduction
of a High Energy Laser system into the U.S. Navy.
Combat System integration of the laser capabilities
into warfighting threads will be key in developing
and fielding a true directed energy capability at
sea. In addition to tackling laser integration issues
during the early phases of development through
the USS Dahlgren capability, other aspects of the
effects/kill chain will be exercised. A laser weapon
system can be utilized as both a weapon and as a very
high quality sensor for the ship thus able to satisfy
multiple elements of many mission threads. LaWS
and its follow-on laser weapon system variants are
being designed for use against asymmetric threats
to include small boats and UAVs, both of which
have been tested in the PRTR environment with
the USS Dahlgren.
Railgun and LaWS serve as two prime examples
of systems early in their development cycle that will
benefit from being exercised via MLT&E. Mission
testing via the USS Dahlgren does not simply focus
on combat system integration, but fully exercises the
systems end-to-end capability where other non-material
aspects of overall performance can be investigated.
Non-material aspects of tactics, training and
human performance can also be evaluated in the USS
Dahlgren capability.
Challenges and Way Ahead
Under the working capital fund model, Warfare
Centers receive and are authorized to execute specific
task orders for particular sponsors. The resources
required to modify/build infrastructure and laboratories
to perform MLT&E are not currently considered
a necessity in task statements to get the job done. This
is amplified by a fiscally constrained environment
where infrastructure investments for extended capabilities
are taking a back seat to satisfying immediate
tactical needs. In addition, it is extremely difficult to
coordinate schedules across multiple labs which all
have their individual demands.
It is also a tremendous challenge to fully replicate a
shipboard environment at sea in a laboratory environment.
This operational context can only be simulated
with the major focus of the MLT&E environment
devoted to the technical execution of systems across
an end-to-end mission thread. The involvement of
warfighters in this environment provides a great deal
of operational insight while they receive beneficial
training on the new systems under development.
NSWCDD continues to develop the infrastructure
architecture products to accurately represent
the physical laydown of all assets. This allows leadership
to develop a prioritized plan on infrastructure
upgrades to make the best decisions on impacting
operational developments with the largest return on
investment from a military worth and affordability
perspective. This includes the necessary trenching
for logical wiring and physical connections required
across individual labs to represent the functional
areas of effects/kill chains. In addition, NSWCDD is
defining the critical mission threads through effects/
kill chain products to provide the logical sequencing
and connection points of individual systems for endto-
end mission thread execution.
Conclusion
MLT&E in a laboratory environment is fast becoming
a necessity, and even more so during the early
phases of development to identify potential I&I issues
as early as possible. The ability to resolve these issues
early in the Systems Engineering acquisition cycle provides
a better chance of getting the right requirements
under affordable conditions resulting in integrated
warfighting capabilities.