When engineers conduct experiments, the data from the work need to be stored somewhere for future reference. Using hard drives and other forms of removable memory is a popular method because it is secure, but this might not always be convenient when another person needs to recall that work. Engineers at Naval Surface Warfare Center, Carderock Division have run into the issue before and want to put an end to it once and for all.
Bradley Campbell of Carderock’s Simulations and Analysis Branch and Eric Giesberg from the Full-Scale Trials Branch used Carderock’s Technical Director’s Innovation Challenge as an opportunity to present an idea that would make data that are often hard to find accessible to all of their fellow engineers.
Titled “Data Curation: Enabling Access to T&E Data,” Campbell and Giesberg foresee a day when a person in need of information from another experiment can simply go into a secure database and pull the necessary data from it.
“We currently don’t have a good way of storing that data, accessing it and making it usable to our engineers and scientists,” said Campbell, who has been at Carderock. Their end goal is to bring consistency to their data storage dilemma to the command.
“It’s a complicated problem, especially if you start thinking about making something flexible enough for even other groups to use,” Giesberg said.
According to Giesberg, they made an attempt a couple years ago, but did not find legitimate solutions at the time. However, the technological advancements through the years made now a good time for the duo to reengage with the problem and suggest a usable product.
“New technologies have helped enable a lot of things, that’s why we came back and proposed it as a TDIC,” he said.
Both Campbell and Giesberg understand that many others, like them, aren’t software- and programming-focused individuals. As a remedy, they want their database to be simplified enough on the front end that its users will not need to know the metadata behind what they are searching for.
Their first hurdle was fully grasping the programming aspect of the project. “We’re both trained as engineers, not software people, so that was our first chance of working together on something like that,” said Giesberg, who has been onboard Carderock for seven years. From there, they had to find the type of database that would best work for their situation. Although there are plenty of ideas to draw from, Campbell and Giesberg needed to be assured that they would have a level of flexibility that has become common in more recent database concepts.
“The standard go-to databases in the past have been called SQL (structured query language) databases, which are rows of columns,” Campbell said. “You have tables, which has rigid structures of columns and you fill in those, and that’s it.” Instead, they opted to look into NoSQL databases, a newer technology that gave them the latitude to upload data in a nontraditional format and organize it within the program.
As part of their research, Campbell and Giesberg attended the 2018 Scientific Python Conference in Austin, Texas. At the conference, they were exposed to Kitware, a New York-based technology company working on a project called Girder that has many parallels to the concept that the two of them were looking into.
“We ended up meeting with a few of the Kitware employees down at their Virginia office and decided to take a bigger look into Girder,” Campbell said. They presented pieces of what Girder could do, showing off a product that could be likened to a share drive with options for administrators to control the information put into the system all the way down to a folder’s metadata. With it being an open-source platform, Giesberg said working with Girder created a secondary challenge for the pair on how to properly use open-source for the information they deal with. Girder only covers the back end of their project, though, and the two are still looking for a suitable way to provide the front-end solution for regular users who simply need to access or upload data.
For the time being, Campbell and Giesberg have taken to consulting with others to make advancements or contribute to other similar ideas, such as working with the Naval Surface Warfare Center, Corona Division’s cross-Warfare Centers Naval Innovative Science and Engineering (NISE) 219 effort to catalog sets of data, where Carderock is leading the infrastructure part of the overall effort.
Campbell and Giesberg hope that the steps taken now can eventually turn into a division-wide change on how data curation is viewed and that one day, they will have a way for their counterparts to securely and efficiently share data.