It’s Time to Stop Reinventing the Software Factory Wheel
There are so many software factories being built and operated across the U.S. Department of Defense (DoD) and its ecosystem of industry partners that it would be nearly impossible to count them all. If pressed, I would say that there are anywhere from 30 to 50 different entities across the services that are considered “official” software factories.
This count, however, misses the large number of software development environments being built for individual programs, individual Program Executive Offices (PEOs), warfare centers, and contractor facilities. There are likely hundreds of these shadow software factories that are operational or under development within the DoD today.
It’s easy to understand why this is happening. The entirety of the DoD is becoming increasingly software-enabled. Software enablement allows programs to innovate quickly and push impactful new capabilities to the warfighter at the pace of technological advancement. This pace is further enabled by the evolution of the modern software factory.
We’ve recently seen contractors awarded $90 million contracts to build software factories that are a shell of what already exists in other DoD organizations. Instead of spending an additional $90 million, these program offices could simply use an existing DevSecOps baseline, such as Platform One or Black Pearl to spin up their software factory.
The tools and applications provided by modern software factories are now table stakes for how development teams work today. They enable easier collaboration – even among remote workers and distributed workforces. They enable automation to accelerate development while also integrating testing and security checks to help expedite the adoption of the created applications.
Software factories are key enablers of our digital warfighting future – which is why they’re multiplying rapidly across the DoD. This probably doesn’t sound like much of a problem. But, just like having too much ice cream, pizza, or too many delicious IPAs – too much of a good thing can actually hurt you.
It All Adds Up
If you look across these hundreds of disparate software environments, you’ll see some unique attributes. This is understandable – the tools and processes a team requires are dependent on the technology stack they are building with, the skills and culture of the team, the organization structure, and the mission they support. However, I would argue that upwards of 80 percent of the tools and infrastructure are the same across all software factories.
Think of a software factory like an automobile assembly line. Different assembly plants for different companies will have some different equipment, but they will probably function in a very similar way. Similar building infrastructure, power, robotic arms, welders, automated paint booths, etc. will be found across most companies.
Automakers might build specialized equipment for their particular needs, but they will focus their innovation on the areas that give them specific competitive advantages. Auto manufacturers could not be competitive if they invented new concrete foundations, designed electrical infrastructure, or built robotic control systems from scratch for each new vehicle.
This is where the problem arises for the DoD, which is effectively reinventing the software factory wheel, from scratch, over and over. Not only is this expensive, but it’s inefficient. It is slowing programs down by requiring each program to begin by developing a basic software factory infrastructure that has already been implemented many times across the DoD. This also inhibits the program’s ability to invest in the specialized tooling that would make the software factory uniquely suited to its mission objectives.
Is Consolidation the Antidote?
The proliferation of software factories has not gone unnoticed, and has led some DoD leadership to ask, “How many software factories do we really need?” The knee-jerk reaction may be to consolidate software factories, with an objective to reduce them down to some magic number. However, I would argue that would be just as ineffective and inefficient for the military as continuing to spin up new software factories from scratch.
Software factories are key enablers of our digital warfighting future – which is why they’re multiplying rapidly across the DoD. This probably doesn’t sound like much of a problem. But, just like having too much ice cream, pizza, or too many delicious IPAs – too much of a good thing can actually hurt you.
The specific tools and processes required to build a warfighter payroll system are different from those required to build a missile defense system. The bottom 80 percent of the tools and processes required are often the same, but the top 20 percent can be very, very different.
Shoehorning programs into a small number of “standardized” software factories will all but ensure development friction where teams do not have the specialized tools they need. The larger the software factory, and the more diverse the programs within it, the more bureaucratic the software factory will become. The software factory will have too many stakeholders to move quickly. This will slow the pace of innovation and ultimately reduce the delivery of capabilities to the warfighter.
Ultimately, the DoD must balance two opposing pressures. The first is a desire to reap economies of scale cost reduction. Building a software factory for each program does not allow the program to benefit from economies of scale that come with bulk licenses, shared resources, pooled labor, etc. This would tend to lead towards consolidation. The second pressure is the need for agility and specialization. Smaller, program-specific software factories will always be better, faster, and more capable of directly addressing the program’s needs.
The sweet spot seems to be identifying moderately sized communities of interest with similar software development and mission requirements. For example, the Navy’s surface combat system community within PEO IWS is composed of a large number of programs, but they share leadership, organizational policies, development culture, technology stacks, and other characteristics. Many of the programs also require similar specialized tools like RADAR simulators, Link-16 emulators, and others. Building software factories along these lines can help deliver economies of scale, while minimizing the shoehorning and bureaucratic bloat.
However, while correctly identifying communities of interest can help balance the need for economies of scale and specialization. It is still wildly inefficient from a financial and time perspective if these software factories are built from scratch. If leadership keeps getting $50M+ bills for each new software factory, it will inevitably still lead to “enterprise” consolidation, which, as mentioned, will stifle innovation.
Software Factory Baselines are Over Built from Scratch
As I discussed, 20 percent to 40 percent of a software factory may be different from program to program and project to project. However, the vast majority are often comprised of the same tools and applications. Instead of building that all over again from scratch, the DoD should establish baselines that can be used across all programs as the foundation of future software factories.
I want to be completely clear: this doesn’t come without cost.
The different program offices and organizations must still pay to use that baseline. However, they won’t incur the massive cost of engineering and building the software factory from scratch—simply the licensing fees to use it. This is an order of magnitude less expensive and significantly faster – meaning military programs can be executed more rapidly and at a lower cost.
The specific tools and processes required to build a warfighter payroll system are different from those required to build a missile defense system. The bottom 80 percent of the tools and processes required are often the same, but the top 20 percent can be very, very different.
We’ve recently seen contractors awarded $90 million contracts to build software factories that are a shell of what already exists in other DoD organizations. Instead of spending an additional $90 million, these program offices could simply use an existing DevSecOps baseline, such as Platform One or Black Pearl to spin up their software factory.
Instead of $90M, that program could have been up and running for $2-5M. Instead of spending $90M on the basics, they could have arrived at a more capable solution for $5M, set aside $5M for licensing, used another $5M to customize with specialized tools, and had $75M left over to invest in mission capabilities.
When the U.S. Navy needed to develop a software factory for the Aegis Combat System, it used Black Pearl as the baseline. The same is true for the Rapid Integration Autonomy Lab (RAIL). This is the model that the entire DoD should embrace for its future software factories.
Too much of a good thing can be bad. Software factories are an essential part of modern military application and software development, but that doesn’t mean the DoD needs to keep creating new ones from scratch for every program and project they embark on. By establishing communities of interest, the DoD can limit the amount of different software factories they develop. By establishing and leveraging baselines for software factories, they can move faster and cut costs by not reinventing the wheel over and over.
The author, Michael MacFadden, is the Chief Technology Officer at Sigma Defense Systems.
