RESTON, Va. — The U.S. satellite sector must embrace modern software design practices like agile and DevSecOps if America’s space efforts are to remain ahead of foreign competitors like China, Nicolas Chaillan, the recently departed chief software officer of the U.S. Air Force, told CyberSatGov this week.
He said in his Thursday lunchtime keynote that many of the traditional national security satellite providers are stuck in a Department of Defense (DoD) ecosystem that is burdened by enormous technical debt, and unable to move “at the speed of relevance.”
One consequence of that kind of technical debt, he said, was the decade-long reliance of the U.S. on Russian Soyuz rockets, launched from the former Soviet Cosmodrome in Baikonur, Kazakhstan, to get personnel to and from the International Space Station (ISS). Once the Space Shuttle program was shuttered in 2011, the U.S. had no national capacity to reach the ISS, until
SpaceX developed the Dragon and successfully launched it earlier this year.
“I would argue it’s a little bit of a disgrace that we had to wait for SpaceX to bring a capability to allow us to send Americans back to the ISS and not depend on the Russians,” he said.
The capability gap is worse with the Chinese, he warned. “I would argue that they are ahead of us, not behind us, on both AI and ML [Artificial Intelligence/Machine Learning]. Once you lead in AI and ML you can end up learning things and enabling things that the human brain cannot even comprehend. By definition, we’re going to end up falling even more behind.”
To underline his point, he compared SpaceX with the Department of Defense. SpaceX has 200 full stack engineers, he said, while DoD has 4,000 developers working on the F-35. SpaceX operates nine platforms, including the various Falcon versions and the drone ship that allows the returning reusable rockets to land at sea.
As much as 80% of the source code used in each of those platforms is shared among them all, Chaillan said, “Compare that to … 4% of F-22 [code] is shared with the F-35.”
SpaceX used a modular approach to software that Chaillan compared to Lego blocks.
“They can swap Lego blocks in or out. They can build the entire stack [thousands of] times a day; they can test it automatically to see if their new features are breaking things; look at the software [to check it’s] behaving the way it should be behaving.”
Integration is often a major challenge for traditional satellite manufacturers — checking that new or updated software runs smoothly on the esoteric hardware in which it is often embedded. Because SpaceX has chosen to use commodity hardware, instead, the company is able to test much more easily, Chaillan said.
“They can actually have an entire hardware-in-the loop process on premise where the software that they build that day is tested on the hardware in real time — three times a day. They are able now to deploy that software in production. They can upgrade the software the day before the launch.”
The key decision SpaceX made, he said, was to build its technology stack using commodity central processing units [CPUs]. “None of this [specialized] space hardware, we’re not going to use any secret sauce, we’re just going to use commercial CPUs. And we’re going to go open source with Linux. That enables [SpaceX] to use effectively all of the tools that companies use today to build software. They’re not stuck in this very niche market space where they have to deal with a very small number of players, with vendor lock-in left and right and IP limitations.”
Chaillan compared that to the F-35 program, “which effectively takes a year to seven years to upgrade software, depending on complexity, and with no ability to do testing end-to-end [before deployment on the actual hardware.]”
By understanding the difference “we start to realize the technology debt,” Chaillan said.
He added later that things aren’t that much better among traditional defense contractors. “The [defense industrial base] is mostly comprised of former government people that don’t really know the pace of industry. They’re often stuck in a bubble that is the DoD bubble, where effectively there’s no competition and we don’t need to bring talent in from the outside.”
Chaillan, a French-born tech entrepreneur brought in to help shake up the creaky Air Force acquisition machinery for software, quit very publicly last month in a “drop-the-mike” posting on LinkedIn, in which he said the Air Force was not “walking the walk” on IT modernization.
When he joined the Department of the Air Force, Chaillan said he was “shocked” by the way things were run. “If I send some of these [Air Force acquisition] leaders for two weeks at Google or SpaceX, their head would explode,” he said.
In fact, he noted, one problem with programs that allowed DoD leaders to work temporarily in the private sector was that “when we send people to these companies, they ended up seeing the light and not coming back.”