A massive, nearly five-hour shutdown of the US aviation system that disrupted more than 20,000 flights may have been the fault of a single router installed by government contractor L3Harris.
The incident is the latest painful example of the ageing US air traffic control telecom system and the struggles to replace systems while keeping the old communications working.
In recent years, the Federal Aviation Administration's ageing systems have failed after errant fibre cuts, accidental file deletions, and a strong chemical smell caused by a melted component. The system will fail again until it is properly upgraded, government officials said.
"This was an FAA system that didn't meet the moment," US transportation secretary Sean Duffy said on Sunday AEDT.
Duffy said on Friday L3Harris was conducting routine maintenance to switch telecommunication lines from old copper wire to fibre at at least one remote location.
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
Officials told Reuters news agency there may have been problems with routers installed in both Arkansas and Texas.
"We're working to determine whether we had incompetence, meaning was it user error or installer error from L3Harris, or did they install a faulty router? We haven't determined what the fault was," Duffy said.
L3Harris said it took the issue seriously and was "supporting the FAA in its investigation to understand the conditions that led to the 13 November ground stop of our nation's air travel".
L3 noted it has completed more than 70 per cent of the planned airspace communications conversion from legacy copper wires to a fibre-optic system.
The FAA spends $US1 billion ($1.5 billion) annually to maintain its old copper wire system and is racing to replace it with more modern fibre-based communications.
"We need L3Harris or whoever provided that router to us to do a better job," Duffy said, noting that "virtually every single institution in America has gone from copper to fibre, from analog to digital".
The issue began overloading communications at 17 of the FAA's 22 en route centres across the country, which prompted the FAA to issue a nationwide ground stop because it could not identify a root cause.
Duffy said the new routers started flooding other routers with over 1,000 connection attempts every four to five seconds, which sparked communications issues with planes and radar at FAA facilities.
Bedford said the FAA had to take the action when it lost "the ability for those routers to communicate data". The FAA is still trying to understand the root cause of what went wrong.
The FAA said communications did not go down but led to instability in data at the centres. It did not lead to centres going dark. The routers being replaced are about 25 years old, Duffy said, and were supposed to be replaced in 2006.
Small mishaps leading to catastrophic consequences are not new to the US air traffic control network. A 2023 FAA nationwide ground stop that lasted 90 minutes and disrupted 11,000 flights was caused by a government contractor unintentionally deleting files.
That led to the first nationwide ground stop since the September 11 attacks.
Friday's outage hit as a surge in travellers began weekend travel.
The FAA got $US12.5 billion ($18.5 billion) to upgrade its badly ageing air traffic control and telecom system from the US Congress, and Duffy has asked for $US50 billion ($74 billion) more for air traffic control and airports.
