The motion filed by Bergdahl's attorney Eugene Fidell with a military appeals court argued that General Mark Milley's nomination to serve as the Army's chief of staff could impact the case, especially with Milley needing Senate confirmation to become the service's top officer.
Bergdahl was released in May 2014 in a prisoner swap with the Taliban after five years in captivity.
Many Republican lawmakers condemned the deal the Obama administration reached that brought Bergdahl home, calling it irresponsible. Republicans control the Senate.
Milley is serving as the "convening authority" overseeing the case.
"A convening authority exercises quasi-judicial power," Fidell wrote in his motion filed with the Court of Appeals for the Armed Forces.
"Appellant (Bergdahl) has the right to demand that those powers not be exercised by an officer over whom a 'confirmation sword of Damocles' so plainly hangs," the motion said.
Bergdahl was charged in March with desertion and misbehavior before the enemy, and could be sentenced to life in prison if convicted of the most serious count.
A preliminary hearing is set for September at Fort Sam Houston in Texas.