This is a very complex matter, and the noisiest critics of the exchange fail to consider the important fact that at the cessation of the war, hostilities, and active military presence in Afghanistan, the US is hard pressed to establish a basis for continuing to hold the five Taliban members. While the Bush administration purposely avoided designating them as prisoners of war in order to subvert the protections guaranteed by the Geneva Convention and international law, it is important to keep in mind that the prisoners have been held since 2002 without any criminal charges.
In order to justify holding these prisoners indefinitely after the war, we have to charge, try and sentence them. There are tenuous arguments that can be made, however, that these prisoners defy a classification envisioned by the Convention, but it will almost universally be viewed around the world that the Geneva Convention or international law would not endorse a situation where captured combatants during war, who are not charged with specific crimes, remain detained at the cessation of the conflict.
Thus, if it were important to keep the Taliban 5 detained perpetually, they needed to have been charged, and because after twelve years we could not find a basis, on which to charge them criminally, they would be released. If they are guilty of crimes during their time as provincial governors (as reports indicate), that would be for the Afghan government to consider as the US has no jurisdiction over that.
As far as Bergdahl being a traitor, deserter, etc., that may be, and all the more reason to secure him from the Taliban and allow the military tribunals to examine the facts and pass judgment. This is not a valid reason to allow an American POW to remain imprisoned. You can bet that if he died as a prisoner, the same politicians, who are vilifying him and his family, would be accusing the president of his death.