Unions don't help, but the problem is even deeper than that. Non-union government employees aren't wildly better.
It starts with the hiring process, where merit gets pushed aside easily. The people chosen for interviews and for referral to the hiring manager aren't the most qualified. They're the ones who connected all the dots and used the right buzzwords in their resume and cover letter, even if it's all a bunch of crap. You can be the most qualified applicant imaginable, but if you don't use the right verbiage in your paperwork, the hiring manager will never even know you exist.
Next, even among those who are interviewed, there is a lot of preferential treatment for certain groups - certain minorities, veterans, etc. Again, merit gets deprioritized.
Of course, once on the job, there just isn't a lot of reason to really put out big effort. Your pay is set through one of the pay scales, which are largely based on seniority and credentials. There are performance bonuses but they aren't big money. The worst employee and the best make very close to the same money, and of course, the best employees aren't liked by the mediocre and crappy ones. There's pressure not to show off.
If an employee isn't doing anything criminal but just sucks at his job, you can't just ditch him. You have to document his crappiness over a long period of time and give him tons of chances to improve. And of course, he can appeal his termination and potentially sue. Bottom line is that it's expensive and an enormous pain in the a$$ to terminate a federal employee for cause. If you're a supervisor, the employee needs to be a total $hitshow and an embarrassment to justify the time, cost, and hassle.
All those factors combine to make for a culture of mediocrity and indifference.