It’s less the fighters themselves but what they represent

The empire was equipped to fight another CIS not the Rebellion and when the Rebels showed up took too long to turn around while also wasting money on projects that consolidate power into singular large warships when the threat they faced relied on many nimble units to disrupt and decay imperial rule

It isn’t the weakness to starfighters but rather the fact that 90% of the time there isn’t even a ship positioned to properly stop the fighter that was the problem