I believe the comet swarm theory also rested on this being a fairly short as well as really massive and rare one-time event, which makes it rather unlikely (though obviously not impossible) that we just happen to be looking just as it happens.

It also doesn't explain the long-term dimming of the star. The alien megastructure kinda does. It's not a perfect explanation either, but from what I understand, slightly more convincing than the giant comet swarm.