You're right that the Electoral College only determines the presidency, and that gerrymandering mainly has an impact on the U.S. House and state legislatures.
How do I explain the crappy politicians who are elected statewide? Again, the "infotainment" culture the media has created. The presence of dark money and dirty money (candidates aren't allowed to formally coordinate with PACs, but those PACs can still spend unlimited amounts boosting candidates or attacking candidates without their consent).
And I'm not saying the voting public is entirely blameless. But with all of the blatant corruption and disingenuous behavior from career politicians to which you seem to allude, can you really be surprised that many people refrain from voting?
What would you specifically change, if you could singlehandedly implement fixes to the way American electoral politics works?