Well, you can easily mute the active track, or put tracks in folders and mute those as a group, but there is no macro solution to specify which track.
The only option I know of right now is what you're doing, and that's start at the beginning and walk to the one you want. A pain, but since you'll only do it once, not a huge deal in terms of overall time.
Again: I agree it would be nice to address a specific track by number or track label. There may be a way to do it I haven't found yet (and I'm not a macro beginner... but I'm always learning.)
Hopefully someone will point out something we have missed so far.