Yeah, it is annoying, the clicks are the worst. Even changing screens when audio is playing causes a momentary click/silence in audio. Changing buffer sizes can cause the clicks also. Focusrite had the best way, I never had clicks when using them, but you could daisy chain 2 Saffires, I had the Pro56 and the 28, both connected via Firewire, but was seen as only 1 device, no need for aggregate set ups, which I think is the whole cause of this, that and it seems when an application wants access to the audio interface, momentarily, all audio inputs are open...total guess, but their shouldn't be clicks due to screen redraws/changes.
I think Apple OS has something to do with it also, but using word clock, ticking or unticking Drift Correction, it all fails at some point. Personally, I've spent too much time trying to sort this by unplugging everything, re-installing, going into apple plists and deleting things etc. it's clearly flawed.