This bites me every day and as far as I know every other DAW has had this since the dawn of time.
I often have many tiny slices of audio in a row with no space at either end. You cannot select by swiping and you cannot select by shift clicking.
Please tell me that there is a modifier I don't know about.