You could, but you would not be able to enjoy the benefits of stereo sound.
Just in case you're wondering what the benefits of stereo sound are, Stereophonic sound (stereo) is a method of sound reproduction that creates an illusion of directionality and perspective by using two independent speakers, as with human hearing.