Turning phantom power on will enable phantom power on both inputs on the 22 VSL.
Obviously if you do not need phantom power for a device you should not use it. However, it should not be problematic to have phantom power turned on unless you're using an old ribbon mic (or some other device that can be damaged by 48v).