You would need to set this up in your DAW and use the ADAT output to an External PreAmp with direct outs such as the DP88 or similar ADAT device and use ADAT outputs: 1/2 for LR, 3/4 for Rear Left, Rear Right, 5 for Center and 6 for your subwoofer.
Studio One does not support mixing in 5.1, other DAWs such as Logic, Cubase, Nuendo, Reaper, Pro Tools and other support mixing in surround.
If you're asking if it can be used to send AC-3 or DTS over RCA-SPDIF to a home receiver to decode surround sound from a game or a media player playing a DVD or Blu-Ray, this is not supported. You can try however your results may vary.
