Latency is the nature of how every computer processes real time audio. When you introduce an audible point of reference to this processing, you hear a delay. This processing is relative to the general power of your computer, how many things you are processing at one time and of course, buffer settings. Computers fill these buffers to process audio smoothly. This takes a relative amount of time which gives you latent delayed audio you hear. Increasing buffer or block sizes, you have more latency but audio will be process more smoothly. Decreasing buffer or block sizes, reduces the latent delay you hear but increases the chances of interruptions in audio processing. These interruptions are heard as clicks, pops or audio drop outs.
You computer is your ultimate tool. It processes everything. The more audio tracks, FX, VST plugins, etc... the more your computer needs to process smoothly. It boils down to a trade off. You have to find your own settings that will suite your own method of work and computer's general ability. Even a slow computer can record smoothly and even a super computer has an eventual limit. Nature of computers.
EZ drummer is a separate issue. If you are using Studio One 3 Artist, you will need the VST Plugin Add On to use any VST plugins in the Artist version. Studio One 3 Professional has VST plugin functionality already. If you have either the Add On for Artist or using Professional, and EZ drummer still does not work. You may need to go to the EZ drummer vendor website and make sure that you have there latest available update.
