Yes try a 64 buffer setting for sure. It may reduce your latency slightly. The idea is to keep the buffers reasonably low while tracking. Especially if you are monitoring through software and maybe using plugins on the way in which you can do in Studio One.

Midi latency is fast. 1 mS for a note on. So on an instrument track you might only have 1 mS or so on the way in but the output latency will determine how was fast a virtual synth is going to respond. 4-5 mS or so here would be fine. This latency would also be fine for tracking something live and monitoring it. When dry and delayed signals are heard together, some tight delay or flanging like effects can result.

You can always ignore DAW latency anyway by monitoring with zero latency through the Focusrite Control software. Understand and use Focusrite Control to advantage. You can route signals from inputs to outputs etc.. You could record say vocals and guitar to a click in Studio One and hear all that with no latency, then playback and overdub easily etc..