Computer I/O Considerations
Posted: Fri Aug 22, 2014 5:41 am
The "Attention StudioOne Experts (and anyone else who knows)" thread in this forum got me started on this.
Hopefully there are a few tidbits in here somewhere that will be useful to somebody.
First some terminology, then some application.
Feel free to skip over the stuff you already know.
It's helpful to remember the difference between "storage" and "memory".
They're both measured in the same units (i.e. Bytes, Megabytes, Gigabytes, Terabytes, etc.), but that doesn't mean they're the same thing.
"Storage" equals "disk space" on most computers, and it can be inside your machine or in another device that is attached to it with some kind of cable or wireless connection.
"Storage" retains its data even when the computer is turned off.
"Cloud Storage" like DropBox, Google Drive, etc. is accessed over the Internet and is limited by the speed of your Internet connection.
Most Internet connections transfer data at only a fraction of the speed of a local storage device.
That means Cloud storage is useful for archives, backups and some types of database applications, but it's not fast enough to handle lots of dynamic data going back and forth.
"Memory" (i.e. RAM) only works while the computer is powered up, but it's MUCH faster than accessing data from any kind of storage.
Unfortunately, everything in RAM disappears when the machine is rebooted or turned off.
In order to run any program, some or all of it has to be read from storage and loaded into RAM.
Studio One, VST's and other plug-ins all work that way.
Any user data (song files, audio files, etc.) that have to be actively processed by a running program also have to be in RAM; the computer's brain can't manipulate anything unless it's first brought into memory.
The computer's operating system (Windows, IOS, Linux, etc.) is almost always responsible for physically moving the data around when any application program like a word processor or a DAW requests it.
This includes both local and networked data.
If the amount of required data won't all fit into the available RAM, the computer's operating system helps with very quickly loading it and unloading it in chunks, as needed to satisfy the application's needs.
Sometimes these data swaps between storage and RAM have to happen thousands of times per minute (or more).
Think of a computer as your office, with a filing cabinet and a desk in it.
You are a computer program.
"Storage" is the filing cabinet full of documents.
When you want to use a document, you have to pull it out of the cabinet and lay it on the desk in front of you so you can read it or write on it.
The surface of the desk is the "RAM".
You might have to put the first document away in order to make room on the desk for the next one.
The operating system is a secretary who swaps documents between the file cabinet and the desk for you when needed, sometimes thousands of times a second (because you are a very demanding boss).
If this doesn't happen fast enough (perhaps because she has to take turns with another secretary trying to access the same filing cabinet, or you have to alternate with another boss trying to use the same desk) you get upset because it makes you wait and throws off your timing.
[This is obviously an incomplete analogy but it covers the basics. The Wachowski brothers obviously stole it from me when they wrote "The Matrix" series of movies.]
DAWs like Studio One have to move a lot of data around while they're working
At the same time they're very dependent on accurate timing, like music is.
Any delays in accessing their data can mess things up drastically.
They ALWAYS work better and more reliably when all the information they need is stored in a location that can be accessed quickly.
They simply cannot do their job if they don't get the needed chunks of data transferred from storage to RAM fast enough.
Different types of storage respond at different speeds:
- Some hard drives are faster than others, and "SSD" drives are hugely faster than mechanical ones.
- Transferring data to/from a USB drive is almost always slower than moving the same stuff to/from an internal hard drive.
- Moving data across a network connection is usually even slower; frequently it's a lot slower.
This is partly because when data is transferred across a network it has to be broken up into small pieces called "packets".
Each of these packets has a bunch of other stuff added to the beginning and end of it to tell the receiving computer how it needs to be reassembled at the other end.
The disassembly/reassembly process and the added data slow things down.
- If you access data across a network when it's on a USB drive connected to a different computer, it means you have two bottle-necks between your computer and the data it needs; potentially slower still.
Many programs only load part of themselves into RAM when they start up.
Other program components are loaded later when they're needed.
Sometimes that happens while the program and storage devices are already busy working hard at something.
[I don't know for sure if Studio One works that way, but I suspect it does.]
That's one reason a program can seem to load fine from relatively slow network storage initially, but then misbehave later under load.
By that time lots of things are using the computer's resources concurrently (possibly including the network storage location) so things can get pretty sluggish or even stall competely.
Fast storage means fast swaps, slow storage means slow swaps.
If they don't happen fast enough, programs run slowly, or get unreliable, or don't work at all.
Studio One requires good computer performance and fast data transfers.
Otherwise it can't keep up with everything it has to do, and you start to have audible problems
Because of all this, it's best to put the Studio One program, plug-ins and data each it its proper place, and do everything you can to optimize the performance and capacity of your computer.
1. Try to have at least 4GB of RAM on your computer.
If your computer is using a 32-bit OS it can't use more than 4GB of RAM under any circumstances.
If you have a machine with a 64-bit OS, try to get yourself to 8GB or 16GB.
More than 16GB probably won't really speed anything up.
2. Read and pay attention to the "Minimum Requirements" list for any program you want to install.
Understand that "Minimum" really does mean exactly that.
If your computer barely meets the minimum requirements, the program will probably load if it's the only thing your computer is doing, but it may bog down if you ask it to do anything very demanding or try to run multiple programs concurrently.
The minimum requirements for Studio One were obviously published before "Core Duo" CPU's became obsolete and newer processors became so inexpensive.
If you are in the market for a new computer, the mid-2014 CPU pricing "sweet spot" for running powerful DAWs is probably a 2.4GHz (or faster) Intel i5 or i7 processor with a 64-bit operating system and 8 to 16 GB of RAM.
Faster is better, but gets more expensive pretty quickly.
It's futile to try to run Studio One (or any other program) on a machine that's simply too weak to handle the demands of the job.
Don't try it unless you enjoy frustration.
3. The Studio One program, all VST's/plug-ins and the critical data required by them should always be installed onto your fastest internal hard drive if possible.
Try to avoid installing these on any kind external storage, especially a network-based storage device.
If you're using a desktop PC and don't have enough space on the system drive, you can upgrade to a bigger one or add a second internal hard drive (they're really cheap these days).
If you're on a machine that only supports one hard drive, try to make sure that disk is as fast as possible and has enough room for everything.
4. It will usually work fine to put your S1 data (Macros, Presets, Songs, Sound Sets and Templates) on external storage, then use the "Locations" Options in S1 to tell it where to find them.
It's still possible this might not work well if your external storage doesn't respond fast enough to keep up.
In that case you might be able to copy the active data from external storage to a folder on your internal drive while you're working on it, then put it back on the external drive when you're done.
5. When you're working on music, make sure you turn off all other programs you're not using to free up as much computer capacity as possible.
This is especially important if your computer spec's are close the program's "minimum requirements".
6. If you insist on trying use a network storage location for program installations or active data make sure it's a high-performance device and your network is fast and reliable.
Attempt to arrange things so you're the only user hitting it during the times you're working on music.
You MIGHT get adequate performance from it that way.
Be prepared for it to work poorly when under high load.
7. Be aware that wireless networks can cause lots of problems.
Because of the way wireless networking protocols work, the data stream can slow down and speed up arbitrarily.
The whole connection sometimes has to be renegotiated at unpredictable times.
When the signal is weak, the data rate slows down to compensate, and data packets sometimes have to be sent multiple times before they are received error-free.
These things obviously aren't good for any program attempting to send or receive a continuous stream of data.
If you have lots of problems with dropouts, excessive buffering delays or serious stalls, it might be a wireless network issue.
Move your wireless equipment closer to your wireless router, or try a cabled connection instead.
8. Troubleshooting - It helps to setup things to be as simple and direct as possible to start with, then if you want to change or add stuff, do it one thing at a time so you won't end up with multiple issues affecting or magnifying each other and making problem diagnosis more difficult.
Example: If your music stuff is dependent on a network in some way, use cabled network connections until you're sure it all works, then try switching to wireless if that's your goal.
Computers, storage devices and networks vary widely, so your mileage may vary.
It's possible somebody might get away with doing everything I recommend against, but I believe that will be a rare exception.
I'm attempting to share some useful knowledge with the Studio One community because I hate to see people going through the same struggles I did while I was learning all this stuff the hard way.
The ideas work for me, and are based on 43 years of using computers, 25 years as a professional computer consultant, 29 years using electronic musical instruments and music-related computer programs, and over 40 years as a semi-professional cellist.
No, that doesn't mean I'm 137 years old; I did most of them concurrently...
But yeah, I'm pretty old... Bragging? Maybe a little... Too wordy? Absolutely.
My apologies if this is too much, but I hope it helps somebody anyway.
Sometimes I get going and can't stop until I get it all out.
PS: Don't configure your network devices to use "Jumbo Frames" unless every device on your network supports them and are ALL configured to use them (especially the Ethernet Switch).
Otherwise, they probably won't actually be used anyway; they may even cause problems for some devices.
Implementing Jumbo Frames on a network usually doesn't speed up transfers that involve long sequences of small chunks of data, like the data-stream used by many DAW's.
They can sometimes help when very large data files need to be transferred across a network connection all at once, but even if they do work the improvement in throughput is frequently only a few percent.
The disadvantages usually outweigh the benefits, except in some limited applications.
PPS: "Network Overhead" significantly affects the speed of data transfers over Ethernet.
A "100 Mbps" (12.5 megabytes per second) network will usually max out at about 65 Mbps (8 megabytes per second) of actual throughput.
1 Gbps networks are not really 10 times as fast as 100 Mbps networks because of the way they use network cables and how the signaling has to be done.
They usually do about 540 Mbps (68 megabytes per second) under good conditions.
In the real world, local disk I/O is generally much faster.
PPS: Any spelling, punctuation or grammatical errors are completely my fault but I'm not going to edit this crazy post anymore. The run-on sentences and unnecessary verbosity are here because that's the way I really talk.
Hopefully there are a few tidbits in here somewhere that will be useful to somebody.
First some terminology, then some application.
Feel free to skip over the stuff you already know.
It's helpful to remember the difference between "storage" and "memory".
They're both measured in the same units (i.e. Bytes, Megabytes, Gigabytes, Terabytes, etc.), but that doesn't mean they're the same thing.
"Storage" equals "disk space" on most computers, and it can be inside your machine or in another device that is attached to it with some kind of cable or wireless connection.
"Storage" retains its data even when the computer is turned off.
"Cloud Storage" like DropBox, Google Drive, etc. is accessed over the Internet and is limited by the speed of your Internet connection.
Most Internet connections transfer data at only a fraction of the speed of a local storage device.
That means Cloud storage is useful for archives, backups and some types of database applications, but it's not fast enough to handle lots of dynamic data going back and forth.
"Memory" (i.e. RAM) only works while the computer is powered up, but it's MUCH faster than accessing data from any kind of storage.
Unfortunately, everything in RAM disappears when the machine is rebooted or turned off.
In order to run any program, some or all of it has to be read from storage and loaded into RAM.
Studio One, VST's and other plug-ins all work that way.
Any user data (song files, audio files, etc.) that have to be actively processed by a running program also have to be in RAM; the computer's brain can't manipulate anything unless it's first brought into memory.
The computer's operating system (Windows, IOS, Linux, etc.) is almost always responsible for physically moving the data around when any application program like a word processor or a DAW requests it.
This includes both local and networked data.
If the amount of required data won't all fit into the available RAM, the computer's operating system helps with very quickly loading it and unloading it in chunks, as needed to satisfy the application's needs.
Sometimes these data swaps between storage and RAM have to happen thousands of times per minute (or more).
Think of a computer as your office, with a filing cabinet and a desk in it.
You are a computer program.
"Storage" is the filing cabinet full of documents.
When you want to use a document, you have to pull it out of the cabinet and lay it on the desk in front of you so you can read it or write on it.
The surface of the desk is the "RAM".
You might have to put the first document away in order to make room on the desk for the next one.
The operating system is a secretary who swaps documents between the file cabinet and the desk for you when needed, sometimes thousands of times a second (because you are a very demanding boss).
If this doesn't happen fast enough (perhaps because she has to take turns with another secretary trying to access the same filing cabinet, or you have to alternate with another boss trying to use the same desk) you get upset because it makes you wait and throws off your timing.
[This is obviously an incomplete analogy but it covers the basics. The Wachowski brothers obviously stole it from me when they wrote "The Matrix" series of movies.]
DAWs like Studio One have to move a lot of data around while they're working
At the same time they're very dependent on accurate timing, like music is.
Any delays in accessing their data can mess things up drastically.
They ALWAYS work better and more reliably when all the information they need is stored in a location that can be accessed quickly.
They simply cannot do their job if they don't get the needed chunks of data transferred from storage to RAM fast enough.
Different types of storage respond at different speeds:
- Some hard drives are faster than others, and "SSD" drives are hugely faster than mechanical ones.
- Transferring data to/from a USB drive is almost always slower than moving the same stuff to/from an internal hard drive.
- Moving data across a network connection is usually even slower; frequently it's a lot slower.
This is partly because when data is transferred across a network it has to be broken up into small pieces called "packets".
Each of these packets has a bunch of other stuff added to the beginning and end of it to tell the receiving computer how it needs to be reassembled at the other end.
The disassembly/reassembly process and the added data slow things down.
- If you access data across a network when it's on a USB drive connected to a different computer, it means you have two bottle-necks between your computer and the data it needs; potentially slower still.
Many programs only load part of themselves into RAM when they start up.
Other program components are loaded later when they're needed.
Sometimes that happens while the program and storage devices are already busy working hard at something.
[I don't know for sure if Studio One works that way, but I suspect it does.]
That's one reason a program can seem to load fine from relatively slow network storage initially, but then misbehave later under load.
By that time lots of things are using the computer's resources concurrently (possibly including the network storage location) so things can get pretty sluggish or even stall competely.
Fast storage means fast swaps, slow storage means slow swaps.
If they don't happen fast enough, programs run slowly, or get unreliable, or don't work at all.
Studio One requires good computer performance and fast data transfers.
Otherwise it can't keep up with everything it has to do, and you start to have audible problems
Because of all this, it's best to put the Studio One program, plug-ins and data each it its proper place, and do everything you can to optimize the performance and capacity of your computer.
1. Try to have at least 4GB of RAM on your computer.
If your computer is using a 32-bit OS it can't use more than 4GB of RAM under any circumstances.
If you have a machine with a 64-bit OS, try to get yourself to 8GB or 16GB.
More than 16GB probably won't really speed anything up.
2. Read and pay attention to the "Minimum Requirements" list for any program you want to install.
Understand that "Minimum" really does mean exactly that.
If your computer barely meets the minimum requirements, the program will probably load if it's the only thing your computer is doing, but it may bog down if you ask it to do anything very demanding or try to run multiple programs concurrently.
The minimum requirements for Studio One were obviously published before "Core Duo" CPU's became obsolete and newer processors became so inexpensive.
If you are in the market for a new computer, the mid-2014 CPU pricing "sweet spot" for running powerful DAWs is probably a 2.4GHz (or faster) Intel i5 or i7 processor with a 64-bit operating system and 8 to 16 GB of RAM.
Faster is better, but gets more expensive pretty quickly.
It's futile to try to run Studio One (or any other program) on a machine that's simply too weak to handle the demands of the job.
Don't try it unless you enjoy frustration.
3. The Studio One program, all VST's/plug-ins and the critical data required by them should always be installed onto your fastest internal hard drive if possible.
Try to avoid installing these on any kind external storage, especially a network-based storage device.
If you're using a desktop PC and don't have enough space on the system drive, you can upgrade to a bigger one or add a second internal hard drive (they're really cheap these days).
If you're on a machine that only supports one hard drive, try to make sure that disk is as fast as possible and has enough room for everything.
4. It will usually work fine to put your S1 data (Macros, Presets, Songs, Sound Sets and Templates) on external storage, then use the "Locations" Options in S1 to tell it where to find them.
It's still possible this might not work well if your external storage doesn't respond fast enough to keep up.
In that case you might be able to copy the active data from external storage to a folder on your internal drive while you're working on it, then put it back on the external drive when you're done.
5. When you're working on music, make sure you turn off all other programs you're not using to free up as much computer capacity as possible.
This is especially important if your computer spec's are close the program's "minimum requirements".
6. If you insist on trying use a network storage location for program installations or active data make sure it's a high-performance device and your network is fast and reliable.
Attempt to arrange things so you're the only user hitting it during the times you're working on music.
You MIGHT get adequate performance from it that way.
Be prepared for it to work poorly when under high load.
7. Be aware that wireless networks can cause lots of problems.
Because of the way wireless networking protocols work, the data stream can slow down and speed up arbitrarily.
The whole connection sometimes has to be renegotiated at unpredictable times.
When the signal is weak, the data rate slows down to compensate, and data packets sometimes have to be sent multiple times before they are received error-free.
These things obviously aren't good for any program attempting to send or receive a continuous stream of data.
If you have lots of problems with dropouts, excessive buffering delays or serious stalls, it might be a wireless network issue.
Move your wireless equipment closer to your wireless router, or try a cabled connection instead.
8. Troubleshooting - It helps to setup things to be as simple and direct as possible to start with, then if you want to change or add stuff, do it one thing at a time so you won't end up with multiple issues affecting or magnifying each other and making problem diagnosis more difficult.
Example: If your music stuff is dependent on a network in some way, use cabled network connections until you're sure it all works, then try switching to wireless if that's your goal.
Computers, storage devices and networks vary widely, so your mileage may vary.
It's possible somebody might get away with doing everything I recommend against, but I believe that will be a rare exception.
I'm attempting to share some useful knowledge with the Studio One community because I hate to see people going through the same struggles I did while I was learning all this stuff the hard way.
The ideas work for me, and are based on 43 years of using computers, 25 years as a professional computer consultant, 29 years using electronic musical instruments and music-related computer programs, and over 40 years as a semi-professional cellist.
No, that doesn't mean I'm 137 years old; I did most of them concurrently...
But yeah, I'm pretty old... Bragging? Maybe a little... Too wordy? Absolutely.
My apologies if this is too much, but I hope it helps somebody anyway.
Sometimes I get going and can't stop until I get it all out.
PS: Don't configure your network devices to use "Jumbo Frames" unless every device on your network supports them and are ALL configured to use them (especially the Ethernet Switch).
Otherwise, they probably won't actually be used anyway; they may even cause problems for some devices.
Implementing Jumbo Frames on a network usually doesn't speed up transfers that involve long sequences of small chunks of data, like the data-stream used by many DAW's.
They can sometimes help when very large data files need to be transferred across a network connection all at once, but even if they do work the improvement in throughput is frequently only a few percent.
The disadvantages usually outweigh the benefits, except in some limited applications.
PPS: "Network Overhead" significantly affects the speed of data transfers over Ethernet.
A "100 Mbps" (12.5 megabytes per second) network will usually max out at about 65 Mbps (8 megabytes per second) of actual throughput.
1 Gbps networks are not really 10 times as fast as 100 Mbps networks because of the way they use network cables and how the signaling has to be done.
They usually do about 540 Mbps (68 megabytes per second) under good conditions.
In the real world, local disk I/O is generally much faster.
PPS: Any spelling, punctuation or grammatical errors are completely my fault but I'm not going to edit this crazy post anymore. The run-on sentences and unnecessary verbosity are here because that's the way I really talk.