That is a loose quote from a SXSW panel on scalable web ventures. This just hits way to close to home today to not write about. In our case it was both the DB and the File System. We were doing some last minute load testing, you can never be too sure, and after almost the entire day and roping in two other teams to dig into the db and SAN we realized the db had not been setup right. All files associated with the specific database were on the same storage LUN. This caused us a 95% reduction in throughput of our service. Splitting up the t-logs, data files, etc. onto different LUNs got us back to where we expected. The bottom line is we wouldn’t even have noticed this with out trending everything possible in cacti on our hosts and while win2k3 disk trending has some hurdles for SAN attached disk, it still pointed us in the right direction.