Well I don’t want to make this blog about aggregating others blogs but I came across a great post regarding metrics this afternoon that I think we should read.

How and what to measure via Ask the Wizard

Trending is all the rage and rightfully so. To put it as simply as possible this allows us to know what is going to break before it does. We have been using tools like cacti and HitBox for some time now to try and predict what to expect in our upcoming events.

One thing I have always struggled with is the bigger picture. What does the trend look like over the year or into next? Are we, as a whole, gaining ground or loosing it? What makes up those metrics? A new tool similar to Amazon’s Alexa has come out recently called compete.com Compete really isn’t that different but it does something important for me. It reinforces the data that Alexa has had a lock on for years. Using this data I can confidently say yes it does appear we are down 18% year over year.

The next step in all this is translating that into operations speak.. So this means we can reduce our footprint by 18%? Well no not exactly we are serving more dynamic content then ever and oh did we mention the new video service launching next week.. This is in part what makes operations interesting trying to find better ways to help scale but with reduced cost, smaller footprints, and oh right less POWER. Sounds like a topic for another post…..

Good movie quote, recently brought to me by a colleague when discussing the virtues of accountability and owning the health, maintenance and restoration for systems we are responsible for. Broader topic was “monitoring.”

I don’t expect magic, in buckets or within the context of goodness. I do expect that if we are really empowered to engineer and manage systems, we need to ensure they are built correctly and alert us on state and utilization. Empowerment = given the funds we request to build and instrument correctly. Thus far, I haven’t recieved any resistance to properly teed-up requests.

Thus…a good quote: “I don’t wanna hear about no motherf***in’ ifs. All I wanna hear from your ass is, You ain’t got no problem, David (Jules). I’m on the motherf***er!”

Operations Catch Phrases:

4. That’s just like that Seinfield episode where Jerry said…

3. Mainstreaming Web 2.0 Trends

2. I’m no network engineer, but…

1. Magic buckets of goodness

I woke up this morning and my alarm clock proudly displayed the time. I was so bummed.

I looked across the room at the digital atomic clock + thermometer on the bookcase. Perfectly correct as usual. Damn.

My last hope: the old-looking analog clock that actually has an internal Atomic radio receiver. It too, was right.

I had been so convinced that this was really going to be it. Y2K was such a huge letdown; I really thought that this time would be different. This time it wasn’t some simple 2-digit vs. 4-digit date thing. THIS was Daylight Saving Time! Something this sacred simply had to wreak havoc on our technology. How would it deal with this change? Surely my TiVo was going to record the wrong show, the power would go out, and my car would not start. Yet to my ever-living disappointment, all that happened was that my heat turned on an hour late so the house was a little cold when I got up this morning. So that’s how I began my day: coldly disappointed.

Have we mentioned we work Sundays?

Well, we don’t work every Sunday, but we work whatever it takes when it comes to an event. And it just so happens it’s event time again.

Like Joe blogged, the week has been full of nearly full-time vendor-management. In addition to that full time job, we’ve also been busy trying to coordinate with our content production teams to make sure that everything was in order. And, we’ve been busy trying to make sure all of our devices, all of our services, and all monitoring and instrumentation was all buttoned up.

But now we’ve reached Terminal Velocity. There’s no stopping now. All the wheels are in motion. Between now and the end of this week, we will find out just how much our hard work and dedication will pay off. Will the site stay up? We’re pretty sure it will. Will the load balancer melt down? We don’t think so. And will we provide a killer experience for all of our users.

Sixteen days and a few hours after a case was opened we have what appears to be a resolution to a compression problem that has been driving us insane. Our vendors senior architect worked what appears to be day and night for the past few days to patch his code and come up with a solution to a problem that was discussed earlier this week as “not possible.”

As frustrating as the experience was I am amazed at the level of commitment to resolving this issue within the time frame that we gave them. Now its time to test it out for real…..

Last Friday as soon as Zach and I got into the office we started discussing a pile of issues we had been having with a certain unnamed network load balancing piece of hardware. Part of our frustration has been that that the vendors product is this “magic bucket of goodness” that they hold very close to their heart and we have little insight into the specific architecture of the device. This puts us at the mercy of the vendors support, and our social engineering skills, when we have a issue as nasty as the one we have been dealing with.