Short clip: Technorati reducing costs during weekend downtime

Sumi Das: You talked about how you're indexing all those blog posts. How do you ensure that the data centers you've designed and built are efficient from a cost perspective and also from an energy perspective? Dorion Carroll: Right, so it's really interesting. I mean, again, we are a very small company and at the moment, we have a single data center. You know, people may have the impression that, you know, we're as big as Google. Well, we're not. We're not nearly as big as Google. Having a single data center is definitely a disadvantage in some ways, but in terms of how do we balance the cost versus the benefit, that's where we are right now. Interestingly, we've been looking at next generations for several of our core components within the infrastructures. So, we have our data acquisitions systems. The things that go out and crawl the web, bring things back, check to see whether blog posts have changed and if they have pass it on to the indexing infrastructure and then we have our website infrastructure as well, so sort of these three main components. Our architect Ian Kallen has actually advanced us considerably in terms of how to get out of a single data center. We're taking a look at the Amazon web services. We've launched a new crawler now in the middle of August. We're sort of, "in this process" of slowly migrating from the old infrastructure to the new infrastructures. We actually have to be able to run things in parallel. This is one of the tricks you use when you're trying to change the tires while you are driving across country. You can't necessarily do everything at once. There is no big switch. You can just pull and automatically switch over. So, using Amazon, we're actually discovering some very interesting capabilities, the elastic computing framework, which says, you know, if a whole bunch of blogs are posting and you know now we can grow the number of servers we have just auto-magically, spawn a bunch of new servers, and on the weekends, it turns out bloggers don't blog as much on the weekends. So, we can actually tune that down. Those are cost savings to us. We don't actually have to go into our own data center. We can use APIs. We are writing software to detect these things automatically. So, if we have queues that are starting to fill up, we can launch more machines and drain those queues faster. If the queues are empty and nothing seems to be processing, we can actually takes the machines down. So, for us there's actually a balance between our own data center. We operate about 700 machines to keep Technorati.com running and then we've also got some of the elastic capabilities up in the Amazon cloud.

==== Transcribed by Automatic Sync Technologies ====