Cloud Hosting Provider Hardware Benchmarks

While running some hardware benchmarks on my new Core i7 Apple MacBook Pro, I got the idea to run the same benchmarks across several Cloud hosting platforms to see how their hardware stacks up. The tool I used to perform the benchmarks is Geekbench from Primate Labs. Now, this benchmark was designed to accurately test processor and memory performance so these benchmarks do not take into account disk i/o, network i/o, etc. In addition to the performance results, we learn a bit about the underlying hardware on the host machines that run these platforms. The different providers benchmarked here are: Amazon EC2, GoGrid, Linode, Rackspace Cloud Servers and Storm on Demand (by LiquidWeb).

Continue Reading

Apple MacBook Pro Core i7 GeekBench Benchmarks & Review

So about a week ago I got very impulsive and dediced to upgrade from my 3.06 C2D MacBook Pro to the new Core i7 based model that was recently released. I wasn’t simply after the latest and greatest processor however, there were some other significant changes that I was eager to experience first hand, so this is the model I purchased:

  • 15 Inch Matte High Resolution Display (big selling point)
  • Core i7 2.66 GHz
  • 4GB RAM
  • NVIDIA GeForce 330M 512MB

I really wanted to purchase a laptop with some newer architecture than the Core 2 Duo chips that have been out for a while as I’m looking to stay with the same machine for at least 2-3 years, which if you know me is near impossible! However, I finally brought myself to purchase AppleCare now that I’m comfortable with a machine that will last me for a while.

Continue Reading

Cloud / VPS Apache Performance Comparison

UPDATE #1: I received several comments about the testing methodology used in these benchmarks so I wanted to explain further. Each server instance at the Rackspace Cloud, Linode and Amazon are running the exact same Linux distribution (Debian 5.0 Lenny), kernel, architecture (x86_64) and version of Apache (2.2.9 w/ worker MPM).

UPDATE #2: After much concern was raised I went ahead and reran the benchmarks on the small instances after tuning a few settings on the Linode server.

echo 99999 > /proc/sys/net/ipv4/netfilter/ip_conntrack_max

Also, instead of running the tests from a consistent remote location I ran them on the local machines this way I could guarantee that network latency wouldn’t impact the test results in any way.

—–

In the past few weeks there has been a lot of talk about the differences between various Cloud and VPS providers (see my post), mainly revolving around performance. I decided to take some time last week to run some very straight forward and real world tests to gauge performance among the most popular providers: The Rackspace Cloud, Amazon EC2 and Linode.

For the benchmark I decided to use ApacheBench and Siege as *most* users on these platforms are serving some type of web content via Apache. Even if different web server software is used, it is unlikely that the results and trends would differ. As part of a series of posts, I will be running the same benchmarks against static content and dynamic content. This way we can be sure to look fairly at all aspects of each particular platform.

As mentioned, I decided to use ApacheBench and Siege as the testing platforms so I performed a default install of Apache 2.2 with the Worker MPM on each server instance. The server OS platform is Debian 5.0 Lenny on all servers.

These benchmarks are very easy to duplicate, so if you would like to test the results simply replicate the total connections or time and concurrency settings mentioned below.

I did not tweak any of the Apache settings on any of the server instances. I used the default settings of:

<IfModule mpm_worker_module>
StartServers                             2
MaxClients                              150
MinSpareThreads                  25
MaxSpareThreads                 75
ThreadsPerChild                    25
MaxRequestsPerChild          0
</IfModule>

I ran the same tests against each platform and since I was only hitting the default HTML page generated by the Apache install I decided to raise the concurrency limit up to something significant to generate some real load. Each benchmark was run three times over three different days and I took the best result from each set of tests.

ApacheBench: 100,000 Total Connections / 100 Concurrent Connections
Siege: 10 Minutes Under Siege / 50 Concurrent Connections

Let’s get to the results!

Small Size Instances

ApacheBench Results

Rackspace 256MB Cloud Server Linode 360MB Media Temple DV 512
Concurrency Level 100 100 100
Requests [#/sec] (average) 10,706.64 4,554.43 N/A
Time Per Request [ms] (mean) 9.340 21.957 N/A
Transfer Rate [Kbytes/sec] Received 3336.14 1,414.39 N/A

Siege Results

Rackspace 256MB Cloud Server Linode 360MB Media Temple DV 512
Transactions 1,587,997 1,165,412 N/A
Availability 100.00% 100.00% N/A
Response Time 0.04 secs 0.05 secs N/A
Transaction Rate 2,647.68 trans/sec 1,942.90   trans/sec N/A
Successful Transactions 1,587,997 1,165,412 N/A
Failed Transactions 0 0 N/A
Longest Transaction 0.42 0.61 N/A
Shortest Transaction 0.00 0.00 N/A

Review

The Rackspace Cloud Server instance, with only 256 MB of RAM, performed well under the stress as did the Linode 360 server, but there is still a significant difference in the overall performance. This could be due to host server utilization, but according to the Linode control panel the host machine my instance was running on is “low”.

In the Siege results we get some similar insights. The 256 MB Rackspace Cloud Server processed significantly more transactions than the Linode server. Importantly, the transaction rate was also much higher.

There is always a lot of discussion around the Linode 360 server instance being a better value than the 256 MB Rackspace Cloud Server, but the tests show otherwise. In my opinion this is most likely due to Rackspace Cloud’s lower utilization of host hardware and a more robust network.

Large Size Instances

Now lets take a look at the larger instances. I wanted to work Amazon’s EC2 platform into these performance tests and their large instance is most comparable to the Cloud Server and Linode instances of the 8GB variety.

ApacheBench Results

Rackspace 8192MB Cloud Server Linode 8640MB Amazon EC2 Large
Concurrency Level 100 100 100
Requests [#/sec] (average) 10,729.61 9,109.91 6,782.93
Time Per Request [ms] (average) 9.320 10.977 14.743
Transfer Rate [Kbytes/sec] Received 3,363.62 2,829.17 2,113.04

Siege Results

Rackspace 8192MB Cloud Server Linode 8640MB Amazon EC2 Large
Transactions 1,564,152 1,636,652 891,956
Availability 100.00% 100.00% 100.00%
Response Time 0.04 secs 0.04 secs 0.07   secs
Transaction Rate 2,606.79 trans/sec 2,727.34   trans/sec 1,485.80   trans/sec
Successful Transactions 1,564,152 1,636,652 891,956
Failed Transactions 0 0 0
Longest Transaction 0.49 0.22 0.65
Shortest Transaction 0.00 0.00 0.00

Review

As we saw with the smaller instances in the ApacheBench test, the Cloud Server with fewer resources is outpacing both the Linode server and the Large EC2 instance. But in the Siege benchmark we see the Linode instance take the crown for total transaction rate. I would note the larger memory allocation on the Linode server instance might have something to do with these results so be sure to compare price for your particular application.

Amazon’s Large EC2 instance was the slowest of the bunch during the ApacheBench testing but did complete all of the transactions thrown at it during the Siege benchmark.

Final Thoughts

The tests show some significant performance benefits when running on the Rackspace Cloud Servers platform on the low end and similar performance to larger Linode and Amazon instances on the high end even though the comparable Cloud Server had much less memory and therefore lower cost. In my opinion, I believe lower hardware utilization and better host server hardware contribute to these results.

Apache settings can definitely be tweaked to increase performance on any of these platforms; this was just a baseline test using the defaults provided. The next round of tests will focus on serving a dynamic web applications.

If you have any thoughts, questions or comments about the benchmarks performed here please leave a comment below. I’d love to hear from you!

Rackspace / Mosso Cloud Sites Demos

I’ve posted video demos for Cloud Servers and Cloud Files from The Rackspace Cloud, now lets take a look at Cloud Sites, which is the flagship cloud platform from Rackspace.

Cloud Sites is a fully managed platform-as-a-service hosting architecture which makes it unbelievably easy to provision new web sites and web applications on enterprise grade hosting infrastructure. Because the entire infrastructure is managed by the Rackspace Cloud team up to your application/code there are some limitations in the level of access that you have to the environment and the amount of customization that you can achieve, but the benefits of the platform FAR outweigh any inconvenience caused by this. Essentially, my general rule is that if what you are looking to host will work in the Cloud Sites environment, RUN IT ON CLOUD SITES.

No platform offers greater levels of scalability and redundancy out of the box. From the moment you provision a site/app onto Cloud sites it is load-balanced across multiple physical web servers and specialized database clusters as well. All centrally managed with the best hosting control panel in the industry, business class email from Mailtrust (a sister company), and direct file-system access via FTP/SFTP. Cloud Sites can also easily be used in conjunction with Cloud Files and Cloud Servers.

For $100/month there is simply no other compelling offer in the market, especially when you factor in the 24x7x365 LIVE Fanatical Support delivered by the Rackspace Cloud / Mosso team.

I posted some video demos showing the functionality of the control panel as well as performing some basis tasks within the platform (FTP, database management, etc).

Adding A New Site

Get the Flash Player to see this content.

Creating & Managing Databases

Get the Flash Player to see this content.

FTP

Get the Flash Player to see this content.

Mosso Cloud Sites & Media Temple GS – Put To The Test

Strong web site performance, reliability, and uptime – these are all traits you would expect from an enterprise-class Cloud hosting platform, right? That is exactly how I feel now – and exactly what I expected almost three years ago when I started looking at the very early cloud hosting platforms of the time.

My name is Chad and today I am a Mosso employee. Long before I joined the Mosso team, however, I enjoyed (sometimes) hosting many of my own web sites and those of friends, colleagues, family, etc. Back in late 2006 I was looking for a better platform for the sites I hosted. I was tired of trying to manage my own solution, and I was frustrated with the downtime and poor support of all the VPS providers I had used over the years. Cloud hosting was a very new concept and, in fact, I don’t think it was even referred to as “cloud” just yet, but there were two main providers in this space at that time: Mosso & Media Temple. Like many, I was concerned about the initial concepts but very intrigued with the technology so I splurged and purchased hosting accounts both with Mosso and with the Media Temple Grid platform. I decided to do some testing on two basic fronts: response time & uptime. As you may have figured by now, one platform excelled in comparison, and I was so intrigued that I ended up coming to work for Mosso!

So now, in early 2009, I thought, “Why not duplicate those same tests and see what the results would be?” And that is exactly what I did. I set up identical static web sites serving a 100KB image and identical dynamic sites, a vanilla WordPress 2.7.1 install on both platforms (Cloud Sites from Mosso and Grid-Service from Media Temple). Next I decided to leverage Pingdom for all the testing, as they have a wide variety of testing locations around the world. These tests for uptime and response time were performed via a full HTTP check (not ping) and occurred in 1 minute and 5 minute intervals on different copies of the static and dynamic sites. I set the test to run during the middle of a typical business week and collected roughly 72 hours of data.

The results were very impressive. Uptime percentages for Mosso Cloud Sites were a solid 100% compared to about 99.0% on average with Media Temple GS, and downtime stuck solid to zero seconds, while Media Temple was down 1-5 times across the different test sites resulting in almost 30 minutes of downtime on average per site. Average response times for Mosso Cloud Sites were at least half of that of MT GS, with Cloud Sites sometimes operating almost as much as five times faster!

This test is easy to duplicate if you would like to try it firsthand. If you have any questions, please do not hesitate to contact me. Please see the results below and you can see the direct Pingdom reports here: http://www.pingdom.com/reports/ndv2m2bh6fnw/