August 28, 2008A Public Service Announcement: Some Hosts Overload Their Servers
Some hosts overload their servers. No, really, it’s true. 
I know some of you know this, but many people don’t and are quite surprised when they ask me see what I can do to improve the performance of their board. One of the first things I do before taking on a client with this type of request is run a check to see how many other sites are hosted on the same server. If the number is over 100, I don’t bother doing much other than telling the potential client to get a better host. How do you find this out? It’s not hard, really.
One of my current clients is building out a new site. While I was online editing (using vi directly on the server, as I like to do) I found that just saving the files would sometimes take several seconds. 
And this is for a 2K file, not something incredibly large. So the first thing I did was run the “top” command, and here’s what I saw:
load average: 4.29, 3.06, 2.92
On some boxes, a load of 3.0 might not be bad. The “big board” that I am frequently talking about runs on the same server as the phpBBDoctor blog. If I hit 2.0 I’m really surprised. And since I have a quad-cpu box, I don’t really get worried until I get closer to 4.0. Yet the client box I’m working on is regularly over 6.0 or even 8.0 during the day. The stats posted above were captured at 2am central time, so hardly a busy time of the day. Yet still the numbers are what I would consider quite high.
I gave that information to my client and suggested they needed to talk to their host about the server load. It should not take 3+ seconds to have a file saved from vi by any stretch.
Then, it’s on to my next tool. I take the IP address for the domain (easily found with a simple “ping” command in most cases). I then use a tool like that found at www.myipneighbors.com which will tell me how many other domains also respond to that IP. Generally hosts are using virtual hosting so they don’t run out of IP addresses. By doing this a host can essentially offer unlimited domains on the same box with only one IP address. Hmmm, unlimited domains. Does that start to sound like a bad omen? 
I went through this with a different client who was at a host that will remain unnamed but rhymes with why-power-web. Their very active site was on the same server that had over 500 sites, and that was the most I had personally seen. However, when I did the IP lookup on the server I was talking about above, here’s what I got: there are 737 sites sharing this IP address. 737! That’s a fairly large airplane, and way too many sites to host on a single server.
Conclusion? You get what you pay for. I am afraid that my client is going to be quite disappointed when they launch their new site on this new host.
Oh, and the host response to the first concern, about the server load being so high? It was something like, “Our boxes have multiple CPU’s so a high server load is not an issue.” They may have multiple CPU’s, but a high load is certainly an issue. We’ll have to see how long it takes them to fuss at my client because phpBB2 is “overloading” their already overloaded box.
Comments (17)
Very cool, thanks for that http://www.myipneighbors.com address.
One of the people I help with a web site that is using a cheap server that I am sure is shared comes back with only their web site under their IP, even though their site is very slow at times. Strange…
Another one (on GoDaddy) came back with 582!
Comment by Everett — August 28, 2008 @ 11:50 am
Whow,
I consider my hosting like quite good, serious and not very cheap.
However, when I look up my domain using the site you mention I find I’m sharing IP with 478 other domains.
What should I make of this? **scratch_head_smiley**
How do you run a top command?
Btw, I have no speed problems at all.
Comment by dogs and things — August 28, 2008 @ 1:44 pm
This probably isn’t the case here, but one IP can point to multiple computers, say if they are load-balanced.
Comment by Dog Cow — August 28, 2008 @ 4:22 pm
So far I win with 737 then?
Everett, the link is based on IP lookup. It may be that if you’re on a virtual or partitioned server, you have your IP all to yourself but you still share the server. In that case you are relying on the partitioning software to keep one virtual machine from taking more than its share of the resources.
dogs and things: the “top” command is a standard linux command. I log in to my server (command line, old school
) and simply type “top” to execute it. I use it to verify how busy my server is, how much RAM is being used, what my swap space looks like, and other things. Quite often if I am doing anything at all on my computer I will have the top command running in a window in the background, and old habit from when I was trying to rectify this issue.
The three numbers I posted are the aggregate CPU load for “now”, the average over the last five minutes, and the average over the last 15 minutes. So if the numbers are going up, something is spiking the processor(s). If the numbers are going down, the system is recovering. Numbers go up and down all day long, based on the numbers of people logged on, the search processes they run, background tasks and so on.
Comment by Dave Rathbun — August 28, 2008 @ 4:50 pm
737 does have me beat.
I ran it again and now it says 644. Getting closer!
If you want to see the load, you could make a php file on the your site with the following code:
<?php $load = exec("uptime"); echo $load . "<br />"; ?>I have a little footer on my site that shows the first number in that load, along with the time to display the page, number of queries, and so forth…. but only for admins.
Comment by Everett — August 28, 2008 @ 9:30 pm
Ooops, for reference, I have a html br appended to the echo $load above, I guess that doesn’t come through.
Comment by Everett — August 28, 2008 @ 9:32 pm
Great link there, Dave. Not only did you help solidify my research, but you gave me a new tool to play with.
Comment by Micheal — August 29, 2008 @ 8:58 am
Micheal, to be honest the service I posted is not the one that I have historically used. I never bookmarked it because I was always able to find it easily, and now that I want to share it, I can’t find it.
The “ip neighbor” tool does seem to work, but I stopped short of formatting it as a link (or putting it in a “Related Links” section at the bottom of my post as I often do) because I’m not exactly endorsing it just yet. If I can find the service that I have been using for years, I will add it as a link.
Everett: to post code, it unfortunately needs to be in “code” brackets and you also have to convert the < into the HTML equivalent of < to make it show up.
For what I almost considered a “throw-away” post this topic certainly seems to have provided some value.
It was really just posted for filler. 
Comment by Dave Rathbun — August 29, 2008 @ 9:57 am
How do you do it?
I can barely get one comment on a post, nevermind 8 or 9. 
I guess I need to write some more technical content. I’ll give you a run for your money, Dave. 
Comment by Micheal — August 29, 2008 @ 11:01 am
@Everett,
What am I supposed to do with that file, if I put it on my server and call it in my browser nothing shows, just the famous big blank space, but nothing is echoed?
Comment by dogs and things — August 29, 2008 @ 12:30 pm
Dave: I got you beat. 740 on one server.
Not mine, fortunately, but still.
Comment by Micheal — August 29, 2008 @ 1:08 pm
Micheal wrote:
Possibly a rhetorical question, but I will answer anyway. First, start your blog two years ago.
It takes a long time to build a discussion board. It can just as long (or longer) to build a blog. For grins I went back and looked at the first page of posts for this blog. It started in May of 2006. There were 9 posts made between then and July of 2006. There were 11 comments made. Of those 11 comments, 8 were from me. 
If you find a pacing that works for you, and can stick with it, then people will get used to going to your blog on a regular basis. I have been guilty of several lapses in posting over the years this blog has been in place but have been able to recover. But I don’t think I ever dropped out for more than five or six weeks straight.
I also make it a point to try to respond to any comment, and most especially to comments from first-time posters. Since I have Wordpress configured to require approval for the first post, it makes it a little easier to identify those folks.
@Everett, I think I was able to reformat your comment to show the code the way you wanted it to be shown. If I did it incorrectly, please let me know.
@dogs and things: some hosts disable the exec() function for security reasons. It’s running an operating system level command, which could literally be just about anything. Do you know if you’re running on a windows or linux host?
Comment by Dave Rathbun — August 29, 2008 @ 1:33 pm
Hi again,
It seems that your edit of Everett’s code did wonders, now I receive echo.
It says: 21:26:52 up 124 days, 2:09, 0 users, load average: 0.18, 0.29, 0.33
Is that good or bad numbers?
Comment by dogs and things — August 29, 2008 @ 2:28 pm
If the exec() is disabled, use back ticks,
so just do this:
echo `uptime`If that doesn’t work, then I don’t know.
Comment by Dog Cow — August 29, 2008 @ 3:34 pm
dogs and things wrote:
It’s good.
It means your server has been up for a while without interuption. The load average of 0.18 is the immediate load, 0.29 is higher and reflects the load over the past 5 minutes, and 0.33 is higher still and reflects the average load over the past 15 minutes. It means your server was busier about fifteen minutes ago but is coming back down now.
On my box, the top command aggregates all of the cpu’s together, so my theoretical “maximum” load would be 4.0 since I have 4 cpu’s and that would be 100% * 4. When I see a load of 0.20 overall it means that each cpu is contributing 0.05 to that number and my server is barely breaking a sweat. If you have a single cpu box a load of 0.33 means that you’re loaded (one third of the cpu is busy) but not overly so. Some would argue that since my server rarely exceeds 2.0 (50% capacity) that I am paying too much for hardware that I don’t need. But on the other hand, I also have hardware that can handle spikes in activity without going to its knees.
Here’s what I got just a few minutes ago for my server:
Comment by Dave Rathbun — August 29, 2008 @ 3:58 pm
dont feel bad, my hosting provider is hosting 810 on my server
Comment by A_O_C — November 21, 2008 @ 4:31 pm
Load is pretty meaningless. On most modern hardware, a high load (>2 per CPU) is at best a poor indicator of possible problems. Usually a spike in load lags far behind the actual problem, which might be a blocked IO channel on a virtual host. A load less than 1 per CPU usually indicates — but not always — that the host is under-utilized.
Look, almost all servers these days (except for scientific computing and Java-based servers) are running stuff that is I/O bound. As soon as a process requests IO, there is a context switch and the possibility for the kernel to run another process. Since CPUs are so fast these days, and what web servers actually do so computationally frivolous, this IO/context-switching can happen 10 to 20 times a second per process. This means you can easily have 10 processes “waiting to run” that will run this second. I’ve seen loads of 30 or 40 while the average CPU idle time was 80%.
Comment by Otheus — November 22, 2011 @ 5:18 am