Emergent Properties » Optimizing MySQL and Apache for Low Memory Usage, Part 1

Emergent Properties
Order from disorder - Thoughts about where technology is going.
05.10.06
Optimizing MySQL and Apache for Low Memory Usage, Part 1
Posted inGeneral at 2:41 pm by matt
MySQLand Apache can consume quite a bit of memory, if you’re not careful.This post discusses how to reduce the amount of memory they use withoutkilling performance. The caveat, of course, is that you’re not going tobe able to run a site with a large database and large amount of trafficwith these settings. I’m going to try to explain the WHY more than theWHAT. All of this is in conjunction with my goal of reducing the amountof ram I use on my Xen based virtual server, as discussed previouslyin,Low Memory Computing.
Before I begin, I’d like to say that you should also look at varioussystem utilities that consume ram. Services like FTP and SMTP can andshould be passed off to xinetd. Also, you should look at shells besidesbash, such as dash. And, if you’re really serious about low memory, youmight look at using something likeBusyBox, which brings you into the realm of realembedded systems. Personally, I just want to get as much as I can outof a standard linux distribution. If I need more horsepower, I want tobe able to move to bigger, faster virtual machines and/or dedicatedservers. For now, optimizing a small virtual machine will do.
First off, Apache. My first statement is, if you can avoid it, try to.Lighttpd andthttpd are both very good no frills webservers, and you canrun lighttpd with PHP. Even if you’re running a high volume site, youcan seriously gain some performance by passing off static content(images and javascript files, usually) to a lightweight, super-fastHTTPd server such as Lighttpd.
The biggest problem with Apache is the amount of ram is uses. I’lldiscuss the following techniques for speeding up Apache and loweringthe ram used.
Loading Fewer Modules
Handle Fewer Simultaneous Requests
Recycle Apache Processes
Use KeepAlives, but not for too long
Lower your timeout
Log less
Don’t Resolve Hostnames
Don’t use .htaccess
Loading Fewer Modules
First things first, get rid of unnecessary modules. Look throughyour config files and see what modules you might be loading. Are youusing CGI? Perl? If you’re not using modules, by all means, don’t loadthem. That will save you some ram, but the BIGGEST impact is in howApache handles multiple requests.
Handle Fewer Simultaneous Requests
The more processes apache is allowed to run, the more simultaneousrequests it can serve. As you increase that number, you increase theamount of ram that apache will take. Looking at TOP would suggest thateach apache process takes up quite a bit of ram. However, there are alot of shared libraries being used, so you can run some processes, you just can’t run a lot. With Debian 3.1 and Apache2, the following lines are the default:
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 20
MaxRequestsPerChild 0
I haven’t found documentation on this, but prefork.c seems to be themodule that’s loaded to handle things w/ Apache2 and Debian 3.1. Othermechanisms could or could not be much more memory efficient, but I’mnot digging that deep, yet. I’d like to know more, though, so post acomment and let me know. Anyway, the settings that have worked for meare:
StartServers 1
MinSpareServers 1
MaxSpareServers 5
MaxClients 5
MaxRequestsPerChild 300
What I’m basically saying is, “set the maximum amount of requeststhat this server can handle at any one time to 5.” This is pretty low,and I wouldn’t try to do this on a high volume server. However, thereis something you can and should do on your webservers to get the mostout of them, whether you’re going for low memory or not. That is tweakthe keepalive timeout.
Recycle Apache Processes
If you noticed, I changed the MaxRequestsPerChild variable to 500,from 0. This variable tells Apache how many requests a given childprocess can handle before it should be killed. You want to killprocesses, because different page requests will allocate more memory.If a script allocates a lot of memory, the Apache process under whichit runs will allocate that memory, and it won’t let it go. If you’rebumping up against the memory limit of your system, this could causeyou to have unnecessary swapping. Different people use differentsettings here. How to set this is probably a function of the trafficyou receive and the nature of your site. Use your brain on this one.
Use KeepAlives, but not for too long
Keepalives are a way to have apersistent connectionbetween a browser and a server. Originally, HTTP was envisioned asbeing “stateless.” Prior to keepalive, every image, javascript, frame,etc. on your pages had to be requested using a separate connection tothe server. When keepalives came into wide use with HTTP/1.1, webbrowsers were able to keep a connection to a server open, in order totransfer multiple files across that same connection. Fewer connections,less overhead, more performance. There’s one thing wrong, though.Apache, by default, keeps the connections open for a bit too long. Thedefault seems to be 15 seconds, but you can get by easily with 2 or 3seconds.
This is saying, “when a browser stops requesting files, wait for Xseconds before terminating the connection.” If you’re on a decentconnection, 3 seconds is more than enough time to wait for the browserto make additional requests. The only reason I can think of for settinga higher KeepAliveTimeout is to keep a connection open for the NEXTpage request. That is, user downloads page, renders completely, clicksanother link. A timeout of 15 would be appropriate for a site that haspeople clicking from page to page, very often. If you’re running a lowvolume site where people click, read, click, etc., you probably don’thave this. You’re essentially taking 1 or more apache processes andsaying, “for the next 15 seconds, don’t listen to anyone but this oneguy, who may or may not actually ask for anything.” The server isoptimizing one case at the expense of all the other people who arehopefully hitting your site.
Lower Your Timeout
Also, just in case, since you’re limiting the number of processes,you don’t want one to be “stuck” timing out for too long, so i suggestyou lower your “normal” Timeout variable as well.
Log Less
If you’re trying to maximize performance, you can definitely log less.Modules such as Mod_Rewrite will log debugging info. If you don’t needthe debugging info, get rid of it. The Rewrite log is set with the RewriteLog command. Also, if you don’t care about looking at certain statistics, you can choose to not log certain things, like the User-Agent or the Http-Referer. I like seeing those things, but it’s up to you.
Don’t Resolve Hostnames
This one’s easy. Don’t do reverse lookups inside Apache. I can’tthink of a good reason to do it. Any self respecting log parser can dothis offline, in the background.
HostnameLookups Off
Don’t Use .htaccess
You’ve probably seen the AllowOverride None command. Thissays, “don’t look for .htaccess files” Using .htaccess will causeApache to 1) look for files frequently and 2) parse the .htaccess filefor each request. If you need per-directory changes, make the changesinside your main Apache configuration file, not in .htaccess.
Well, that’s it for Part 1, I’ll be back soon with Part 2, where I’lltalk about MySQL optimization & possibly a few other things thatcrop up.
Credits:
I’d like to give credit to a few articles that were helpful inputting this information together.  I’m not a master at this, I’m justtrying to compile lots of bits and pieces together into one place. Thanks go to:
Virtual Threads - Tuning Apache
codepoetry - Optimizing a VPS for Getting Dugg
Debian Administration