HAProxy Architecture Guide

-------------------HAProxyArchitecture Guide-------------------version 1.2.18willy tarreau2008/05/25This document provides real world examples with working configurations.Please note that except stated otherwise, global configuration parameterssuch as logging, chrooting, limits and time-outs are not described here.===================================================1. Simple HTTP load-balancing with cookie insertion===================================================A web application often saturates the front-end server with high CPU loads,due to the scripting language involved. It also relies on a back-end databasewhich is not much loaded. User contexts are stored on the server itself, andnot in the database, so that simply adding another server with simple IP/TCPload-balancing would not work.+-------+|clients| clients and/or reverse-proxy+---+---+|-+-----+--------+----| _|_db+--+--+ (___)| web | (___)+-----+ (___)192.168.1.1 192.168.1.2Replacing the web server with a bigger SMP system would cost much more thanadding low-cost pizza boxes. The solution is to buy N cheap boxes and installthe application on them. Install haproxy on the old one which will spread theload across the new boxes.192.168.1.1 192.168.1.11-192.168.1.14 192.168.1.2-------+-----------+-----+-----+-----+--------+----| | | | | _|_db+--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)| LB1 | | A | | B | | C | | D | (___)+-----+ +---+ +---+ +---+ +---+ (___)haproxy 4 cheap web serversConfig on haproxy (LB1) :-------------------------listen webfarm 192.168.1.1:80mode httpbalance roundrobincookie SERVERID insert indirectoption httpchk HEAD /index.html HTTP/1.0server webA 192.168.1.11:80 cookie A checkserver webB 192.168.1.12:80 cookie B checkserver webC 192.168.1.13:80 cookie C checkserver webD 192.168.1.14:80 cookie D checkDescription :-------------- LB1 will receive clients requests.- if a request does not contain a cookie, it will be forwarded to a validserver- in return, a cookie "SERVERID" will be inserted in the response holding theserver name (eg: "A").- when the client comes again with the cookie "SERVERID=A", LB1 will know thatit must be forwarded to server A. The cookie will be removed so that theserver does not see it.- if server "webA" dies, the requests will be sent to another valid serverand a cookie will be reassigned.Flows :-------(client) (haproxy) (server A)>-- GET /URI1 HTTP/1.0 ------------> |( no cookie, haproxy forwards in load-balancing mode. )| >-- GET /URI1 HTTP/1.0 ---------->| <-- HTTP/1.0 200 OK -------------<( the proxy now adds the server cookie in return )<-- HTTP/1.0 200 OK ---------------< |Set-Cookie: SERVERID=A |>-- GET /URI2 HTTP/1.0 ------------> |Cookie: SERVERID=A |( the proxy sees the cookie. it forwards to server A and deletes it )| >-- GET /URI2 HTTP/1.0 ---------->| <-- HTTP/1.0 200 OK -------------<( the proxy does not add the cookie in return because the client knows it )<-- HTTP/1.0 200 OK ---------------< |>-- GET /URI3 HTTP/1.0 ------------> |Cookie: SERVERID=A |( ... )Limits :--------- if clients use keep-alive (HTTP/1.1), only the first response will havea cookie inserted, and only the first request of each session will beanalyzed. This does not cause trouble in insertion mode because the cookieis put immediately in the first response, and the session is maintained tothe same server for all subsequent requests in the same session. However,the cookie will not be removed from the requests forwarded to the servers,so the server must not be sensitive to unknown cookies. If this causestrouble, you can disable keep-alive by adding the following option :option httpclose- if for some reason the clients cannot learn more than one cookie (eg: theclients are indeed some home-made applications or gateways), and theapplication already produces a cookie, you can use the "prefix" mode (seebelow).- LB1 becomes a very sensible server. If LB1 dies, nothing works anymore.=> you can back it up using keepalived (see below)- if the application needs to log the original client's IP, use the"forwardfor" option which will add an "X-Forwarded-For" header with theoriginal client's IP address. You must also use "httpclose" to ensurethat you will rewrite every requests and not only the first one of eachsession :option httpcloseoption forwardforThe web server will have to be configured to use this header instead.For example, on apache, you can use LogFormat for this :LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b " combinedCustomLog /var/log/httpd/access_log combinedHints :-------Sometimes on the internet, you will find a few percent of the clients whichdisable cookies on their browser. Obviously they have troubles everywhere onthe web, but you can still help them access your site by using the "source"balancing algorithm instead of the "roundrobin". It ensures that a given IPaddress always reaches the same server as long as the number of servers remainsunchanged. Never use this behind a proxy or in a small network, because thedistribution will be unfair. However, in large internal networks, and on theinternet, it works quite well. Clients which have a dynamic address will notbe affected as long as they accept the cookie, because the cookie always hasprecedence over load balancing :listen webfarm 192.168.1.1:80mode httpbalance sourcecookie SERVERID insert indirectoption httpchk HEAD /index.html HTTP/1.0server webA 192.168.1.11:80 cookie A checkserver webB 192.168.1.12:80 cookie B checkserver webC 192.168.1.13:80 cookie C checkserver webD 192.168.1.14:80 cookie D check==================================================================2. HTTP load-balancing with cookie prefixing and high availability==================================================================Now you don't want to add more cookies, but rather use existing ones. Theapplication already generates a "JSESSIONID" cookie which is enough to tracksessions, so we'll prefix this cookie with the server name when we see it.Since the load-balancer becomes critical, it will be backed up with a secondone in VRRP mode using keepalived under Linux.Download the latest version of keepalived from this site and install iton each load-balancer LB1 and LB2 :http://www.keepalived.org/You then have a shared IP between the two load-balancers (we will still use theoriginal IP). It is active only on one of them at any moment. To allow theproxy to bind to the shared IP on Linux 2.4, you must enable it in /proc :# echo 1 >/proc/sys/net/ipv4/ip_nonlocal_bindshared IP=192.168.1.1192.168.1.3 192.168.1.4 192.168.1.11-192.168.1.14 192.168.1.2-------+------------+-----------+-----+-----+-----+--------+----| | | | | | _|_db+--+--+ +--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)| LB1 | | LB2 | | A | | B | | C | | D | (___)+-----+ +-----+ +---+ +---+ +---+ +---+ (___)haproxy haproxy 4 cheap web serverskeepalived keepalivedConfig on both proxies (LB1 and LB2) :--------------------------------------listen webfarm 192.168.1.1:80mode httpbalance roundrobincookie JSESSIONID prefixoption httpcloseoption forwardforoption httpchk HEAD /index.html HTTP/1.0server webA 192.168.1.11:80 cookie A checkserver webB 192.168.1.12:80 cookie B checkserver webC 192.168.1.13:80 cookie C checkserver webD 192.168.1.14:80 cookie D checkNotes: the proxy will modify EVERY cookie sent by the client and the server,so it is important that it can access to ALL cookies in ALL requests foreach session. This implies that there is no keep-alive (HTTP/1.1), thus the"httpclose" option. Only if you know for sure that the client(s) will neveruse keep-alive (eg: Apache 1.3 in reverse-proxy mode), you can remove thisoption.Configuration for keepalived on LB1/LB2 :-----------------------------------------vrrp_script chk_haproxy { # Requires keepalived-1.1.13script "killall -0 haproxy" # cheaper than pidofinterval 2 # check every 2 secondsweight 2 # add 2 points of prio if OK}vrrp_instance VI_1 {interface eth0state MASTERvirtual_router_id 51priority 101 # 101 on master, 100 on backupvirtual_ipaddress {192.168.1.1}track_script {chk_haproxy}}Description :-------------- LB1 is VRRP master (keepalived), LB2 is backup. Both monitor the haproxyprocess, and lower their prio if it fails, leading to a failover to theother node.- LB1 will receive clients requests on IP 192.168.1.1.- both load-balancers send their checks from their native IP.- if a request does not contain a cookie, it will be forwarded to a validserver- in return, if a JESSIONID cookie is seen, the server name will be prefixedinto it, followed by a delimitor ('~')- when the client comes again with the cookie "JSESSIONID=A~xxx", LB1 willknow that it must be forwarded to server A. The server name will then beextracted from cookie before it is sent to the server.- if server "webA" dies, the requests will be sent to another valid serverand a cookie will be reassigned.Flows :-------(client) (haproxy) (server A)>-- GET /URI1 HTTP/1.0 ------------> |( no cookie, haproxy forwards in load-balancing mode. )| >-- GET /URI1 HTTP/1.0 ---------->| X-Forwarded-For: 10.1.2.3| <-- HTTP/1.0 200 OK -------------<( no cookie, nothing changed )<-- HTTP/1.0 200 OK ---------------< |>-- GET /URI2 HTTP/1.0 ------------> |( no cookie, haproxy forwards in lb mode, possibly to another server. )| >-- GET /URI2 HTTP/1.0 ---------->| X-Forwarded-For: 10.1.2.3| <-- HTTP/1.0 200 OK -------------<| Set-Cookie: JSESSIONID=123( the cookie is identified, it will be prefixed with the server name )<-- HTTP/1.0 200 OK ---------------< |Set-Cookie: JSESSIONID=A~123 |>-- GET /URI3 HTTP/1.0 ------------> |Cookie: JSESSIONID=A~123 |( the proxy sees the cookie, removes the server name and forwardsto server A which sees the same cookie as it previously sent )| >-- GET /URI3 HTTP/1.0 ---------->| Cookie: JSESSIONID=123| X-Forwarded-For: 10.1.2.3| <-- HTTP/1.0 200 OK -------------<( no cookie, nothing changed )<-- HTTP/1.0 200 OK ---------------< |( ... )Hints :-------Sometimes, there will be some powerful servers in the farm, and some smallerones. In this situation, it may be desirable to tell haproxy to respect thedifference in performance. Let's consider that WebA and WebB are two oldP3-1.2 GHz while WebC and WebD are shiny new Opteron-2.6 GHz. If yourapplication scales with CPU, you may assume a very rough 2.6/1.2 performanceratio between the servers. You can inform haproxy about this using the "weight"keyword, with values between 1 and 256. It will then spread the load the mostsmoothly possible respecting those ratios :server webA 192.168.1.11:80 cookie A weight 12 checkserver webB 192.168.1.12:80 cookie B weight 12 checkserver webC 192.168.1.13:80 cookie C weight 26 checkserver webD 192.168.1.14:80 cookie D weight 26 check========================================================2.1 Variations involving external layer 4 load-balancers========================================================Instead of using a VRRP-based active/backup solution for the proxies,they can also be load-balanced by a layer4 load-balancer (eg: Alteon)which will also check that the services run fine on both proxies :| VIP=192.168.1.1+----+----+| Alteon |+----+----+|192.168.1.3 | 192.168.1.4 192.168.1.11-192.168.1.14 192.168.1.2-------+-----+------+-----------+-----+-----+-----+--------+----| | | | | | _|_db+--+--+ +--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)| LB1 | | LB2 | | A | | B | | C | | D | (___)+-----+ +-----+ +---+ +---+ +---+ +---+ (___)haproxy haproxy 4 cheap web serversConfig on both proxies (LB1 and LB2) :--------------------------------------listen webfarm 0.0.0.0:80mode httpbalance roundrobincookie JSESSIONID prefixoption httpcloseoption forwardforoption httplogoption dontlognulloption httpchk HEAD /index.html HTTP/1.0server webA 192.168.1.11:80 cookie A checkserver webB 192.168.1.12:80 cookie B checkserver webC 192.168.1.13:80 cookie C checkserver webD 192.168.1.14:80 cookie D checkThe "dontlognull" option is used to prevent the proxy from logging the healthchecks from the Alteon. If a session exchanges no data, then it will not belogged.Config on the Alteon :----------------------/c/slb/real 11enaname "LB1"rip 192.168.1.3/c/slb/real 12enaname "LB2"rip 192.168.1.4/c/slb/group 10name "LB1-2"metric roundrobinhealth tcpadd 11add 12/c/slb/virt 10enavip 192.168.1.1/c/slb/virt 10/service httpgroup 10Note: the health-check on the Alteon is set to "tcp" to prevent the proxy fromforwarding the connections. It can also be set to "http", but for this theproxy must specify a "monitor-net" with the Alteons' addresses, so that theAlteon can really check that the proxies can talk HTTP but without forwardingthe connections to the end servers. Check next section for an example on how touse monitor-net.============================================================2.2 Generic TCP relaying and external layer 4 load-balancers============================================================Sometimes it's useful to be able to relay generic TCP protocols (SMTP, TSE,VNC, etc...), for example to interconnect private networks. The problem comeswhen you use external load-balancers which need to send periodic health-checksto the proxies, because these health-checks get forwarded to the end servers.The solution is to specify a network which will be dedicated to monitoringsystems and must not lead to a forwarding connection nor to any log, using the"monitor-net" keyword. Note: this feature expects a version of haproxy greaterthan or equal to 1.1.32 or 1.2.6.| VIP=172.16.1.1 |+----+----+ +----+----+| Alteon1 | | Alteon2 |+----+----+ +----+----+192.168.1.252 | GW=192.168.1.254 | 192.168.1.253| |------+---+------------+--+-----------------> TSE farm : 192.168.1.10192.168.1.1 | | 192.168.1.2+--+--+ +--+--+| LB1 | | LB2 |+-----+ +-----+haproxy haproxyConfig on both proxies (LB1 and LB2) :--------------------------------------listen tse-proxybind :3389,:1494,:5900 # TSE, ICA and VNC at once.mode tcpbalance roundrobinserver tse-farm 192.168.1.10monitor-net 192.168.1.252/31The "monitor-net" option instructs the proxies that any connection coming from192.168.1.252 or 192.168.1.253 will not be logged nor forwarded and will beclosed immediately. The Alteon load-balancers will then see the proxies alivewithout perturbating the service.Config on the Alteon :----------------------/c/l3/if 1enaaddr 192.168.1.252mask 255.255.255.0/c/slb/real 11enaname "LB1"rip 192.168.1.1/c/slb/real 12enaname "LB2"rip 192.168.1.2/c/slb/group 10name "LB1-2"metric roundrobinhealth tcpadd 11add 12/c/slb/virt 10enavip 172.16.1.1/c/slb/virt 10/service 1494group 10/c/slb/virt 10/service 3389group 10/c/slb/virt 10/service 5900group 10Special handling of SSL :-------------------------Sometimes, you want to send health-checks to remote systems, even in TCP mode,in order to be able to failover to a backup server in case the first one isdead. Of course, you can simply enable TCP health-checks, but it sometimeshappens that intermediate firewalls between the proxies and the remote serversacknowledge the TCP connection themselves, showing an always-up server. Sincethis is generally encountered on long-distance communications, which ofteninvolve SSL, an SSL health-check has been implemented to workaround this issue.It sends SSL Hello messages to the remote server, which in turns replies withSSL Hello messages. Setting it up is very easy :listen tcp-syslog-proxybind :1514 # listen to TCP syslog traffic on this port (SSL)mode tcpbalance roundrobinoption ssl-hello-chkserver syslog-prod-site 192.168.1.10 checkserver syslog-back-site 192.168.2.10 check backup=========================================================3. Simple HTTP/HTTPS load-balancing with cookie insertion=========================================================This is the same context as in example 1 above, but the webserver uses HTTPS.+-------+|clients| clients+---+---+|-+-----+--------+----| _|_db+--+--+ (___)| SSL | (___)| web | (___)+-----+192.168.1.1 192.168.1.2Since haproxy does not handle SSL, this part will have to be extracted from theservers (freeing even more ressources) and installed on the load-balanceritself. Install haproxy and apache+mod_ssl on the old box which will spread theload between the new boxes. Apache will work in SSL reverse-proxy-cache. If theapplication is correctly developped, it might even lower its load. However,since there now is a cache between the clients and haproxy, some securitymeasures must be taken to ensure that inserted cookies will not be cached.192.168.1.1 192.168.1.11-192.168.1.14 192.168.1.2-------+-----------+-----+-----+-----+--------+----| | | | | _|_db+--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)| LB1 | | A | | B | | C | | D | (___)+-----+ +---+ +---+ +---+ +---+ (___)apache 4 cheap web serversmod_sslhaproxyConfig on haproxy (LB1) :-------------------------listen 127.0.0.1:8000mode httpbalance roundrobincookie SERVERID insert indirect nocacheoption httpchk HEAD /index.html HTTP/1.0server webA 192.168.1.11:80 cookie A checkserver webB 192.168.1.12:80 cookie B checkserver webC 192.168.1.13:80 cookie C checkserver webD 192.168.1.14:80 cookie D checkDescription :-------------- apache on LB1 will receive clients requests on port 443- it forwards it to haproxy bound to 127.0.0.1:8000- if a request does not contain a cookie, it will be forwarded to a validserver- in return, a cookie "SERVERID" will be inserted in the response holding theserver name (eg: "A"), and a "Cache-control: private" header will be addedso that the apache does not cache any page containing such cookie.- when the client comes again with the cookie "SERVERID=A", LB1 will know thatit must be forwarded to server A. The cookie will be removed so that theserver does not see it.- if server "webA" dies, the requests will be sent to another valid serverand a cookie will be reassigned.Notes :-------- if the cookie works in "prefix" mode, there is no need to add the "nocache"option because it is an application cookie which will be modified, and theapplication flags will be preserved.- if apache 1.3 is used as a front-end before haproxy, it always disablesHTTP keep-alive on the back-end, so there is no need for the "httpclose"option on haproxy.- configure apache to set the X-Forwarded-For header itself, and do not doit on haproxy if you need the application to know about the client's IP.Flows :-------(apache) (haproxy) (server A)>-- GET /URI1 HTTP/1.0 ------------> |( no cookie, haproxy forwards in load-balancing mode. )| >-- GET /URI1 HTTP/1.0 ---------->| <-- HTTP/1.0 200 OK -------------<( the proxy now adds the server cookie in return )<-- HTTP/1.0 200 OK ---------------< |Set-Cookie: SERVERID=A |Cache-Control: private |>-- GET /URI2 HTTP/1.0 ------------> |Cookie: SERVERID=A |( the proxy sees the cookie. it forwards to server A and deletes it )| >-- GET /URI2 HTTP/1.0 ---------->| <-- HTTP/1.0 200 OK -------------<( the proxy does not add the cookie in return because the client knows it )<-- HTTP/1.0 200 OK ---------------< |>-- GET /URI3 HTTP/1.0 ------------> |Cookie: SERVERID=A |( ... )========================================3.1. Alternate solution using Stunnel========================================When only SSL is required and cache is not needed, stunnel is a cheapersolution than Apache+mod_ssl. By default, stunnel does not process HTTP anddoes not add any X-Forwarded-For header, but there is a patch on the officialhaproxy site to provide this feature to recent stunnel versions.This time, stunnel will only process HTTPS and not HTTP. This means thathaproxy will get all HTTP traffic, so haproxy will have to add theX-Forwarded-For header for HTTP traffic, but not for HTTPS traffic sincestunnel will already have done it. We will use the "except" keyword to tellhaproxy that connections from local host already have a valid header.192.168.1.1 192.168.1.11-192.168.1.14 192.168.1.2-------+-----------+-----+-----+-----+--------+----| | | | | _|_db+--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)| LB1 | | A | | B | | C | | D | (___)+-----+ +---+ +---+ +---+ +---+ (___)stunnel 4 cheap web servershaproxyConfig on stunnel (LB1) :-------------------------cert=/etc/stunnel/stunnel.pemsetuid=stunnelsetgid=proxysocket=l:TCP_NODELAY=1socket=r:TCP_NODELAY=1[https]accept=192.168.1.1:443connect=192.168.1.1:80xforwardedfor=yesConfig on haproxy (LB1) :-------------------------listen 192.168.1.1:80mode httpbalance roundrobinoption forwardfor except 192.168.1.1cookie SERVERID insert indirect nocacheoption httpchk HEAD /index.html HTTP/1.0server webA 192.168.1.11:80 cookie A checkserver webB 192.168.1.12:80 cookie B checkserver webC 192.168.1.13:80 cookie C checkserver webD 192.168.1.14:80 cookie D checkDescription :-------------- stunnel on LB1 will receive clients requests on port 443- it forwards them to haproxy bound to port 80- haproxy will receive HTTP client requests on port 80 and decrypted SSLrequests from Stunnel on the same port.- stunnel will add the X-Forwarded-For header- haproxy will add the X-Forwarded-For header for everyone except the localaddress (stunnel).========================================4. Soft-stop for application maintenance========================================When an application is spread across several servers, the time to update allinstances increases, so the application seems jerky for a longer period.HAproxy offers several solutions for this. Although it cannot be reconfiguredwithout being stopped, nor does it offer any external command, there are otherworking solutions.=========================================4.1 Soft-stop using a file on the servers=========================================This trick is quite common and very simple: put a file on the server which willbe checked by the proxy. When you want to stop the server, first remove thisfile. The proxy will see the server as failed, and will not send it any newsession, only the old ones if the "persist" option is used. Wait a bit thenstop the server when it does not receive anymore connections.listen 192.168.1.1:80mode httpbalance roundrobincookie SERVERID insert indirectoption httpchk HEAD /running HTTP/1.0server webA 192.168.1.11:80 cookie A check inter 2000 rise 2 fall 2server webB 192.168.1.12:80 cookie B check inter 2000 rise 2 fall 2server webC 192.168.1.13:80 cookie C check inter 2000 rise 2 fall 2server webD 192.168.1.14:80 cookie D check inter 2000 rise 2 fall 2option persistredispatchcontimeout 5000Description :-------------- every 2 seconds, haproxy will try to access the file "/running" on theservers, and declare the server as down after 2 attempts (4 seconds).- only the servers which respond with a 200 or 3XX response will be used.- if a request does not contain a cookie, it will be forwarded to a validserver- if a request contains a cookie for a failed server, haproxy will insiston trying to reach the server anyway, to let the user finish what he wasdoing. ("persist" option)- if the server is totally stopped, the connection will fail and the proxywill rebalance the client to another server ("redispatch")Usage on the web servers :--------------------------- to start the server :# /etc/init.d/httpd start# touch /home/httpd/www/running- to soft-stop the server# rm -f /home/httpd/www/running- to completely stop the server :# /etc/init.d/httpd stopLimits------If the server is totally powered down, the proxy will still try to reach itfor those clients who still have a cookie referencing it, and the connectionattempt will expire after 5 seconds ("contimeout"), and only after that, theclient will be redispatched to another server. So this mode is only usefulfor software updates where the server will suddenly refuse the connectionbecause the process is stopped. The problem is the same if the server suddenlycrashes. All of its users will be fairly perturbated.==================================4.2 Soft-stop using backup servers==================================A better solution which covers every situation is to use backup servers.Version 1.1.30 fixed a bug which prevented a backup server from sharingthe same cookie as a standard server.listen 192.168.1.1:80mode httpbalance roundrobinredispatchcookie SERVERID insert indirectoption httpchk HEAD / HTTP/1.0server webA 192.168.1.11:80 cookie A check port 81 inter 2000server webB 192.168.1.12:80 cookie B check port 81 inter 2000server webC 192.168.1.13:80 cookie C check port 81 inter 2000server webD 192.168.1.14:80 cookie D check port 81 inter 2000server bkpA 192.168.1.11:80 cookie A check port 80 inter 2000 backupserver bkpB 192.168.1.12:80 cookie B check port 80 inter 2000 backupserver bkpC 192.168.1.13:80 cookie C check port 80 inter 2000 backupserver bkpD 192.168.1.14:80 cookie D check port 80 inter 2000 backupDescription-----------Four servers webA..D are checked on their port 81 every 2 seconds. The sameservers named bkpA..D are checked on the port 80, and share the exact samecookies. Those servers will only be used when no other server is availablefor the same cookie.When the web servers are started, only the backup servers are seen asavailable. On the web servers, you need to redirect port 81 to localport 80, either with a local proxy (eg: a simple haproxy tcp instance),or with iptables (linux) or pf (openbsd). This is because we want thereal web server to reply on this port, and not a fake one. Eg, withiptables :# /etc/init.d/httpd start# iptables -t nat -A PREROUTING -p tcp --dport 81 -j REDIRECT --to-port 80A few seconds later, the standard server is seen up and haproxy starts to sendit new requests on its real port 80 (only new users with no cookie, of course).If a server completely crashes (even if it does not respond at the IP level),both the standard and backup servers will fail, so clients associated to thisserver will be redispatched to other live servers and will lose their sessions.Now if you want to enter a server into maintenance, simply stop it fromresponding on port 81 so that its standard instance will be seen as failed,but the backup will still work. Users will not notice anything since theservice is still operational :# iptables -t nat -D PREROUTING -p tcp --dport 81 -j REDIRECT --to-port 80The health checks on port 81 for this server will quickly fail, and thestandard server will be seen as failed. No new session will be sent to thisserver, and existing clients with a valid cookie will still reach it becausethe backup server will still be up.Now wait as long as you want for the old users to stop using the service, andonce you see that the server does not receive any traffic, simply stop it :# /etc/init.d/httpd stopThe associated backup server will in turn fail, and if any client still triesto access this particular server, he will be redispatched to any other validserver because of the "redispatch" option.This method has an advantage : you never touch the proxy when doing servermaintenance. The people managing the servers can make them disappear smoothly.4.2.1 Variations for operating systems without any firewall software--------------------------------------------------------------------The downside is that you need a redirection solution on the server just forthe health-checks. If the server OS does not support any firewall software,this redirection can also be handled by a simple haproxy in tcp mode :globaldaemonquietpidfile /var/run/haproxy-checks.pidlisten 0.0.0.0:81mode tcpdispatch 127.0.0.1:80contimeout 1000clitimeout 10000srvtimeout 10000To start the web service :# /etc/init.d/httpd start# haproxy -f /etc/haproxy/haproxy-checks.cfgTo soft-stop the service :# kill $(