Scaling wild LAMP for managed hosting
at Web4all
Clement Debiaune
PHP full stack developer
Sometimes DevOp and Sysadmin
Freelancer
Dev, Sysadmin, Volunteer @ Web4all
@EUnibozu
Provides shared hosting, emails (Zimbra) and DNS
Non profit association
100% cost invested in equipment
All volunteers
2006 - NOW()
Why ?
We're Geeks !
Looks really nice on the CV
SCALE WILD WEBSITES !
(they are not ours)
And how to keep it transparent
Outdated CMS - PHP3/4 apps
Drupal memory eater
Softs under development
PHP frameworks
Python Perl
Hackers!
And the nice ones...
One upon a time, in 2006
1 single server @ OVH
(and a backup server)
2006 to 2014
Rule of thumb
Scale progressively
From 10 customers to 3000+
Externalise
MySQL - DNS - Emails - add a NFS Filer
Real scaling starts here
add more Apache servers
keep /tmp + /var/www + PHP sessions on the NFS filer
keep /tmp + /var/www + PHP sessions on the NFS filer
Load Balancer !
HaProxy is great
Virtualize as much as you can
And then it scales !
Need more power ?
Clone an Apache VM
Setup (network, LB)
Setup (network, LB)
Done !
2014
We bought our servers (80k € invested)
Everything is virtualized
VMWare Cluster
138 Ghz
1.25 TB total memory
5 Dell PowerEdge R620
Files
VM on their SAN (ISCSI)
Shared files (apache files) on a Nexenta cluster (ZFS+NFS)
Need more power ? Add an Apache VM
Bottlenecks
Mysql
Easy to scale vertically
ours has 32Gb RAM + 12 vCPU
and handle 12 apaches
Wait, what ? NFS ?
Versatile
Supported everywhere
Stable
Can be used with any FS (we use ZFS)
Faster than you think
Easy to setup
NFS filer available everywhere to rent
Be careful with NFS
Has to rely on a very good network
Kernel side = panic = hard reboot
Be careful where you rent your servers!
Still ; widely used for scaling
PHP sessions ?
In their shared folder over NFS
Performances are not that bad
A little secret
The load balancers
and their STICKY SESSIONS
avoid lock issues
a single session will stay on the same apache
How does it scale ?
per day
12 Apache servers
15 million requests
1.2 million POST requests
8 GB logs to store
250 million queries
4.6 million connections
1 TB traffic
requires specific software to scale horizontally
replication is not so good
100% transparent
100% reliable
No code required
Crashed apache servers taken out by the LB
Snapshots every hour with ZFS
ZFS replication + snapshots for backups
Thanks
Not afraid of French ?
www.web4all.fr
@EUnibozu
Scaling wild LAMP for shared hosting
By Clement Debiaune