119 lines
4.8 KiB
Plaintext
119 lines
4.8 KiB
Plaintext
|
|
Opencaching MySQL Master/slave replication
|
|
|
|
1. Overview
|
|
|
|
You can use MySQL master/slave-replication to execute readonly-queries
|
|
on slave servers. This is especially usefull because the most expensive
|
|
queries are search quries that do not need to modify tables (expect temporary
|
|
tables). Beside the server load the execution of long running queries on
|
|
the slave reduces locking issues experienced with higher load scenarios.
|
|
|
|
Beware that a real load balancing is only achieved when you have at least
|
|
a total of 3 MySQL servers available. One server is the master where fast
|
|
select quries and all insert/update quries are executed. The 2nd and 3rd server
|
|
(the first slave) executes all search queries. The search queries are balanced
|
|
between this 2 slaves by a custom relative weight.
|
|
|
|
Because most search-queries in search.php require the generation of temporary
|
|
tables, these queries are logged in the binary log. Therefore if a search query
|
|
is executed on the master, all slaves would execute this query again, which would
|
|
drop down the overall performance.
|
|
|
|
If you have only 2 physical servers available and the master has additional resource
|
|
available, you can setup a second mysql server on that physical server on a different
|
|
tcp port.
|
|
|
|
2. Time lack between master and slave
|
|
|
|
On fast networks and medium server loads, the time difference required to update
|
|
the slaves is nearly 0 seconds. With increasing load or other technical problems like
|
|
network interupts the time lack grows. The OC source code detects this situation
|
|
and does only redirect queries to slaves that are within a defined time difference.
|
|
|
|
However, you have to take care about what queries exactly will be redirected to the
|
|
slaves. If a user generates a new log entry and is redirect to viewcache.php, after
|
|
that short time difference, the new log is probably not replicated to the slaves.
|
|
Therefore the logs in viewcache.php should be read from the master.
|
|
|
|
TODO: Store in cookie, when the last critical write operation has been done by the
|
|
user and disable slave redirection for this user, until all slaves are up2date.
|
|
=> can be done by calling sql_slave_exclude()
|
|
|
|
3. Configuration and scripts to use replication
|
|
|
|
3.1 settings.inc.php
|
|
|
|
You need to configure each slave database in the configuration see settings-dist.inc.php
|
|
for available options.
|
|
|
|
3.2 util2/replication_monitor/replication_monitor.sh
|
|
|
|
This script updates a timestamp in the database every 10 seconds.
|
|
This timestamp will be replicated to each slave and can therefore
|
|
detect in a reliable way what time lack is in the binary log.
|
|
|
|
See remarks in the script how to install and use it.
|
|
|
|
3.3 util2/cron/modules/replication_monitor.class.php
|
|
|
|
util2/cron/runcron.php should invoked once a minute. It will check if all slaves
|
|
are reachable and what time lack they have. It will also check at what
|
|
position the binary log is on the slave servers (see next section).
|
|
|
|
The DB user requires privilege "REPLICATION CLIENT". If you require SSL
|
|
replication, this may be a security risk.
|
|
|
|
3.4 util2/replication_monitor/purge_master_logs.php
|
|
|
|
This script purges old binary logs that are no more required by any slave.
|
|
|
|
See remarks in the script how to install and use it.
|
|
|
|
3.5 Excludes
|
|
|
|
Add to the my.cnf of the slave servers the following configuration:
|
|
|
|
replicate-ignore-table=ocde.sys_temptables
|
|
replicate-ignore-table=ocde.mapresult
|
|
replicate-ignore-table=ocde.mapresult_data
|
|
replicate-ignore-table=ocde.map2_data
|
|
|
|
4. Fallback
|
|
|
|
If no slave is online, the code will fallback to the master.
|
|
However, because temporary tables are logged in the binary log,
|
|
the resync of the slaves will consume more time than without
|
|
a fallback.
|
|
|
|
5. Remarks
|
|
|
|
5.1 Relative weights in settings.inc.php
|
|
|
|
Relative weight might be difficult to set up. When determining
|
|
what slave to use, the following query is issued (with filter
|
|
of slaves that are not up to date):
|
|
|
|
SELECT RAND()*weight AS rand_order, id, server
|
|
FROM `sys_dbslaves` ORDER BY rand_order DESC LIMIT 1
|
|
|
|
The follwing weights for 2 or 3 slaves will result in
|
|
the follwing loads:
|
|
|
|
Weight 1 Weight 2 Weight 3 Load 1 Load 2 Load 3
|
|
100 100 50% 50%
|
|
100 75 63% 37%
|
|
100 50 75% 25%
|
|
100 25 87% 13%
|
|
100 100 100 33% 33% 33%
|
|
100 75 75 50% 25% 25%
|
|
100 50 50 66% 17% 17%
|
|
|
|
5.2 Scripts using slave servers
|
|
|
|
newcaches.php
|
|
newcachesrest.php
|
|
newlogs.php
|
|
search.php
|
|
tops.php
|