My MongoDB and PHP Famous 5 Minutes Tutorial

Did you ever wanted to integrate MongoDB monitoring with a third party service?
Did you wanted to integrate MongoDB performance counters with your load stress environment?

This 5 minutes tutorial will help you expose these details to any third party service or application using a simple JSON service, PHP and MongoDB:

1. Install Apache and PHP and php-devel: sudo yum -y install httpd php php-devel php-pear
2. Install PHP MongoDB Driver: sudo pecl install mongo
3. Add extension=mongo.so to you /etc/php.ini
4. Create a short monitoring code as mongo.php and place it at /var/www/html/:

header('Cache-Control: no-cache, must-revalidate');

header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');

header('Content-type: application/json;charset=UTF-8');

class Monitor {

public $VirtualMemory_MB;

public $PageFaults;

public $CurrentConnections;

public $NetworkIO_In_MB;

public $NetworkIO_Out_MB;

public $SpeedIO_Avg_ms;

public $LockRatio_Per;

public $ReadersLocked;

public $WritersLocked;

public $ObjectsInDatabase;

public $DatabaseDataSize_MB;

public $DatabaseIndexSize_MB;

public $IndexMissRatio_Per;

public $CursorsTimedOut;

public $DeleteSelectRatio_Per;

public $InsertSelectRatio_Per;

public $UpdateSelectRatio_Per;

}

if (!isset($_REQUEST['db'])) {

header($_SERVER['SERVER_PROTOCOL'] . ' 500 Internal Server Error', true, 500);

echo json_encode(array('status'=>0,'message'=>'db is missing'));

exit;

}

$databaseName = $_REQUEST['db'];

// connect

$m = new MongoClient();

// select a database

$db = $m->$databaseName;

$ret = $db->execute('db.stats();');

$result = new Monitor;

$result->ObjectsInDatabase = $ret["retval"]["objects"];

$result->DatabaseDataSize_MB = $ret["retval"]["dataSize"];

$result->DatabaseIndexSize_MB = $ret["retval"]["indexSize"];

//var_dump($ret);

$ret = $db->execute('db.serverStatus();');

$result->VirtualMemory_MB = $ret["retval"]["mem"]["virtual"];

$result->PageFaults = $ret["retval"]["extra_info"]["page_faults"];

$result->CurrentConnections = $ret["retval"]["connections"]["current"];

$result->NetworkIO_In_MB = $ret["retval"]["network"]["bytesIn"]/1000000;

$result->NetworkIO_Out_MB = $ret["retval"]["network"]["bytesOut"]/1000000;

$result->SpeedIO_Avg_ms = $ret["retval"]["backgroundFlushing"]["average_ms"];

$result->LockRatio_Per = $ret["retval"]["globalLock"]["lockTime"]/$ret["retval"]["globalLock"]["totalTime"]*100;

$result->ReadersLocked = $ret["retval"]["globalLock"]["currentQueue"]["readers"];

$result->WritersLocked = $ret["retval"]["globalLock"]["currentQueue"]["writers"];

$result->IndexMissRatio_Per = $ret["retval"]["indexCounters"]["btree"]["missRatio"];

$result->CursorsTimedOut = $ret["retval"]["cursors"]["timedOut"];

$result->DeleteSelectRatio_Per = $ret["retval"]["opcounters"]["delete"]/$ret["retval"]["opcounters"]["query"]*100;

$result->InsertSelectRatio_Per = $ret["retval"]["opcounters"]["insert"]/$ret["retval"]["opcounters"]["query"]*100;

$result->UpdateSelectRatio_Per = $ret["retval"]["opcounters"]["update"]/$ret["retval"]["opcounters"]["query"]*100;

echo json_encode($result);

?>

Bottom Line

You MongoDB is ready for monitoring by an external service: http://localhost/mongo.php

Keep Performing,

Moshe Kaplan

mongoDB Insider Tips

This time I would like to share with you several insider tips that can be used to better understand mongoDB and its internals.

Memory Limitation: No, mongoDB does not support it (very similar to SQL Server before adjusting the memory limitations):

Virtual memory and resident sizes may appear to be very large for the mongod process. This is  by design: virtual memory space should be just larger than the size of the data files open and mapped. Resident size will vary depending on the amount of memory not used by other processes on the machine. Yes, if you allocate more memory to the machine, mongoDB will happily consume it,

mongoDB defines its storage size. You can compact it.

Mongo tends to preallocate a significant size of disk.

If your data store is too large you may use the following commands to reduce it:

• db.repairDatabase()
• db.compact()

Use ext4 file system

When ext3 is used and mongoDB allocates new data files, these files have to be filled with zeroes that are written back to the underlying disk (yes, extra unneeded writes to disk).

When ext4 or xfs are used, mongoDB uses the falloc syscall that marks the file as allocated and zeroed, without actually writing anything back to the underlying storage. The result is a better insert performance on ext4 than ext3.

Quorum is for voting, data replication is async by default.

Unlike Cassandra, mongoDB uses Quorum only for primary server selection. Write is done only to primary server and answer is returned to the client immediately. The secondary servers (or some will say the slaves) copy the data in async way.

Synchronous replication is possible but not recommended: Query response to user can be delayed to ensure that data was replicated to slaves. However it may result with 2 orders of magnitude performance decrease: 

mongoDB stores data as BSON and communicates in JSON

BSON or JSON? This is not a question anymore, as mongoDB these days automatically serialize the JSONs into BSONs to save space on disk.

Bottom Line

Now that you better understand the mongoDB architecture, it is time to make more out of it,

Keep Performing,

Moshe Kaplan

Some Best Practices for your Next MySQL Installation with SSD

Do you plan on installing a new MySQL server? 
Did you ask yourself what is the best file system for it? 
What modification are recommended? 
What MySQL version to use?
Do you consider using SSD?

You got to the right place.

Linux, SSD and MySQL. The Best Practices

1. Up to date CentOS (6.3). MySQL RPM are easily available for this platform w/o the need to compile them.
2. ext4 file system. It is has some goodies over ext3 and is even more recommended if you plan to use SSD (and you may need SSD if your system is resource demanding).
3. File system discard option. Recommended for SSD to avoid the need to read before write.
4. Consider some extra cheaper disks. SSD disks are highly expensive and can have a relatively short life if they are not used properly. Consider placing some cheaper disks (SAS or SATA) to handle tasks that do not require the high end SSD disks.
5. Some more recommendations about Linux and SSD from Patrick's:
1. File system layer: remove 'relatime' if present and add 'noatime,nodiratime,discard' to reduce writes and improve performance and disk life expectancy.
2. Scheduler: use the 'deadline' scheduler instead of 'CFQ' to match SSD behavior.
3. Swap: set a small swappiness
4. tmp: move /tmp and /var/tmp to RAM disk to improve performance and avoid unneeded writes to disk.
5. Partition Alignment: when creating a partition, enter a start sector of at least 2,048 and divisible by 512 to align pages.
6. MySQL 5.6. 5.6 if finally in GA and it good idea to start working with it before 5.5 become obsolete.

Bottom Line
Few decisions can enhance your MySQL performance. Make your decisions right.

Keep Performing,
Moshe Kaplan