DZone's Definitive Guide to Cloud Providers is here!

I was a part of the team that was contributing to the DZone's  Definitive Guide to Cloud Providers, and it's finally here!
If you consider what cloud solution is best for you, take a look at this guide.

Keep Performing,
Moshe Kaplan

How Much RAM Should You Have for MongoDB?

Why RAM is So Important?
RAM is much faster than hard disks. For example, we gained X7 performance boost when we used MySQL In Memory engine rather than InnoDB engine.

Is It Only For Query Caching?
Most of us are familiar with query caching, when results of a query are saved in cache in case it will be called again.
However, RAM can be used to store a copy of your database in the RAM as well. This enables you getting best performance even when you do queries that were not done before. 

So How Much RAM Do I Need?
My Recommendation that your system RAM should be larger than MongoDB data files + index files: SizeOf(RAM) > SizeOf(Data Files) + SizeOf(Index Files).
That way, your files will always be in memory and you will avoid swapping.

What If My Data is Too Big?
In this case you should choose one of the following strategies:

1. Shard your MongoDB between several servers.
2. Tune your queries (this is a good idea in general).
3. Design your databases and collections to support "Data Hierarchy".
4. Choose SSD as your hard disk solution.
5. Consider vSMP Foundation for Memory Expansion from ScaleMP to increase RAM up to 256TB as our readers suggested (P.S don't forget them to ask for the special price :-)

Bottom Line
Careful design will do miracles to your production environment.

Keep Performing,
Moshe Kaplan

mongoDB Performance Tuning (Full Presentation from Israel MongoDB User Group)

I gave a lecture today at the Israel MongoDB User Group regarding MongoDB performance tuning and how scale with it.
It was a great event that was organized by Wix, 10gen and Trainologic.

Did you miss the event? Well, don't miss the lesson. You can find below the full presentation:

mongoDB Performance from Moshe Kaplan

Bottom Line
Boost your MongoDB...

Keep Performing,
Moshe Kaplan

MongoDB, Users and Permissions

NoSQL and Enterprise Security?
That is not the first thing that comes to mind when you consider using NoSQL. It is not a big surprise as the early adapters of NoSQL were Internet companies.
An evident for that you can find in MongoDB, where authentication is dimmed by default.

How to Enable MongoDB Authentication?

1. Create an Admin user (otherwise you will have issues to connect your server) from the local console:
1. use admin;
2. db.addUser({ user: "", pwd: "", roles: [ "userAdminAnyDatabase" ]})
2. Enable authentication in the /etc/mongo.conf: auth=true
3. Restart the mongod instance to enable authentication.

How to Add Additional users?

Select the database that you want to add user to:

use

db.addUser( { user: "", pwd: "", roles: [ "", ""]})

And select the a user role from the following permissions list:

1. read
2. readWrite
3. dbAdmin
4. userAdmin
5. clusterAdmin
6. readAnyDatabase
7. readWriteAnyDatabase
8. userAdminAnyDatabase
9. dbAdminAnyDatabase

How to Provide Permissions to Other Databases?

This one is done with a "copy" like method, where userSource defines the database that the user definition should be copied from:

use

db.addUser( { user: "", userSource: "", roles: [ "" ] } )

In case you want to provide read permissions to all databases you may use the readAnyDatabases

Bottom Line

Not very complex, but more secure. 

Keep Performing,

Moshe Kaplan

Lastest PHP 5.4 with MySQL via yum?

This is probably the link you are looking for: http://www.webtatic.com/packages/php54/
This one will save you a lot of hours dealing with old PHP 5.3 and newest MySQL 5.6 incompatibilities.

My MongoDB and PHP Famous 5 Minutes Tutorial

Did you ever wanted to integrate MongoDB monitoring with a third party service?
Did you wanted to integrate MongoDB performance counters with your load stress environment?

This 5 minutes tutorial will help you expose these details to any third party service or application using a simple JSON service, PHP and MongoDB:

1. Install Apache and PHP and php-devel: sudo yum -y install httpd php php-devel php-pear
2. Install PHP MongoDB Driver: sudo pecl install mongo
3. Add extension=mongo.so to you /etc/php.ini
4. Create a short monitoring code as mongo.php and place it at /var/www/html/:

header('Cache-Control: no-cache, must-revalidate');

header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');

header('Content-type: application/json;charset=UTF-8');

class Monitor {

public $VirtualMemory_MB;

public $PageFaults;

public $CurrentConnections;

public $NetworkIO_In_MB;

public $NetworkIO_Out_MB;

public $SpeedIO_Avg_ms;

public $LockRatio_Per;

public $ReadersLocked;

public $WritersLocked;

public $ObjectsInDatabase;

public $DatabaseDataSize_MB;

public $DatabaseIndexSize_MB;

public $IndexMissRatio_Per;

public $CursorsTimedOut;

public $DeleteSelectRatio_Per;

public $InsertSelectRatio_Per;

public $UpdateSelectRatio_Per;

}

if (!isset($_REQUEST['db'])) {

header($_SERVER['SERVER_PROTOCOL'] . ' 500 Internal Server Error', true, 500);

echo json_encode(array('status'=>0,'message'=>'db is missing'));

exit;

}

$databaseName = $_REQUEST['db'];

// connect

$m = new MongoClient();

// select a database

$db = $m->$databaseName;

$ret = $db->execute('db.stats();');

$result = new Monitor;

$result->ObjectsInDatabase = $ret["retval"]["objects"];

$result->DatabaseDataSize_MB = $ret["retval"]["dataSize"];

$result->DatabaseIndexSize_MB = $ret["retval"]["indexSize"];

//var_dump($ret);

$ret = $db->execute('db.serverStatus();');

$result->VirtualMemory_MB = $ret["retval"]["mem"]["virtual"];

$result->PageFaults = $ret["retval"]["extra_info"]["page_faults"];

$result->CurrentConnections = $ret["retval"]["connections"]["current"];

$result->NetworkIO_In_MB = $ret["retval"]["network"]["bytesIn"]/1000000;

$result->NetworkIO_Out_MB = $ret["retval"]["network"]["bytesOut"]/1000000;

$result->SpeedIO_Avg_ms = $ret["retval"]["backgroundFlushing"]["average_ms"];

$result->LockRatio_Per = $ret["retval"]["globalLock"]["lockTime"]/$ret["retval"]["globalLock"]["totalTime"]*100;

$result->ReadersLocked = $ret["retval"]["globalLock"]["currentQueue"]["readers"];

$result->WritersLocked = $ret["retval"]["globalLock"]["currentQueue"]["writers"];

$result->IndexMissRatio_Per = $ret["retval"]["indexCounters"]["btree"]["missRatio"];

$result->CursorsTimedOut = $ret["retval"]["cursors"]["timedOut"];

$result->DeleteSelectRatio_Per = $ret["retval"]["opcounters"]["delete"]/$ret["retval"]["opcounters"]["query"]*100;

$result->InsertSelectRatio_Per = $ret["retval"]["opcounters"]["insert"]/$ret["retval"]["opcounters"]["query"]*100;

$result->UpdateSelectRatio_Per = $ret["retval"]["opcounters"]["update"]/$ret["retval"]["opcounters"]["query"]*100;

echo json_encode($result);

?>

Bottom Line

You MongoDB is ready for monitoring by an external service: http://localhost/mongo.php

Keep Performing,

Moshe Kaplan

mongoDB Insider Tips

This time I would like to share with you several insider tips that can be used to better understand mongoDB and its internals.

Memory Limitation: No, mongoDB does not support it (very similar to SQL Server before adjusting the memory limitations):

Virtual memory and resident sizes may appear to be very large for the mongod process. This is  by design: virtual memory space should be just larger than the size of the data files open and mapped. Resident size will vary depending on the amount of memory not used by other processes on the machine. Yes, if you allocate more memory to the machine, mongoDB will happily consume it,

mongoDB defines its storage size. You can compact it.

Mongo tends to preallocate a significant size of disk.

If your data store is too large you may use the following commands to reduce it:

• db.repairDatabase()
• db.compact()

Use ext4 file system

When ext3 is used and mongoDB allocates new data files, these files have to be filled with zeroes that are written back to the underlying disk (yes, extra unneeded writes to disk).

When ext4 or xfs are used, mongoDB uses the falloc syscall that marks the file as allocated and zeroed, without actually writing anything back to the underlying storage. The result is a better insert performance on ext4 than ext3.

Quorum is for voting, data replication is async by default.

Unlike Cassandra, mongoDB uses Quorum only for primary server selection. Write is done only to primary server and answer is returned to the client immediately. The secondary servers (or some will say the slaves) copy the data in async way.

Synchronous replication is possible but not recommended: Query response to user can be delayed to ensure that data was replicated to slaves. However it may result with 2 orders of magnitude performance decrease: 

mongoDB stores data as BSON and communicates in JSON

BSON or JSON? This is not a question anymore, as mongoDB these days automatically serialize the JSONs into BSONs to save space on disk.

Bottom Line

Now that you better understand the mongoDB architecture, it is time to make more out of it,

Keep Performing,

Moshe Kaplan