MongoDB, Users and Permissions

NoSQL and Enterprise Security?
That is not the first thing that comes to mind when you consider using NoSQL. It is not a big surprise as the early adapters of NoSQL were Internet companies.
An evident for that you can find in MongoDB, where authentication is dimmed by default.

How to Enable MongoDB Authentication?

1. Create an Admin user (otherwise you will have issues to connect your server) from the local console:
1. use admin;
2. db.addUser({ user: "", pwd: "", roles: [ "userAdminAnyDatabase" ]})
2. Enable authentication in the /etc/mongo.conf: auth=true
3. Restart the mongod instance to enable authentication.

How to Add Additional users?

Select the database that you want to add user to:

use

db.addUser( { user: "", pwd: "", roles: [ "", ""]})

And select the a user role from the following permissions list:

1. read
2. readWrite
3. dbAdmin
4. userAdmin
5. clusterAdmin
6. readAnyDatabase
7. readWriteAnyDatabase
8. userAdminAnyDatabase
9. dbAdminAnyDatabase

How to Provide Permissions to Other Databases?

This one is done with a "copy" like method, where userSource defines the database that the user definition should be copied from:

use

db.addUser( { user: "", userSource: "", roles: [ "" ] } )

In case you want to provide read permissions to all databases you may use the readAnyDatabases

Bottom Line

Not very complex, but more secure. 

Keep Performing,

Moshe Kaplan

Lastest PHP 5.4 with MySQL via yum?

This is probably the link you are looking for: http://www.webtatic.com/packages/php54/
This one will save you a lot of hours dealing with old PHP 5.3 and newest MySQL 5.6 incompatibilities.

My MongoDB and PHP Famous 5 Minutes Tutorial

Did you ever wanted to integrate MongoDB monitoring with a third party service?
Did you wanted to integrate MongoDB performance counters with your load stress environment?

This 5 minutes tutorial will help you expose these details to any third party service or application using a simple JSON service, PHP and MongoDB:

1. Install Apache and PHP and php-devel: sudo yum -y install httpd php php-devel php-pear
2. Install PHP MongoDB Driver: sudo pecl install mongo
3. Add extension=mongo.so to you /etc/php.ini
4. Create a short monitoring code as mongo.php and place it at /var/www/html/:

header('Cache-Control: no-cache, must-revalidate');

header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');

header('Content-type: application/json;charset=UTF-8');

class Monitor {

public $VirtualMemory_MB;

public $PageFaults;

public $CurrentConnections;

public $NetworkIO_In_MB;

public $NetworkIO_Out_MB;

public $SpeedIO_Avg_ms;

public $LockRatio_Per;

public $ReadersLocked;

public $WritersLocked;

public $ObjectsInDatabase;

public $DatabaseDataSize_MB;

public $DatabaseIndexSize_MB;

public $IndexMissRatio_Per;

public $CursorsTimedOut;

public $DeleteSelectRatio_Per;

public $InsertSelectRatio_Per;

public $UpdateSelectRatio_Per;

}

if (!isset($_REQUEST['db'])) {

header($_SERVER['SERVER_PROTOCOL'] . ' 500 Internal Server Error', true, 500);

echo json_encode(array('status'=>0,'message'=>'db is missing'));

exit;

}

$databaseName = $_REQUEST['db'];

// connect

$m = new MongoClient();

// select a database

$db = $m->$databaseName;

$ret = $db->execute('db.stats();');

$result = new Monitor;

$result->ObjectsInDatabase = $ret["retval"]["objects"];

$result->DatabaseDataSize_MB = $ret["retval"]["dataSize"];

$result->DatabaseIndexSize_MB = $ret["retval"]["indexSize"];

//var_dump($ret);

$ret = $db->execute('db.serverStatus();');

$result->VirtualMemory_MB = $ret["retval"]["mem"]["virtual"];

$result->PageFaults = $ret["retval"]["extra_info"]["page_faults"];

$result->CurrentConnections = $ret["retval"]["connections"]["current"];

$result->NetworkIO_In_MB = $ret["retval"]["network"]["bytesIn"]/1000000;

$result->NetworkIO_Out_MB = $ret["retval"]["network"]["bytesOut"]/1000000;

$result->SpeedIO_Avg_ms = $ret["retval"]["backgroundFlushing"]["average_ms"];

$result->LockRatio_Per = $ret["retval"]["globalLock"]["lockTime"]/$ret["retval"]["globalLock"]["totalTime"]*100;

$result->ReadersLocked = $ret["retval"]["globalLock"]["currentQueue"]["readers"];

$result->WritersLocked = $ret["retval"]["globalLock"]["currentQueue"]["writers"];

$result->IndexMissRatio_Per = $ret["retval"]["indexCounters"]["btree"]["missRatio"];

$result->CursorsTimedOut = $ret["retval"]["cursors"]["timedOut"];

$result->DeleteSelectRatio_Per = $ret["retval"]["opcounters"]["delete"]/$ret["retval"]["opcounters"]["query"]*100;

$result->InsertSelectRatio_Per = $ret["retval"]["opcounters"]["insert"]/$ret["retval"]["opcounters"]["query"]*100;

$result->UpdateSelectRatio_Per = $ret["retval"]["opcounters"]["update"]/$ret["retval"]["opcounters"]["query"]*100;

echo json_encode($result);

?>

Bottom Line

You MongoDB is ready for monitoring by an external service: http://localhost/mongo.php

Keep Performing,

Moshe Kaplan

mongoDB Insider Tips

This time I would like to share with you several insider tips that can be used to better understand mongoDB and its internals.

Memory Limitation: No, mongoDB does not support it (very similar to SQL Server before adjusting the memory limitations):

Virtual memory and resident sizes may appear to be very large for the mongod process. This is  by design: virtual memory space should be just larger than the size of the data files open and mapped. Resident size will vary depending on the amount of memory not used by other processes on the machine. Yes, if you allocate more memory to the machine, mongoDB will happily consume it,

mongoDB defines its storage size. You can compact it.

Mongo tends to preallocate a significant size of disk.

If your data store is too large you may use the following commands to reduce it:

• db.repairDatabase()
• db.compact()

Use ext4 file system

When ext3 is used and mongoDB allocates new data files, these files have to be filled with zeroes that are written back to the underlying disk (yes, extra unneeded writes to disk).

When ext4 or xfs are used, mongoDB uses the falloc syscall that marks the file as allocated and zeroed, without actually writing anything back to the underlying storage. The result is a better insert performance on ext4 than ext3.

Quorum is for voting, data replication is async by default.

Unlike Cassandra, mongoDB uses Quorum only for primary server selection. Write is done only to primary server and answer is returned to the client immediately. The secondary servers (or some will say the slaves) copy the data in async way.

Synchronous replication is possible but not recommended: Query response to user can be delayed to ensure that data was replicated to slaves. However it may result with 2 orders of magnitude performance decrease: 

mongoDB stores data as BSON and communicates in JSON

BSON or JSON? This is not a question anymore, as mongoDB these days automatically serialize the JSONs into BSONs to save space on disk.

Bottom Line

Now that you better understand the mongoDB architecture, it is time to make more out of it,

Keep Performing,

Moshe Kaplan

Some Best Practices for your Next MySQL Installation with SSD

Do you plan on installing a new MySQL server? 
Did you ask yourself what is the best file system for it? 
What modification are recommended? 
What MySQL version to use?
Do you consider using SSD?

You got to the right place.

Linux, SSD and MySQL. The Best Practices

1. Up to date CentOS (6.3). MySQL RPM are easily available for this platform w/o the need to compile them.
2. ext4 file system. It is has some goodies over ext3 and is even more recommended if you plan to use SSD (and you may need SSD if your system is resource demanding).
3. File system discard option. Recommended for SSD to avoid the need to read before write.
4. Consider some extra cheaper disks. SSD disks are highly expensive and can have a relatively short life if they are not used properly. Consider placing some cheaper disks (SAS or SATA) to handle tasks that do not require the high end SSD disks.
5. Some more recommendations about Linux and SSD from Patrick's:
1. File system layer: remove 'relatime' if present and add 'noatime,nodiratime,discard' to reduce writes and improve performance and disk life expectancy.
2. Scheduler: use the 'deadline' scheduler instead of 'CFQ' to match SSD behavior.
3. Swap: set a small swappiness
4. tmp: move /tmp and /var/tmp to RAM disk to improve performance and avoid unneeded writes to disk.
5. Partition Alignment: when creating a partition, enter a start sector of at least 2,048 and divisible by 512 to align pages.
6. MySQL 5.6. 5.6 if finally in GA and it good idea to start working with it before 5.5 become obsolete.

Bottom Line
Few decisions can enhance your MySQL performance. Make your decisions right.

Keep Performing,
Moshe Kaplan

Instant Deployment with git

One of the nicest things in git is that you can actually deploy code to production (or just your test environment) w/o implementing a complicated CI solution.
Off course that using CI such as Jenkins is recommended when compilation is required (JAVA/C/C++/C#) or when TDD/unit tests are use (and you better use them).
Yet, when you just need to deploy, git can be a great service for you.

Note: please note that you must an ssh access from the local machine (not always feasble).

How is done?
Actually it is very simple (and a complete explenation can be found at stackoverflow and toroid):

Step #1: Setup a repository in the development environment and commit a first file
> git init

Step #2: Setup an environment at the web server
> mkdir /path/to/dir/website.git && cd /path/to/dir/website.git
> git init --bare

Step #3: Enable a post receive hook at the web server

> mkdir /var/www/www.mydomain.com

> cat > hooks/post-receive

#!/bin/sh

GIT_WORK_TREE=/var/www/www.mydomain.com git checkout -f

> chmod +x hooks/post-receive

Step #4: Create a trigger at the development environment (and why you need ssh access from the workstation).

> git remote add web ssh://www.mydomain.com/path/to/dir/website.git

> git push web +master:refs/heads/master

Step #5: Code, Commit and Push to production

> echo 'Rocking website!' > index.html
> git add index.html
> git commit -q -m "The index file was commited"
> git push web

Bottom Line
It is a simple task, but tricky if you don't have a direct access to the server (anyone said Jenkins?)

Keep Performing,
Moshe Kaplan

Tracing SQL Server Performance and Stability Issues

The following two undocumented stored procedures will do a miracle to your efforts to trace issues in SQL Server:

1. EXEC sp_readerrorlog;
2. DBCC LOG('DATABASE_NAME', 3);

The first can be used to find failures, backup procedures and other goodies in the database log. The second can be used to trace heavy UPDATE, DELETE and INSERT transactions in the transaction log of a specific database.

Keep Performing,

Moshe Kaplan