Do you face some performance issues in your MongoDB setup?
In this case use the following steps to provide some first aid to your system and gain some space for a long term architecture (such as Sharding)
Step 1: Enable slow queries
Get intelligence about your system behavior and performance bottlenecks. Usually there is a high correlation between the slow queries and your performance bottleneck, so use the following method to enable your system profiling collection:
db.setProfilingLevel(1, 100);
Step 2: Use explain
Explore the problematic queries using explain. You can also use mtools to analyze the logged queries to find high frequent ones.
Step 3: Create indexes
Your analysis should result with new indexes in order to improve the queries
Don't forget to use index buildup in the background to avoid collections locking and system downtime.
Step 4: Use sparse indexes to reduce the size of the indexes
If you use sparse documents, and heavily using the $exists key words in your queries, using sparse indexes (that includes only documents that includes your field) can minimize your index size the boost your query performance.
Step 5: Use secondary preferred to offload queries to slaves
You probably have a replica set and it's waste of resources not using your slaves for read queries (especially for reporting and search operations).
By changing your connection string to secondary preferred, your application will try to run read queries on the slaves before doing that on your master.
Bottom Line
Using these simple method, you can gain time and space before hitting the wall.
Keep Performing,
Moshe Kaplan
News, Personal view and perspective of the software performance field, cloud computing and industry based on my experience
Showing posts with label mongoDB tuning. Show all posts
Showing posts with label mongoDB tuning. Show all posts
Sep 20, 2015
Jul 31, 2015
MongoDB 3.0 WiredTiger is Big News for Multi Tenant Deployments
A database per tenant is a common practice in multi tenant systems.
Why? Well, first it s easier to implement as you don't need to record the tenant id in every document, and second, it easier to avoid security issues by compromising the tenant id.
MongoDB 3.0 Storage Engines
MongoDB introduced a new concept in version 3.0 (that is familiar for MySQL users): you can select your preferred storage engine. You can continue using the old MMAPv1 storage engine (that is still the default one for version 3.0) or select WiredTiger that offers compression, document level lock and better performance in some cases.
What is Wrong with MMAPv1
MMAPv1 creates for every new database a data file that its minimal size is ~70MB (and this one is being filled, a new file is created with a doubled size).
This may not be an issue for large size databases, but if your system design is built on a large number of tenants, that many of them will not have more than few records (lets say trial tenants that decided not to go on w/ your system), you are going to have 70GB disk allocation for every 1,000 tenants.
This behavior will result in another side affect: very high IO usage (mostly for read), that is due to the need to read every time a large file in order to update or insert very few rows.
WiredTiger Comes to Rescue
WiredTiger includes a compression method that can cut 85% of your storage needs in a large database (trust me that I saw this number in a 10TB billing system). That is great, but more important, it does not allocate this 70MB file per each database (it satisfies with two small files for indexes and data).
The result: for a 3,500 tenants system, the database was shrunk from 330GB to under 1GB... not to mention that IOPS that were dropped from 12,000 to 600...
Bottom Line
If you are using MongoDB for multi tenant system, WiredTiger can cut your storage needs in 99% and your IOPS in 95%. This can save a fortune.
Keep Performing,
Moshe Kaplan
Why? Well, first it s easier to implement as you don't need to record the tenant id in every document, and second, it easier to avoid security issues by compromising the tenant id.
MongoDB 3.0 Storage Engines
MongoDB introduced a new concept in version 3.0 (that is familiar for MySQL users): you can select your preferred storage engine. You can continue using the old MMAPv1 storage engine (that is still the default one for version 3.0) or select WiredTiger that offers compression, document level lock and better performance in some cases.
What is Wrong with MMAPv1
MMAPv1 creates for every new database a data file that its minimal size is ~70MB (and this one is being filled, a new file is created with a doubled size).
This may not be an issue for large size databases, but if your system design is built on a large number of tenants, that many of them will not have more than few records (lets say trial tenants that decided not to go on w/ your system), you are going to have 70GB disk allocation for every 1,000 tenants.
This behavior will result in another side affect: very high IO usage (mostly for read), that is due to the need to read every time a large file in order to update or insert very few rows.
WiredTiger Comes to Rescue
WiredTiger includes a compression method that can cut 85% of your storage needs in a large database (trust me that I saw this number in a 10TB billing system). That is great, but more important, it does not allocate this 70MB file per each database (it satisfies with two small files for indexes and data).
The result: for a 3,500 tenants system, the database was shrunk from 330GB to under 1GB... not to mention that IOPS that were dropped from 12,000 to 600...
Bottom Line
If you are using MongoDB for multi tenant system, WiredTiger can cut your storage needs in 99% and your IOPS in 95%. This can save a fortune.
Keep Performing,
Moshe Kaplan
Oct 13, 2013
MongoDB Index Tuning and Dex
One of the first things when your system is rolled into production (or better when you have your acceptance tests) is checking the system performance performance and tune it.
When we talk about databases, we usually talk about index tuning, and MongoDB is not different.
How to Detect the Slow Queries?
First enable you MongoDB slow queries log. It's easy, just run db.setProfilingLevel(1, 1) from the command line.
How to Analyze the Slow Queries?
Well, you can use the built in queries (after all the profiling is saved at a MongoDB collection). However, Dex, a new tool from MongoLab, can help you shorten the time to index...
It's a Python based product that can be easily installed and run and provides a list of recommended indexes based on MongoLab best practices.
How to Install and Run?
Should We Just Copy and Paste the New Indexes?
As you may know every added index results in a performance degragation when INSERT, DELETE and UPDATE statement are performed. Therefore, we should carefuly select the indexes to be added:
MongoDB queries tuning is easier than ever: Just Tune It!
Keep Performing,
Moshe Kaplan
When we talk about databases, we usually talk about index tuning, and MongoDB is not different.
How to Detect the Slow Queries?
First enable you MongoDB slow queries log. It's easy, just run db.setProfilingLevel(1, 1) from the command line.
How to Analyze the Slow Queries?
Well, you can use the built in queries (after all the profiling is saved at a MongoDB collection). However, Dex, a new tool from MongoLab, can help you shorten the time to index...
It's a Python based product that can be easily installed and run and provides a list of recommended indexes based on MongoLab best practices.
How to Install and Run?
- Install python: yum -y install python
- Install pip
- Install Dex using pip: pip install dex
- Create a log file: touch mongodb.log
- Run it: sudo dex -p -v -f mongodb.log mongodb://user:password@serverDNS:serverPort/database. Don't forget to use the various flags to get errors in case of wrong configuration.
Should We Just Copy and Paste the New Indexes?
As you may know every added index results in a performance degragation when INSERT, DELETE and UPDATE statement are performed. Therefore, we should carefuly select the indexes to be added:
- We should check that the new indexes do not overlap existing indexes. If there is an overlap, we should remove/unify the indexes definitions (use the .db.collection.getIndexes() method to retrieve the data.
- We should check that the queries are frequently used in order to avoid saving few seconds once a month while doing a SELECT, and pay for that at peak times INSERTs.
MongoDB queries tuning is easier than ever: Just Tune It!
Keep Performing,
Moshe Kaplan
May 26, 2013
How Much RAM Should You Have for MongoDB?
Why RAM is So Important?
RAM is much faster than hard disks. For example, we gained X7 performance boost when we used MySQL In Memory engine rather than InnoDB engine.
Is It Only For Query Caching?
Most of us are familiar with query caching, when results of a query are saved in cache in case it will be called again.
However, RAM can be used to store a copy of your database in the RAM as well. This enables you getting best performance even when you do queries that were not done before.
So How Much RAM Do I Need?
My Recommendation that your system RAM should be larger than MongoDB data files + index files: SizeOf(RAM) > SizeOf(Data Files) + SizeOf(Index Files).
That way, your files will always be in memory and you will avoid swapping.
What If My Data is Too Big?
In this case you should choose one of the following strategies:
Careful design will do miracles to your production environment.
Keep Performing,
Moshe Kaplan
RAM is much faster than hard disks. For example, we gained X7 performance boost when we used MySQL In Memory engine rather than InnoDB engine.
Is It Only For Query Caching?
Most of us are familiar with query caching, when results of a query are saved in cache in case it will be called again.
However, RAM can be used to store a copy of your database in the RAM as well. This enables you getting best performance even when you do queries that were not done before.
So How Much RAM Do I Need?
My Recommendation that your system RAM should be larger than MongoDB data files + index files: SizeOf(RAM) > SizeOf(Data Files) + SizeOf(Index Files).
That way, your files will always be in memory and you will avoid swapping.
What If My Data is Too Big?
In this case you should choose one of the following strategies:
- Shard your MongoDB between several servers.
- Tune your queries (this is a good idea in general).
- Design your databases and collections to support "Data Hierarchy".
- Choose SSD as your hard disk solution.
- Consider vSMP Foundation for Memory Expansion from ScaleMP to increase RAM up to 256TB as our readers suggested (P.S don't forget them to ask for the special price :-)
Careful design will do miracles to your production environment.
Keep Performing,
Moshe Kaplan
May 20, 2013
mongoDB Performance Tuning (Full Presentation from Israel MongoDB User Group)
I gave a lecture today at the Israel MongoDB User Group regarding MongoDB performance tuning and how scale with it.
It was a great event that was organized by Wix, 10gen and Trainologic.
Did you miss the event? Well, don't miss the lesson. You can find below the full presentation:
Bottom Line
Boost your MongoDB...
Keep Performing,
Moshe Kaplan
It was a great event that was organized by Wix, 10gen and Trainologic.
Did you miss the event? Well, don't miss the lesson. You can find below the full presentation:
Bottom Line
Boost your MongoDB...
Keep Performing,
Moshe Kaplan
Mar 6, 2013
mongoDB Insider Tips
This time I would like to share with you several insider tips that can be used to better understand mongoDB and its internals.
Bottom Line
Memory Limitation:
No, mongoDB does not support it (very similar to SQL Server before adjusting the memory
limitations):
Virtual memory and resident sizes may appear
to be very large for the mongod process. This is by design: virtual memory space should be just larger than the size of the data files open and mapped. Resident
size will vary depending on the amount of memory not used by other processes on
the machine. Yes, if you allocate more memory to the machine, mongoDB will happily consume it,
mongoDB defines its storage size. You can compact
it.
Mongo tends to preallocate a significant size
of disk.
If your data store is too large you may use the
following commands to reduce it:
Use ext4 file system
When ext3 is used and mongoDB allocates new
data files, these files have to be filled with zeroes that are written back to the
underlying disk (yes, extra unneeded writes to disk).
When ext4 or xfs are used, mongoDB
uses the falloc syscall that marks the file as allocated and zeroed, without actually writing
anything back to the underlying storage. The result is a better insert
performance on ext4 than ext3.
Quorum is for voting, data replication is async
by default.
Unlike Cassandra, mongoDB uses Quorum only for
primary server selection. Write is done only to primary server and answer is returned to the client immediately. The secondary servers
(or some will say the slaves) copy the data in async way.
Synchronous replication is possible but not recommended: Query response to user can be delayed to
ensure that data was replicated to slaves. However it may result with 2 orders of
magnitude performance decrease:
mongoDB stores data as BSON and communicates in
JSON
BSON or JSON? This is not a question anymore, as
mongoDB these days automatically serialize the JSONs into BSONs to save space
on disk.
Now that you better understand the mongoDB architecture, it is time to make more out of it,
Keep Performing,
Feb 11, 2013
mongoDB Configuration File Tuning
For those of us who are regular to MySQL, its configuration file (my.cnf) and the many options in it (some may say too many), mongoDB seems to be the simple.
With only few dozens of parameters in the configuration file (mongo.conf) and a dozen that are actually related to performance tuning, it may be a relatively short task to tune a mongoDB configuration file. Probably some of us will pay for that in production...
NOTE: this post does not refer the Sharding configuration.
What Can be Done?
Number of Connections
Like many other products (Apache httpd, MySQL...) the number of user connections can affect performance. mongoDB supports that using maxConns = N. Numbers can reach 20,000, but you should adjust it to your own server resources.
Write to Disk
Disk writing is usually a bottleneck in database systems. Therefore, wrtie to disk frequency and initial storage allocation can highly efhttp://stackoverflow.com/questions/8331099/what-is-the-javascript-engine-that-runs-mongodb-shellfect your system performance. Yet notice that delaying disk writing can effect your system recovery (many of you probably familiar with it from MySQL).
You should notice the following two options:
mongoDB provides many supporting services. Disabling some of them (if you do not use them), may help you save some CPU cycles:
With only few dozens of parameters in the configuration file (mongo.conf) and a dozen that are actually related to performance tuning, it may be a relatively short task to tune a mongoDB configuration file. Probably some of us will pay for that in production...
NOTE: this post does not refer the Sharding configuration.
What Can be Done?
Number of Connections
Like many other products (Apache httpd, MySQL...) the number of user connections can affect performance. mongoDB supports that using maxConns = N. Numbers can reach 20,000, but you should adjust it to your own server resources.
Write to Disk
Disk writing is usually a bottleneck in database systems. Therefore, wrtie to disk frequency and initial storage allocation can highly efhttp://stackoverflow.com/questions/8331099/what-is-the-javascript-engine-that-runs-mongodb-shellfect your system performance. Yet notice that delaying disk writing can effect your system recovery (many of you probably familiar with it from MySQL).
You should notice the following two options:
- Delay journal (database log) write to disk using journalCommitInterval = 300. This parameter supports intervals between 2ms (slowest but safest for recovery) and 300ms (fastest but prone to recovery options). The default os 100ms, but you may increase it to 300ms to save resources.
- Preallicate space at mongoDB startup by keeping noprealloc = false.
mongoDB provides many supporting services. Disabling some of them (if you do not use them), may help you save some CPU cycles:
- BSON validation using objcheck = false.
- HTTP Interface using nohttpinterface = true. The mongoDB HTTP interface exposes the database statistics and enables simple REST queries through port 28017.
- Scripting Engine using noscripting = true. mongoDB is using SpiderMonkey JavaScript scripting engine to enable interactive shell. Somehow, I have a feeling that you are not going to disable it.
- REST service using rest = false. See in the HTTP interface.
- Profiling service (inc. slow queries logging) using profile = 0.
Bottom Line
mongoDB is a simple, yet very effective tool to solve many business needs. Doing some tuning will help you avoid issues during peak times.
Keep Performing,
Labels:
mongo.conf,
mongoDB,
mongoDB tuning,
my.cnf,
Performance Tuning
Subscribe to:
Posts (Atom)