“a problem well put is half solved.” ― John Dewey
One of the most important things to well detect a performance issues in a LAMP system is understand the actual processes that causing the issues.
What should we do?
One way to start with is properly configure the Apache logging in a way that we will get fine details about every request performance.
Key Apache httpd Parameters
%B Size of response in bytes, excluding HTTP headers.
%D The time taken to serve the request, in microseconds.
%{VARNAME}e The contents of the environment variable VARNAME.
%T The time taken to serve the request, in seconds.
%X Connection status when response is completed
How to Configure?
Modify your httpd.conf file according to the following article:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" proxy
How do we Analyze it?
Apache log files can be huge (a common case in systems that have some performance challenges).
A useful tool for analyzing the Apache logs, which Cnaan Aviv introduced to me, is GoAccess.
This tool generates reports, statistics and detects errors.
Bottom Line
When done right, solving performance issues is not a black magic. It just need to be done using a well defined method and the right tools.
Keep Performing,
Moshe Kaplan
News, Personal view and perspective of the software performance field, cloud computing and industry based on my experience
Jul 16, 2013
Jun 28, 2013
Changing MySQL Binary Log Files Location to Another Directory
What is the Default?
Usually in most installations, binary log files are located in the MySQL default directory (/var/lib/mysql) just next to the data files.
Why Should I Move the Binary Logs to Another Directory?
Each data modification (INSERT, UPDATE, DELETE...) and data definition (ALTER, ADD, DROP...) statement that you perform in your server are recorded in the Log files.
Therefore, each time you make any of these statements, you actually update both your data files and your log files. The result is high IO utilization that is focused on a specific disk area.
A common recommendation in the database field is to separate these files to two different disks in order to get a better performance.
How to Perform it?
Bottom Line
Few steps and your server is ready for more traffic and data.
Keep Performing,
Moshe Kaplan
Usually in most installations, binary log files are located in the MySQL default directory (/var/lib/mysql) just next to the data files.
Why Should I Move the Binary Logs to Another Directory?
Each data modification (INSERT, UPDATE, DELETE...) and data definition (ALTER, ADD, DROP...) statement that you perform in your server are recorded in the Log files.
Therefore, each time you make any of these statements, you actually update both your data files and your log files. The result is high IO utilization that is focused on a specific disk area.
A common recommendation in the database field is to separate these files to two different disks in order to get a better performance.
How to Perform it?
- Change the log-bin variable in the my.cnf to log-bin=/path/to/new/directory/mysql-bin
- Purge as many files as you can (PURGE BINLOG...) in order to minimize the number of moved files (see stop 4).
- Stop the master (service mysql stop).
- Move the files to the new directory: mv /var/lib/mysql/mysql-bin.* /path/to/new/directory
- Start the master again (service mysql start).
Bottom Line
Few steps and your server is ready for more traffic and data.
Keep Performing,
Moshe Kaplan
Jun 25, 2013
MongoDB and Java
You can find below some hints for initial Java and MongoDB integration
Take a Look at the Requirements
- MongoDB
- MongoDB-Java-Driver
- JDK. If you want use JDK 1.6 or newer, you will get an error like this one: "DBObject cannot be resolved to a type"
Installing Java
yum -y install java-1.7.0-openjdk
yum -y install java-1.7.0-openjdk-devel
Getting MongoDB driver
Compiling Java
javac -d . *.java
java -cp . com/example/mbeans/Main
Define MongoDB Headers in the Code
import com.mongodb.MongoClient;
import com.mongodb.MongoException;
import com.mongodb.WriteConcern;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.BasicDBObject;
import com.mongodb.DBObject;
import com.mongodb.DBCursor;
import com.mongodb.ServerAddress;
import com.mongodb.*;
Connect to MongoDB
DB _db;
public void init() {
try {
System.out.println("Connecting to mongo...");
MongoClient mongoClient = new MongoClient("127.0.0.1" , 27017);
_db = mongoClient.getDB("display");
System.out.println("Connected to mongo...");
} catch (Exception e) {
System.out.println("Failed Connecting Mongo...");
}
Query the Database (Get the Number of Connections)
CommandResult stats = _db.command("serverStatus");
return Integer.valueOf((((DBObject)(stats.get("connections"))).get("current")).toString());
Bottom Line
Java and MongoDB integration is not too difficult, you just need to do the right thinks right...
Keep Performing,
Jun 7, 2013
DZone's Definitive Guide to Cloud Providers is here!
I was a part of the team that was contributing to the DZone's Definitive Guide to Cloud Providers, and it's finally here!
If you consider what cloud solution is best for you, take a look at this guide.
Keep Performing,
Moshe Kaplan
If you consider what cloud solution is best for you, take a look at this guide.
Keep Performing,
Moshe Kaplan
May 26, 2013
How Much RAM Should You Have for MongoDB?
Why RAM is So Important?
RAM is much faster than hard disks. For example, we gained X7 performance boost when we used MySQL In Memory engine rather than InnoDB engine.
Is It Only For Query Caching?
Most of us are familiar with query caching, when results of a query are saved in cache in case it will be called again.
However, RAM can be used to store a copy of your database in the RAM as well. This enables you getting best performance even when you do queries that were not done before.
So How Much RAM Do I Need?
My Recommendation that your system RAM should be larger than MongoDB data files + index files: SizeOf(RAM) > SizeOf(Data Files) + SizeOf(Index Files).
That way, your files will always be in memory and you will avoid swapping.
What If My Data is Too Big?
In this case you should choose one of the following strategies:
Careful design will do miracles to your production environment.
Keep Performing,
Moshe Kaplan
RAM is much faster than hard disks. For example, we gained X7 performance boost when we used MySQL In Memory engine rather than InnoDB engine.
Is It Only For Query Caching?
Most of us are familiar with query caching, when results of a query are saved in cache in case it will be called again.
However, RAM can be used to store a copy of your database in the RAM as well. This enables you getting best performance even when you do queries that were not done before.
So How Much RAM Do I Need?
My Recommendation that your system RAM should be larger than MongoDB data files + index files: SizeOf(RAM) > SizeOf(Data Files) + SizeOf(Index Files).
That way, your files will always be in memory and you will avoid swapping.
What If My Data is Too Big?
In this case you should choose one of the following strategies:
- Shard your MongoDB between several servers.
- Tune your queries (this is a good idea in general).
- Design your databases and collections to support "Data Hierarchy".
- Choose SSD as your hard disk solution.
- Consider vSMP Foundation for Memory Expansion from ScaleMP to increase RAM up to 256TB as our readers suggested (P.S don't forget them to ask for the special price :-)
Careful design will do miracles to your production environment.
Keep Performing,
Moshe Kaplan
May 20, 2013
mongoDB Performance Tuning (Full Presentation from Israel MongoDB User Group)
I gave a lecture today at the Israel MongoDB User Group regarding MongoDB performance tuning and how scale with it.
It was a great event that was organized by Wix, 10gen and Trainologic.
Did you miss the event? Well, don't miss the lesson. You can find below the full presentation:
Bottom Line
Boost your MongoDB...
Keep Performing,
Moshe Kaplan
It was a great event that was organized by Wix, 10gen and Trainologic.
Did you miss the event? Well, don't miss the lesson. You can find below the full presentation:
Bottom Line
Boost your MongoDB...
Keep Performing,
Moshe Kaplan
Apr 15, 2013
MongoDB, Users and Permissions
NoSQL and Enterprise Security?
That is not the first thing that comes to mind when you consider using NoSQL. It is not a big surprise as the early adapters of NoSQL were Internet companies.
An evident for that you can find in MongoDB, where authentication is dimmed by default.
How to Enable MongoDB Authentication?
That is not the first thing that comes to mind when you consider using NoSQL. It is not a big surprise as the early adapters of NoSQL were Internet companies.
An evident for that you can find in MongoDB, where authentication is dimmed by default.
How to Enable MongoDB Authentication?
- Create an Admin user (otherwise you will have issues to connect your server) from the local console:
- use admin;
- db.addUser({ user: "
", pwd: " ", roles: [ "userAdminAnyDatabase" ]}) - Enable authentication in the /etc/mongo.conf: auth=true
- Restart the mongod instance to enable authentication.
How to Add Additional users?
Select the database that you want to add user to:
use
db.addUser( { user: "", pwd: "", roles: [ "", ""]})
And select the a user role from the following permissions list:
How to Provide Permissions to Other Databases?
This one is done with a "copy" like method, where userSource defines the database that the user definition should be copied from:
use
db.addUser( { user: "", userSource: "", roles: [ "" ] } )
In case you want to provide read permissions to all databases you may use the readAnyDatabases
Bottom Line
Not very complex, but more secure.
Keep Performing,
Subscribe to:
Posts (Atom)
