The htaccess tester tool can solve your issues: just place your requested URL and the actual htaccess that is being used, and you will get the actual result.
Keep Performing,
Moshe Kaplan
Showing posts with label Apache. Show all posts
Showing posts with label Apache. Show all posts
Nov 25, 2015
Apache htaccess Debugging Ugly. This Will Save Your A$$...
If you ever created redirection rules in Apache htaccess or configuration file, you probably know that things can easily turn ugly. Without debugging tools and with long testing cycle, the debugging can be painful.
The htaccess tester tool can solve your issues: just place your requested URL and the actual htaccess that is being used, and you will get the actual result.
Keep Performing,
Moshe Kaplan
The htaccess tester tool can solve your issues: just place your requested URL and the actual htaccess that is being used, and you will get the actual result.
Keep Performing,
Moshe Kaplan
Jul 16, 2013
Detecting Performance Bottlenecks in Apache httpd Server
“a problem well put is half solved.” ― John Dewey
One of the most important things to well detect a performance issues in a LAMP system is understand the actual processes that causing the issues.
What should we do?
One way to start with is properly configure the Apache logging in a way that we will get fine details about every request performance.
Key Apache httpd Parameters
%B Size of response in bytes, excluding HTTP headers.
%D The time taken to serve the request, in microseconds.
%{VARNAME}e The contents of the environment variable VARNAME.
%T The time taken to serve the request, in seconds.
%X Connection status when response is completed
How to Configure?
Modify your httpd.conf file according to the following article:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" proxy
How do we Analyze it?
Apache log files can be huge (a common case in systems that have some performance challenges).
A useful tool for analyzing the Apache logs, which Cnaan Aviv introduced to me, is GoAccess.
This tool generates reports, statistics and detects errors.
Bottom Line
When done right, solving performance issues is not a black magic. It just need to be done using a well defined method and the right tools.
Keep Performing,
Moshe Kaplan
One of the most important things to well detect a performance issues in a LAMP system is understand the actual processes that causing the issues.
What should we do?
One way to start with is properly configure the Apache logging in a way that we will get fine details about every request performance.
Key Apache httpd Parameters
%B Size of response in bytes, excluding HTTP headers.
%D The time taken to serve the request, in microseconds.
%{VARNAME}e The contents of the environment variable VARNAME.
%T The time taken to serve the request, in seconds.
%X Connection status when response is completed
How to Configure?
Modify your httpd.conf file according to the following article:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" proxy
How do we Analyze it?
Apache log files can be huge (a common case in systems that have some performance challenges).
A useful tool for analyzing the Apache logs, which Cnaan Aviv introduced to me, is GoAccess.
This tool generates reports, statistics and detects errors.
Bottom Line
When done right, solving performance issues is not a black magic. It just need to be done using a well defined method and the right tools.
Keep Performing,
Moshe Kaplan
Sep 26, 2009
Load Balancer: Pay less Do More
When we face a buildup of a new mega system that needs to handle dozens of Gb/s of traffic, supporting both HTTPS and HTTP and perform advanced logic such as Comet our main obstacle is how to design it to keep performance high and price low.
Well if you are dealing with such a system (I seen several of these in the last few months) you probably already familiar with hardware load balancers, software load balancers, Acceleration servers (SSL Proxies), caching servers, DNS round robin and CDNs. If you are not sure yet what to do with these components and why, stay with us...
However, please notice that the following is a short list of various technologies available in the market with a bottom line, this is not a complete list and each case should be analyzed according to the case (I think I should hire a lawyer next time...). If you want to learn more feel free to read Willy Tarreau article: "Making applications scalable with Load Balancing" which covers the overall aspects of these technologies.
DNS Round Robin:
Major Pros: Cheap and simple.
Major Cons: Not dynamic (unless you monitor and manually change it) and if a server fails clients will have major problems acquiring new server address.
Bottom Line: Use it only when servers are for sure up and running (meaning that every DNS node is HA). Very useful to balance several data centers worldwide.
Hardware Load Balancers:
Major Pros: High throughput (10Gbs and counting) low latency and built in HA using VRRP.
Major Cons: Price ($$$).
Bottom Line: Use it for high throughput low latency load balancing or in other words for layer 3/4 load balancing in the data center gateway to balance the traffic between the servers. Cisco, Radware and F5 are good examples.
Accelerators (SSL Proxies and Compression):
Major Pros: Remove overhead from the application servers and keeping traffic as small as possible.
Major Cons: Another layer in the system.
Bottom Line: Use a stack of Apache+ModSSL servers to encrypt/decrypt and compress/decompress behind the HW load balancer keeping cost low and performance high.
Software Load Balancers:
Major Pros: Low cost.
Major Cons: Slow and low throughput.
Bottom Line: Use it when load balancing is almost in the application level (Layer 7) like HTTP Redirect. HAProxy and Apache mod_proxy are good examples.
CDN (Content Delivery Network or Static Files):
Major Pros: Reduce the number of servers in the system
Major Cons: Another layer in the system.
Bottom Line: Use commercial CDN (Amazon S3 and CloudFront are good examples) or dedicated lightweight HTTP like lighttpd to serve static content. It is also recommended to convert dynamic content to static one if possible.
Application Server:
Major Pros: Well we hope at least part of your system is dynamic...
Major Cons: You know, performance boosting is not a joke after all...
Bottom Line: Use optimized system from system to application level, including caching and in memory database to server the dynamic part of the application.
Database:
Major Pros: Well we hope at least part of your system is dynamic... (X2).
Major Cons: You know, performance boosting is not a joke after all... (X2).
Bottom Line: If you plan a very large database, we recommend planning for HA and Sharding from day 1. It's not so difficult and it'll save you a lot work and sleepless nights in the future. And yes, you can use commodity databases such as MySQL.
The bottom line: a wise use of each component can lead you to highly available system while keeping your budget low.
Keep Performing,
Moshe Kaplan. RockeTier. The Performance Experts.
Well if you are dealing with such a system (I seen several of these in the last few months) you probably already familiar with hardware load balancers, software load balancers, Acceleration servers (SSL Proxies), caching servers, DNS round robin and CDNs. If you are not sure yet what to do with these components and why, stay with us...
However, please notice that the following is a short list of various technologies available in the market with a bottom line, this is not a complete list and each case should be analyzed according to the case (I think I should hire a lawyer next time...). If you want to learn more feel free to read Willy Tarreau article: "Making applications scalable with Load Balancing" which covers the overall aspects of these technologies.
DNS Round Robin:
Major Pros: Cheap and simple.
Major Cons: Not dynamic (unless you monitor and manually change it) and if a server fails clients will have major problems acquiring new server address.
Bottom Line: Use it only when servers are for sure up and running (meaning that every DNS node is HA). Very useful to balance several data centers worldwide.
Hardware Load Balancers:
Major Pros: High throughput (10Gbs and counting) low latency and built in HA using VRRP.
Major Cons: Price ($$$).
Bottom Line: Use it for high throughput low latency load balancing or in other words for layer 3/4 load balancing in the data center gateway to balance the traffic between the servers. Cisco, Radware and F5 are good examples.
Accelerators (SSL Proxies and Compression):
Major Pros: Remove overhead from the application servers and keeping traffic as small as possible.
Major Cons: Another layer in the system.
Bottom Line: Use a stack of Apache+ModSSL servers to encrypt/decrypt and compress/decompress behind the HW load balancer keeping cost low and performance high.
Software Load Balancers:
Major Pros: Low cost.
Major Cons: Slow and low throughput.
Bottom Line: Use it when load balancing is almost in the application level (Layer 7) like HTTP Redirect. HAProxy and Apache mod_proxy are good examples.
CDN (Content Delivery Network or Static Files):
Major Pros: Reduce the number of servers in the system
Major Cons: Another layer in the system.
Bottom Line: Use commercial CDN (Amazon S3 and CloudFront are good examples) or dedicated lightweight HTTP like lighttpd to serve static content. It is also recommended to convert dynamic content to static one if possible.
Application Server:
Major Pros: Well we hope at least part of your system is dynamic...
Major Cons: You know, performance boosting is not a joke after all...
Bottom Line: Use optimized system from system to application level, including caching and in memory database to server the dynamic part of the application.
Database:
Major Pros: Well we hope at least part of your system is dynamic... (X2).
Major Cons: You know, performance boosting is not a joke after all... (X2).
Bottom Line: If you plan a very large database, we recommend planning for HA and Sharding from day 1. It's not so difficult and it'll save you a lot work and sleepless nights in the future. And yes, you can use commodity databases such as MySQL.
The bottom line: a wise use of each component can lead you to highly available system while keeping your budget low.
Keep Performing,
Moshe Kaplan. RockeTier. The Performance Experts.
Jun 2, 2009
Tuning LAMP Architecture
It seems that more and more firms are moving to the LAMP architecture these days. Therefore, there is a buzz regarding Apache, Linux, PHP and MySQL tuning
However, since every technology has it own limitations, the need for extreme architectures that overcome their limitations is on the rise as well (I wrote and presented pretty much regarding Sharding, Cloud Computing, In Memory Databases in last few days).
However, it is always good to get back to basic and remember the small advices from smart people how to tune the products themselves. Therefore, I gathered several recommendations to help you get a little bit more from your own application:
LAMP tunning: I would add to this set of articles that you should make sure that you place all your rich content in a CDN, Amazon S3 or any other alternative and not on your own server.
Some information about Apache tunning and even more about HAProxy tuning (RockeTier software based favorite load balancer).
Keep Performing
Moshe Kaplan. RockeTier. The Cloud and Performance Experts.
However, since every technology has it own limitations, the need for extreme architectures that overcome their limitations is on the rise as well (I wrote and presented pretty much regarding Sharding, Cloud Computing, In Memory Databases in last few days).
However, it is always good to get back to basic and remember the small advices from smart people how to tune the products themselves. Therefore, I gathered several recommendations to help you get a little bit more from your own application:
LAMP tunning: I would add to this set of articles that you should make sure that you place all your rich content in a CDN, Amazon S3 or any other alternative and not on your own server.
Some information about Apache tunning and even more about HAProxy tuning (RockeTier software based favorite load balancer).
Keep Performing
Moshe Kaplan. RockeTier. The Cloud and Performance Experts.
Feb 2, 2009
The Mystery of System Calls
Hi,
It always a pleasure to have real life contributions from colleagues in the industry. This time, Rubi Dagan, a system architect and senior team leader at Metacafe, one of the world's largest video sites ww, shares with us "the mystery of system calls".
Background
Metacafe's software system had many calls to time(), and during stress it was felt much stronger. For example, in the figure 1 you can see that 35% of the syscall time was wasted on time().
However, when taking a look on several other servers, it was found that all requests are being processes without calls to time at all!!! (see figure 2). Hint: use strace -cp `ps ax | grep [h]ttpd | awk '{ print $1 }' | tr '\n' ',' | sed 's/,/ -p /g'` -f to get this information
Solving the mystery...
It always a pleasure to have real life contributions from colleagues in the industry. This time, Rubi Dagan, a system architect and senior team leader at Metacafe, one of the world's largest video sites ww, shares with us "the mystery of system calls".
Background
Metacafe's software system had many calls to time(), and during stress it was felt much stronger. For example, in the figure 1 you can see that 35% of the syscall time was wasted on time().
However, when taking a look on several other servers, it was found that all requests are being processes without calls to time at all!!! (see figure 2). Hint: use strace -cp `ps ax | grep [h]ttpd | awk '{ print $1 }' | tr '\n' ',' | sed 's/,/ -p /g'` -f to get this information
Solving the mystery...
The solution of it is based on the BIOS, in an option named HPET – High Precision Event Timer which when enabled, the kernel do fast lookups without a need to use the time() syscall. This method is able to track the time instead of the kernel. Please notice that this function should be enabled on the kernel.
That’s it, instead of Apache or other program deals with time the HPET mechanism do that. The bottom line is reduce time system calls. See also the thread in StackOverflow.
Bottom Line
This new configuraion reduced syscalls time by ~30% and more which is being translated to a great performance impact on Metacafe servers.
P.S We'll be glad to expose here other cases from the industry. Don't be shame to submit your case and contribute the community.
Best Regards,
Moshe Kaplan. RockeTier. The Performance Experts
This new configuraion reduced syscalls time by ~30% and more which is being translated to a great performance impact on Metacafe servers.
P.S We'll be glad to expose here other cases from the industry. Don't be shame to submit your case and contribute the community.
Best Regards,
Moshe Kaplan. RockeTier. The Performance Experts
Subscribe to:
Posts (Atom)