Archive for August 11th, 2010


Increase performance of website by compressing JavaScript.

Filed Under: Performance, PHP by Nguyễn Bá Khoa — 48 Comments
11/08/2010
Performance tuning of any website involves lots of things to be taken care of. For example, compressing the output / response of the website increase the performance by significant folds. GZIp and Deflate compression techniques are commonly used in servers like Apache, Tomcat, jBoss etc.Also the size of a webpage plays significant role in load time and their by affecting websites performance. Thus if we reduce the size of the webpage by any mechanism, we can save a lot of time in loading.JavaScript plays a very important role in todays “Web 2.0″ applications. Lots of Ajax/DOM parsing etc can be seen commonly in all the web applications. JavaScript files for any websites may reach thousands of lines of codes.Hence we can improve the performance of any page by compressing the code in JavaScript. Lots of mechanisms are available on internet to “minify” the JavaScript code. One of such tool is: JSMin.

JSMin

JSMin is a filter that removes comments and unnecessary whitespaces from JavaScript files. It typically reduces filesize by half, resulting in faster downloads. It also encourages a more expressive programming style because it eliminates the download cost of clean, literate self-documentation.Try JSMin here.I tried following JavaScript code (from jQuery.js file) with JSMin.

function success(){	// If a local callback was specified, fire it and pass it the data	if ( s.success )		s.success( data, status );	// Fire the global callback	if ( s.global )		jQuery.event.trigger( "ajaxSuccess", [xhr, s] );}function complete(){	// Process result	if ( s.complete )		s.complete(xhr, status);	// The request was completed	if ( s.global )		jQuery.event.trigger( "ajaxComplete", [xhr, s] );	// Handle the global AJAX counter	if ( s.global && ! --jQuery.active )		jQuery.event.trigger( "ajaxStop" );}

and following is what JSMin gave me after removing unnecessary spaces and comments.

function success(){if(s.success)s.success(data,status);if(s.global)jQuery.event.trigger("ajaxSuccess",[xhr,s]);}function complete(){if(s.complete)s.complete(xhr,status);if(s.global)jQuery.event.trigger("ajaxComplete",[xhr,s]);if(s.global&&!--jQuery.active)jQuery.event.trigger("ajaxStop");}

Almost 55% of compression achieved. :)

Other tools available in similar lines are:YUI CompressorDojo Toolkit’s ShrinkSafeDean Edward’s Packer

Tags: , , , , ,
Comment

56 Tips/Tricks để cải thiện performance cho PHP

Filed Under: Performance, PHP by Nguyễn Bá Khoa — Leave a comment
11/08/2010

Hum trước lướt trên mạng lượm lặt được bài viết tip về performance php. Vất lên đây cho đỡ quên phát ^^Today i am searching solutions for any tips and tricks about best practice to optimize PHP code performances and i found some of useful articles which you may interested with.One of them is an old post by Reinhold Webber, a good list of consideration practice in optimizing PHP code performances.Here are Webber’s points:

  • If a method can be static, declare it static. Speed improvement is by a factor of 4.
  • echo is faster than print.(* compare with list from phplens by John Lim)
  • Use echo’s multiple parameters instead of string concatenation.
  • Set the maxvalue for your for-loops before and not in the loop.
  • Unset your variables to free memory, especially large arrays.
  • Avoid magic like __get, __set, __autoload
  • require_once() is expensive
  • Use full paths in includes and requires, less time spent on resolving the OS paths.
  • If you need to find out the time when the script started executing, $_SERVER[’REQUEST_TIME’] is preferred to time()
  • See if you can use strncasecmp, strpbrk and stripos instead of regex
  • str_replace is faster than preg_replace, but strtr is faster than str_replace by a factor of 4
  • If the function, such as string replacement function, accepts both arrays and single characters as arguments, and if your argument list is not too long, consider writing a few redundant replacement statements, passing one character at a time, instead of one line of code that accepts arrays as search and replace arguments.
  • It’s better to use select statements than multi if, else if, statements.
  • Error suppression with @ is very slow.
  • Turn on apache’s mod_deflate
  • Close your database connections when you’re done with them
  • $row[’id’] is 7 times faster than $row[id]
  • Error messages are expensive
  • Do not use functions inside of for loop, such as for ($x=0; $x < count($array); $x) The count() function gets called each time.
  • Incrementing a local variable in a method is the fastest. Nearly the same as calling a local variable in a function.
  • Incrementing a global variable is 2 times slow than a local var.
  • Incrementing an object property (eg. $this->prop++) is 3 times slower than a local variable.
  • Incrementing an undefined local variable is 9-10 times slower than a pre-initialized one.
  • Just declaring a global variable without using it in a function also slows things down (by about the same amount as incrementing a local var). PHP probably does a check to see if the global exists.
  • Method invocation appears to be independent of the number of methods defined in the class because I added 10 more methods to the test class (before and after the test method) with no change in performance.
  • Methods in derived classes run faster than ones defined in the base class.
  • A function call with one parameter and an empty function body takes about the same time as doing 7-8 $localvar++ operations. A similar method call is of course about 15 $localvar++ operations.
  • Surrounding your string by ‘ instead of ” will make things interpret a little faster since php looks for variables inside “…” but not inside ‘…’. Of course you can only do this when you don’t need to have variables in the string.
  • When echoing strings it’s faster to separate them by comma instead of dot. Note: This only works with echo, which is a function that can take several strings as arguments.
  • A PHP script will be served at least 2-10 times slower than a static HTML page by Apache. Try to use more static HTML pages and fewer scripts.
  • Your PHP scripts are recompiled every time unless the scripts are cached. Install a PHP caching product to typically increase performance by 25-100% by removing compile times.
  • Cache as much as possible. Use memcached – memcached is a high-performance memory object caching system intended to speed up dynamic web applications by alleviating database load. OP code caches are useful so that your script does not have to be compiled on every request
  • When working with strings and you need to check that the string is either of a certain length you’d understandably would want to use the strlen() function. This function is pretty quick since it’s operation does not perform any calculation but merely return the already known length of a string available in the zval structure (internal C struct used to store variables in PHP). However because strlen() is a function it is still somewhat slow because the function call requires several operations such as lowercase & hashtable lookup followed by the execution of said function. In some instance you can improve the speed of your code by using an isset() trick.Ex. 
    if (strlen($foo) < 5) { echo "Foo is too short"; }

    vs.

    if (!isset($foo{5})) { echo "Foo is too short"; }

    Calling isset() happens to be faster then strlen() because unlike strlen(), isset() is a language construct and not a function meaning that it’s execution does not require function lookups and lowercase. This means you have virtually no overhead on top of the actual code that determines the string’s length.

  • When incrementing or decrementing the value of the variable $i++ happens to be a tad slower then ++$i. This is something PHP specific and does not apply to other languages, so don’t go modifying your C or Java code thinking it’ll suddenly become faster, it won’t. ++$i happens to be faster in PHP because instead of 4 opcodes used for $i++ you only need 3. Post incrementation actually causes in the creation of a temporary var that is then incremented. While pre-incrementation increases the original value directly. This is one of the optimization that opcode optimized like Zend’s PHP optimizer. It is a still a good idea to keep in mind since not all opcode optimizers perform this optimization and there are plenty of ISPs and servers running without an opcode optimizer.
  • Not everything has to be OOP, often it is too much overhead, each method and object call consumes a lot of memory.
  • Do not implement every data structure as a class, arrays are useful, too
  • Don’t split methods too much, think, which code you will really re-use
  • You can always split the code of a method later, when needed
  • Make use of the countless predefined functions
  • If you have very time consuming functions in your code, consider writing them as C extensions
  • Profile your code. A profiler shows you, which parts of your code consumes how many time. The Xdebug debugger already contains a profiler. Profiling shows you the bottlenecks in overview
  • mod_gzip which is available as an Apache module compresses your data on the fly and can reduce the data to transfer up to 80%
  • Excellent Article about optimizing php by John Lim

As Reihold Webber pointed to a post from John Lim (found this article copied without state the source here), then i investigate further and truly that is an excellent best practice tutorial for optimizing the php code performance, covered almost all aspects from low level webserver configuration, PHP configuration, coding styling, and performace comparisson as well.Another good practice for better php performance as written in cluesheet.com are:

  • Do use single quotes over double quotes.
  • Do use switch over lots of if statements
  • Do avoid testing loop conditionals with function tests every iteration eg. for($i=0;i<=count($x);$i++){…
  • Do use foreach for looping collections/arrays. PHP4 items are byval, greater than PHP5 items are byref
  • Do consider using the Singleton Method when creating complex PHP classes.
  • Do use POST over GET for all values that will wind up in the database for TCP/IP packet performance reasons.
  • Do use ctype_alnum,ctype_alpha and ctype_digit over regular expression to test form value types for performance reasons.
  • Do use full file paths in production environment over basename/fileexists/open_basedir to avoid performance hits for the filesystem having to hunt through the file path. Once determined, serialize and/or cache path values in a $_SETTINGS array. $_SETTINGS["cwd"]=cwd(./);
  • Do use require/include over require_once/include_once to ensure proper opcode caching.
  • Do use tmpfile or tempnam for creating temp files/filenames
  • Do use a proxy to access web services (XML or JSOM) on foreign domains using XMLHTTP to avoid cross-domain errors. eg. foo.com<–>XMLHTTP<–>bar.com
  • Do use error_reporting (E_ALL); during debug.
  • Do set Apache allowoverride to “none” to improve Apache performance in accessing files/directories.
  • Do use a fast fileserver for serving static content (thttpd). static.mydomain.com, dynamic.mydomain.com
  • Do serialize application settings like paths into an associative array and cache or serialize that array after first execution.
  • Do use PHP output control buffering for page caching of heavilty accessed pages
  • Do use PDO prepare over native db prepare for statements. mysql_attr_direct_query=>1
  • Do NOT use SQL wildcard select. eg. SELECT *
  • Do use database logic (queries, joins, views, procedures) over loopy PHP.
  • Do use shortcut syntax for SQL insers if not using PDO parameters parameters. eg. INSERT INTO MYTABLE (FIELD1,FIELD2) VALUES ((”x”,”y”),(”p”,”q”));

Another Interesting articles about optimizing php performance:

Bài dịch nhặt được ^^!1_ Nếu có thể khai báo hàm tĩnh thì có thể tăng tốc độ lên 4 lần.2_ Echo thì nhanh hơn Print.3_ Sử dụng nhiều câu lệnh Echo thay vì Echo 1 chuỗi nối vào nhau.4_ Đặt giá trị lớn nhất cho vòng lặp for trước, và ko đặt trong vòng lặp for.5_ Hủy các biến bằng hàm unset() để giải phóng bộ nhớ sau khi sử dụng, đặc biệt là những mảng lớn.6_ Tránh dùng các định nghĩa kiểu: __get, __set, __autoload.7_ Tránh dùng hàm require_once().8_ Sử dụng đường dẫn đầy đủ trong includes và requires.9_ Nếu cần tính thời gian lúc code được thực thi, thì nên dùng    $_SERVER['REQUEST_TIME'] thay vì time().10_ Chú ý sử dụng các hàm strncasecmp, strpbrk and stripos thay vì regex.11_ Str_replace thì nhanh hơn preg_replace nhưng mà strtr thì nhanh hơn str_replace 4 lần.12_ Tốt hơn nên dùng câu lệnh Select hoặc Switch thay vì nhiều câu lệnh if, else.13_ Các biến lỗi sử dụng @ thì rất chậm.14_ Bật Apache mod_deflate.15_ Đóng kết nối DB sau khi dùng xong.16_ $row[‘id'] thì nhanh gấp 7 lần $row[id].17_ Hạn chế gửi Error messeage.18_ Không dùng các hàm khi khai báo vòng lặp. Ví dụ: for($x=0; $x<count($array); $x++).19_ Sử dụng biến cục bộ trong hàm thì nhanh hơn.20_ Sử dụng biến toàn cục thì chậm hơn 2 lần so với dùng biết cục bộ.21_ Sử dụng biến cục bộ ko được định nghĩa thì chậm hơn 9-10 lần biến được định nghĩa trước.22_ Định nghĩa biến toàn cục mà không sử dụng trong hàm thì cũng chậm hơn.23_ Hàm được triệu gọi thì độc lập với các hàm ở trong lớp định nghĩa nó. Ví dụ sau khi sử dụng gọi hàm đó, ta định nghĩa thêm vào vào lớp chứa hàm đó, thì tốc độ performance không thay đổi.24_ Hàm được định nghĩa từ lớp thì chạy nhanh hơn hàm định nghĩa từ lớp cơ sở.25_ Hàm có 1 tham số hoặc hàm có body rỗng thì tương đương với thực thi 7-8 lần phép toán $localvar++. Một hàm bình thường thì tương đương 15 lần phép toán $localvar++.26_ Nếu khai báo chuỗi mà không có biến ở trong thì sử dụng ‘’ sẽ nhanh hơn dùng “”.27_ Khi dùng Echo thì dùng phân cách bằng dấu , sẽ nhanh hơn bằng dấu .28_ Đoạn mã PHP thực thi thì sẽ chậm hơn 2-10 lần so với trang HTML tĩnh khi chạy Apache.29_ Mã PHP được biên dịch lại mỗi lần, trừ khi các đoạn mã được cache. Cài đặt các sản phẩm cache PHP cơ bản để tăng hiệu suất từ 25-100% qua việc giảm số lần biên dịch lại.30_ Cách với tất cả khả năng có thể. Memcache là hệ thống cache đối tượng với hiệu suất cao để tăng tốc các ứng dụng web động bằng việc giảm số lần truy suất DB. Opcode cache thì rất hữu ích, giúp các đoạn mã không phải biên dịch lại sau mỗi lần request.31_ Dùng trick để tính độ dài của chuỗi bằng hàm isset thay vì sử dụng hàm strlen().Ví dụ:1.      if (strlen($foo) < 5) { echo ”Foo is too short”; }2.      if (!isset($foo{5})) { echo ”Foo is too short”; }Dùng hàm isset() sẽ nhanh hơn dùng strlen() vì isset() sẽ không tính toán trường hợp lowercase.32_ Viết ++$i thay cho $i++ sẽ nhanh hơn (3 opcodes so với 4).33_ Không phải mọi thứ đều dùng hướng đối tượng, sẽ tốn bộ nhớ.34_ Không phải cài đặt mọi dữ liệu bằng lớp, có thể dùng mảng cũng hữu ích.35_ Không nên chia ra quá nhiều hàm, tận dụng sử dụng lại code.36_ Sử dụng các hàm được định nghĩa trước.37_ Profile code sẽ giúp ta có cái nhìn tổng quan nếu có hiện tượng thắt cổ chai.38_ mod_gzip có sẵn trong module Apache sẽ nén dữ liệu, và giảm dữ liệu truyền đi tới 80%.39_ Sử dụng foreach để duyệt các mảng, các tập. Các phần tử PHP4 là theo giá trị, hay hơn nhiều so với theo tham chiếu ở PHP5.40_ Sử dụng mẫu thiết kế Singleton với các lớp PHP phức tạp.41_ Sử dụng POST thay cho GET với tất cả các giá trị vì lý do performance của TCP/IP.42_ Sử dụng các hàm kiểm tra kiểu dữ liệu thay vì dùng Biểu thức chính quy (Regular Expression).43_ Dùng include/require thay cho include once/require once để đảm bảo opcode cache.44_ Dùng proxy để truy cập các Web Services (XML hay JSON) từ các tên miền nước ngoài dùng XMLHTTP để tránh lỗi cross-domain.45_ Dùng thông báo lỗi (E_ALL) trong quá trình debug.46_ Đặt allowoverride của Apache là “none ” để cải thiện performance khi truy cập file/ thư mục.47_ Dùng một fileserver nhanh để chạy nội dung tĩnh (thttpd). Ví dụ: static.domain.com.48_ Hạn chế dùng các câu SQL: SELECT *.49_ Dùng các phép logic database: queries, joins, views, procedures.50_ Khởi tạo các biến trước khi sử dụng.51_ Dùng đường dẫn đầy đủ thay vì đường dẫn tương đối.52_ Tăng 1 thuộc tính của đối tượng thì chậm hơn việc dùng biến cục bộ 3 lần. ($this->prop++).53_ Khi xử lý dữ liệu XML, sử dụng Regular Expression thì nhanh hơn dùng DOM hoặc SAX.54_ Dùng tham chiếu làm tham số cho hàm thì nhanh hơn dùng tham số bình thường.55_ Dùng INSERT DELAYED nếu như ko cần biết thời gian đưa dữ liệu vào.56_ Dùng multiple-row INSERT để INSERT nhiều dữ liệu bằng 1 câu Query.– Bài viết dịch từ trang chazzuka.com – (bài viết gốc là 63 practices, nhưng mình chỉ hiểu và dịch được hơn 50 cái, và bổ sung thêm 1 số tips từ bên ngoài khác  :”> )

Trích dẫn: http://www.chazzuka.com/blog/?p=163

Tags: , ,
Comment

Robust and High Performance PHP with IIS: Using FastCGI

Filed Under: Performance, PHP by Nguyễn Bá Khoa — 5 Comments
11/08/2010

Robust and High Performance PHP with IIS: Using FastCGINote: The PHP EasyWindows installer that I wrote will install PHP+FastCGI and configure IIS for you.As a Windows user and a former trainer on ASP and IIS, I have always wanted to build high performance PHP applications running IIS. Although I use Apache and Linux all the time, some of our customers are more comfortable with Windows and IIS.The recommended way to run stable PHP applications in IIS is CGI. But since a new process is created for each request and thrown away when the request is done, CGI efficiency is poor. Furthermore the PHP ISAPI dll was never stable enough for my needs, because ISAPI requires thread-safety, but many PHP extensions are not thread-safe.Fortunately there is an alternative open technology called FastCGI. FastCGI processes are persistent as they are reused to handle multiple requests. This solves the CGI performance problem of creating new processes for each request. In benchmarks, you can expect a x4 to x8 improvement in performance compared to plain-old CGI.Shane Caraveo of ActiveState is the author of the FastCGI wrapper for IIS. He first released it in January 2002, and from my testing it appears to be stable enough for production use. 26 Jan 2004 Update: Zend, the company behind PHP’s core engine, is also recommending FastCGI for Windows. See Zend WinEnabler.Here’s how it works:

  1. IIS loads the FastCGI Process Manager (isapi_fcgi.dll) on startup.
  2. The FastCGI Process Manager initializes itself, creating multiple FastCGI processes (php.exe’s in the Task Manager) and waits for a new connection from the Web server.
  3. When a client request comes in, the FastCGI Process Manager selects and opens a connection to a FastCGI process. The server sends the CGI environment variable information and standard input over to the FastCGI process.
  4. Then the FastCGI process completes its processing and sends the standard output and error information back to the server over the same connection.
  5. When the FastCGI process closes the connection, the request is complete. The FastCGI process then waits for another connection from the FastCGI Process Manager running in IIS. In normal CGI, php.exe would be terminated at this point.

In the above example, you can imagine how much slower CGI is. For every web request PHP would need to re-parse php.ini, reload all the dll extensions and reinitialize all data strucures. With FastCGI, all this is done once per process startup. As an extra bonus, persistent database connections also work.

Benchmarks

The stability of PHP in FastCGI mode is exceptional. I have many long scripts which have problems running in ISAPI, but work beautifully in FastCGI and CGI mode. I also tested with apache bench. Here are my results for a test that includes a query to Oracle, using persistent connections, simulating 5 concurrent web client connections, using PHP 4.3.1 and IIS 5 on a W2K server (P3 800 Mhz, 256 Mb RAM) using the following command:ab -c5 -n100 -k http://server/testoracle.phpWith FastCGI: 

Requests per second:  3.33Time per request   :  1.50 secs

The same query with CGI: 

Requests per second:  0.44Time per request   : 11.49 secs

In other words, performance improved by 766% when using FastCGI. This is consistent with Shane’s findings. A stress test with 10,000 requests worked fine with no memory leaks.Note: the time per request is longer than the requests/second would indicate because we tested concurrently with 5 threads querying at the same time. So with FastCGI, although one request completes in 0.211 seconds, because there are 5 threads, the mean time per request across all threads is 0.042 seconds.

Quick Installation on IIS

I found Shane’s instructions a bit confusing if you are not a guru, so i have written this guide to help others get FastCGI working on IIS:

  1. Make sure you have PHP 4.3.x or later installed. Earlier versions of PHP require extra work to get FastCGI working.
  2. This example assumes your cgi application is located in c:\php\php.exe.
  3. Download http://www.caraveo.com/fastcgi/fastcgi-0.6.zip and unpack isapi_fcgi.dll to c:\php\isapi_fcgi.dll.
  4. Create the following registry key with regedit.exe: 
    HKEY_LOCAL_MACHINE:Software\FASTCGI\.php
  5. Then add to this key the following values: 
    	AppPath  = c:\php\php.exe	BindPath = php-fcgi

    Alternatively, you can download and run the sample registry file fastcgi.reg. You might want to change the AppPath of the sample.

  6. Add the application mappings extensions you want to be sent to the fastcgi dll using the IIS configuration screens. To do so:
    1. From the Internet Service Manager (MMC), select the Web site or the starting point directory of an application.
    2. Open the directory’s property sheets (by right clicking and selecting properties), and then click the Home Directory, Virtual Directory, or Directory tab.
    3. Click the Configuration button, and then click the App Mappings tab.
    4. Click Add, set the file extension to .php, and in the Executable box, type: c:\php\isapi_fcgi.dll, and check the Check that file exists checkbox and save the mapping.
    5. Do the same for similar extensions such as .php3 or .phtml.
    6. Then save all changes and restart the web-server.
  7. To test, request for multiple web pages at the same time, and observe the processes in the Task Manager. After the web requests complete, the php.exe processes will continue running.

Personal Notes

Unlike Apache, there appears to be no way to limit the number of requests a FastCGI process (php.exe) can handle, before restarting it. However it appears that process restarts are automatically occurring, as the process id’s are changing slowly over time.To detect whether FastCGI is running, check $_SERVER['FCGI_SERVER_VERSION']. It should hold something like “2.2.2 0.5.2 beta”.bonus link performance php : http://talks.php.net/index.php/Performance

Tags: , , ,
Comment

A HOWTO on Optimizing PHP with tips and methodologies

Filed Under: Performance, PHP by Nguyễn Bá Khoa — Leave a comment
11/08/2010
If you like this article, visit my blog, PHP Everywhere for related articles.

A HOWTO on Optimizing PHP

PHP is a very fast programming language, but there is more to optimizing PHP than just speed of code execution.In this chapter, we explain why optimizing PHP involves many factors which are not code related, and why tuning PHP requires an understanding of how PHP performs in relation to all the other subsystems on your server, and then identifying bottlenecks caused by these subsystems and fixing them. We also cover how to tune and optimize your PHP scripts so they run even faster.Achieving High PerformanceWhen we talk about good performance, we are not talking about how fast your PHP scripts will run. Performance is a set of tradeoffs between scalability and speed. Scripts tuned to use fewer resources might be slower than scripts that perform caching, but more copies of the same script can be run at one time on a web server. In the example below, A.php is a sprinter that can run fast, and B.php is a marathon runner than can jog forever at the nearly the same speed. For light loads, A.php is substantially faster, but as the web traffic increases, the performance of B.php only drops a little bit while A.php just runs out of steam. Let us take a more realistic example to clarify matters further. Suppose we need to write a PHP script that reads a 250K file and generates a HTML summary of the file. We write 2 scripts that do the same thing: hare.php that reads the whole file into memory at once and processes it in one pass, and tortoise.php that reads the file, one line at time, never keeping more than the longest line in memory. Tortoise.php will be slower as multiple reads are issued, requiring more system calls.Hare.php requires 0.04 seconds of CPU and 10 Mb RAM and tortoise.php requires 0.06 seconds of CPU and 5 Mb RAM. The server has 100 Mb free actual RAM and its CPU is 99% idle. Assume no memory fragmentation occurs to simplify things.At 10 concurrent scripts running, hare.php will run out of memory (10 x 10 = 100). At that point, tortoise.php will still have 50 Mb of free memory. The 11th concurrent script to run will bring hare.php to its knees as it starts using virtual memory, slowing it down to maybe half its original speed; each invocation of hare.php now takes 0.08 seconds of CPU time. Meanwhile, tortoise.php will be still be running at its normal 0.06 seconds CPU time.In the table below, the faster php script for different loads is in bold:

Connections CPU seconds required to satisfy 1 HTTP request CPU seconds required to satisfy 10 HTTP requests CPU seconds required to satisfy 11 HTTP requests
hare.php 0.04 0.40 0.88(runs out of RAM)
tortoise.php 0.06 0.60 0.66

As the above example shows, obtaining good performance is not merely writing fast PHP scripts. High performance PHP requires a good understanding of the underlying hardware, the operating system and supporting software such as the web server and database.BottlenecksThe hare and tortoise example has shown us that bottlenecks cause slowdowns. With infinite RAM, hare.php will always be faster than tortoise.php. Unfortunately, the above model is a bit simplistic and there are many other bottlenecks to performance apart from RAM:(a) NetworkingYour network is probably the biggest bottleneck. Let us say you have a 10 Mbit link to the Internet, over which you can pump 1 megabyte of data per second. If each web page is 30k, a mere 33 web pages per second will saturate the line.More subtle networking bottlenecks include frequent access to slow network services such as DNS, or allocating insufficient memory for networking software.(b) CPUIf you monitor your CPU load, sending plain HTML pages over a network will not tax your CPU at all because as we mentioned earlier, the bottleneck will be the network. However for the complex dynamic web pages that PHP generates, your CPU speed will normally become the limiting factor. Having a server with multiple processors or having a server farm can alleviate this.(c) Shared MemoryShared memory is used for inter-process communication, and to store resources that are shared between multiple processes such as cached data and code. If insufficient shared memory is allocated any attempt to access resources that use shared memory such as database connections or executable code will perform poorly.(d) File SystemAccessing a hard disk can be 50 to 100 times slower than reading data from RAM. File caches using RAM can alleviate this. However low memory conditions will reduce the amount of memory available for the file-system cache, slowing things down. File systems can also become heavily fragmented, slowing down disk accesses. Heavy use of symbolic links on Unix systems can slow down disk accesses too.Default Linux installs are also notorious for setting hard disk default settings which are tuned for compatibility and not for speed. Use the command hdparm to tune your Linux hard disk settings.(e) Process ManagementOn some operating systems such as Windows creating new processes is a slow operation. This means CGI applications that fork a new process on every invocation will run substantially slower on these operating systems. Running PHP in multi-threaded mode should improve response times (note: older versions of PHP are not stable in multi-threaded mode).Avoid overcrowding your web server with too many unneeded processes. For example, if your server is purely for web serving, avoid running (or even installing) X-Windows on the machine. On Windows, avoid running Microsoft Find Fast (part of Office) and 3-dimensional screen savers that result in 100% CPU utilization.Some of the programs that you can consider removing include unused networking protocols, mail servers, antivirus scanners, hardware drivers for mice, infrared ports and the like. On Unix, I assume you are accessing your server using SSH. Then you can consider removing:

deamons such as telnetd, inetd, atd, ftpd, lpd, sambadsendmail for incoming mailportmap for NFSxfs, fvwm, xinit, X

You can also disable at startup various programs by modifying the startup files which are usually stored in the /etc/init* or /etc/rc*/init* directory. Also review your cron jobs to see if you can remove them or reschedule them for off-peak periods.(f) Connecting to Other Servers If your web server requires services running on other servers, it is possible that those servers become the bottleneck. The most common example of this is a slow database server that is servicing too many complicated SQL requests from multiple web servers.

When to Start Optimizing?Some people say that it is better to defer tuning until after the coding is complete. This advice only makes sense if your programming team’s coding is of a high quality to begin with, and you already have a good feel of the performance parameters of your application. Otherwise you are exposing yourselves to the risk of having to rewrite substantial portions of your code after testing.My advice is that before you design a software application, you should do some basic benchmarks on the hardware and software to get a feel for the maximum performance you might be able to achieve. Then as you design and code the application, keep the desired performance parameters in mind, because at every step of the way there will be tradeoffs between performance, availability, security and flexibility.Also choose good test data. If your database is expected to hold 100,000 records, avoid testing with only a 100 record database – you will regret it. This once happened to one of the programmers in my company; we did not detect the slow code until much later, causing a lot of wasted time as we had to rewrite a lot of code that worked but did not scale.

Tuning Your Web Server for PHP We will cover how to get the best PHP performance for the two most common web servers in use today, Apache 1.3 and IIS. A lot of the advice here is relevant for serving HTML also. The authors of PHP have stated that there is no performance nor scalability advantage in using Apache 2.0 over Apache 1.3 with PHP, especially in multi-threaded mode. When running Apache 2.0 in pre-forking mode, the following discussion is still relevant (21 Oct 2003).(a) Apache 1.3/2.0 Apache is available on both Unix and Windows. It is the most popular web server in the world. Apache 1.3 uses a pre-forking model for web serving. When Apache starts up, it creates multiple child processes that handle HTTP requests. The initial parent process acts like a guardian angel, making sure that all the child processes are working properly and coordinating everything. As more HTTP requests come in, more child processes are spawned to process them. As the HTTP requests slow down, the parent will kill the idle child processes, freeing up resources for other processes. The beauty of this scheme is that it makes Apache extremely robust. Even if a child process crashes, the parent and the other child processes are insulated from the crashing child. The pre-forking model is not as fast as some other possible designs, but to me that it is “much ado about nothing” on a server serving PHP scripts because other bottlenecks will kick in long before Apache performance issues become significant. The robustness and reliability of Apache is more important. Apache 2.0 offers operation in multi-threaded mode. My benchmarks indicate there is little performance advantage in this mode. Also be warned that many PHP extensions are not compatible (e.g. GD and IMAP). Tested with Apache 2.0.47 (21 Oct 2003).Apache is configured using the httpd.conf file. The following parameters are particularly important in configuring child processes:

Directive Default Description
MaxClients 256 The maximum number of child processes to create. The default means that up to 256 HTTP requests can be handled concurrently. Any further connection requests are queued.
StartServers 5 The number of child processes to create on startup.
MinSpareServers 5 The number of idle child processes that should be created. If the number of idle child processes falls to less than this number, 1 child is created initially, then 2 after another second, then 4 after another second, and so forth till 32 children are created per second.
MaxSpareServers 10 If more than this number of child processes are alive, then these extra processes will be terminated.
MaxRequestsPerChild 0 Sets the number of HTTP requests a child can handle before terminating. Setting to 0 means never terminate. Set this to a value to between 100 to 10000 if you suspect memory leaks are occurring, or to free under-utilized resources.

For large sites, values close to the following might be better:MinSpareServers 32 MaxSpareServers 64Apache on Windows behaves differently. Instead of using child processes, Apache uses threads. The above parameters are not used. Instead we have one parameter: ThreadsPerChild which defaults to 50. This parameter sets the number of threads that can be spawned by Apache. As there is only one child process in the Windows version, the default setting of 50 means only 50 concurrent HTTP requests can be handled. For web servers experiencing higher traffic, increase this value to between 256 to 1024.Other useful performance parameters you can change include:

Directive Default Description
SendBufferSize Set to OS default Determines the size of the output buffer (in bytes) used in TCP/IP connections. This is primarily useful for congested or slow networks when packets need to be buffered; you then set this parameter close to the size of the largest file normally downloaded. One TCP/IP buffer will be created per client connection.
KeepAlive [on|off] On In the original HTTP specification, every HTTP request had to establish a separate connection to the server. To reduce the overhead of frequent connects, the keep-alive header was developed. Keep-alives tells the server to reuse the same socket connection for multiple HTTP requests. If a separate dedicated web server serves all images, you can disable this option. This technique can substantially improve resource utilization.
KeepAliveTimeout 15 The number of seconds to keep the socket connection alive. This time includes the generation of content by the server and acknowledgements by the client. If the client does not respond in time, it must make a new connection. This value should be kept low as the socket will be idle for extended periods otherwise.
MaxKeepAliveRequests 100 Socket connections will be terminated when the number of requests set by MaxKeepAliveRequests is reached. Keep this to a high value below MaxClients or ThreadsPerChild.
TimeOut 300 Disconnect when idle time exceeds this value. You can set this value lower if your clients have low latencies.
LimitRequestBody 0 Maximum size of a PUT or POST. O means there is no limit.

If you do not require DNS lookups and you are not using the htaccess file to configure Apache settings for individual directories you can set:# disable DNS lookups: PHP scripts only get the IP addressHostnameLookups off # disable htaccess checks<Directory /> AllowOverride none </Directory> If you are not worried about the directory security when accessing symbolic links, turn on FollowSymLinks and turn off SymLinksIfOwnerMatch to prevent additional lstat() system calls from being made:Options FollowSymLinks #Options SymLinksIfOwnerMatch (b) IIS TuningIIS is a multi-threaded web server available on Windows NT and 2000. From the Internet Services Manager, it is possible to tune the following parameters:

Performance Tuning based on the number of hits per day. Determines how much memory to preallocate for IIS. (Performance Tab).
Bandwidth throttling Controls the bandwidth per second allocated per web site. (Performance Tab).
Process throttling Controls the CPU% available per Web site. (Performance Tab).
Timeout Default is 900 seconds. Set to a lower value on a Local Area Network. (Web Site Tab)
HTTP Compression In IIS 5, you can compress dynamic pages, html and images. Can be configured to cache compressed static html and images. By default compression is off.HTTP compression has to be enabled for the entire physical server. To turn it on open the IIS console, right-click on the server (not any of the subsites, but the server in the left-hand pane), and get Properties. Click on the Service tab, and select “Compress application files” to compress dynamic content, and “Compress static files” to compress static content.

You can also configure the default isolation level of your web site. In the Home Directory tab under Application Protection, you can define your level of isolation. A highly isolated web site will run slower because it is running as a separate process from IIS, while running web site in the IIS process is the fastest but will bring down the server if there are serious bugs in the web site code. Currently I recommend running PHP web sites using CGI, or using ISAPI with Application Protection set to high.You can also use regedit.exe to modify following IIS 5 registry settings stored at the following location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Inetinfo\Parameters\

MemCacheSize Sets the amount of memory that IIS will use for its file cache. By default IIS will use 50% of available memory. Increase if IIS is the only application on the server. Value is in megabytes.
MaxCachedFileSize Determines the maximum size of a file cached in the file cache in bytes. Default is 262,144 (256K).
ObjectCacheTTL Sets the length of time (in milliseconds) that objects in the cache are held in memory. Default is 30,000 milliseconds (30 seconds).
MaxPoolThreads Sets the number of pool threads to create per processor. Determines how many CGI applications can run concurrently. Default is 4. Increase this value if you are using PHP in CGI mode.
ListenBackLog Specifies the maximum number of active Keep Alive connections that IIS maintains in the connection queue. Default is 15, and should be increased to the number of concurrent connections you want to support. Maximum is 250.

If the settings are missing from this registry location, the defaults are being used.High Performance on Windows: IIS and FastCGIAfter much testing, I find that the best PHP performance on Windows is offered by using IIS with FastCGI. CGI is a protocol for calling external programs from a web server. It is not very fast because CGI programs are terminated after every page request. FastCGI modifies this protocol for high performance, by making the CGI program persist after a page request, and reusing the same CGI program when a new page request comes in.As the installation of FastCGI with IIS is complicated, you should use the EasyWindows PHP Installer. This will install PHP, FastCGI and Turck MMCache for the best performance possible. This installer can also install PHP for Apache 1.3/2.0.This section on FastCGI added 21 Oct 2003.PHP4′s Zend EngineThe Zend Engine is the internal compiler and runtime engine used by PHP4. Developed by Zeev Suraski and Andi Gutmans, the Zend Engine is an abbreviation of their names. In the early days of PHP4, it worked in the following fashion:The PHP script was loaded by the Zend Engine and compiled into Zend opcode. Opcodes, short for operation codes, are low level binary instructions. Then the opcode was executed and the HTML generated sent to the client. The opcode was flushed from memory after execution.Today, there are a multitude of products and techniques to help you speed up this process. In the following diagram, we show the how modern PHP scripts work; all the shaded boxes are optional. PHP Scripts are loaded into memory and compiled into Zend opcodes. These opcodes can now be optimized using an optional peephole optimizer called Zend Optimizer. Depending on the script, it can increase the speed of your PHP code by 0-50%. Formerly after execution, the opcodes were discarded. Now the opcodes can be optionally cached in memory using several alternative open source products and the Zend Accelerator (formerly Zend Cache), which is a commercial closed source product. The only opcode cache that is compatible with the Zend Optimizer is the Zend Accelerator. An opcode cache speeds execution by removing the script loading and compilation steps. Execution times can improve between 10-200% using an opcode cache.

Where to find Opcode CachesZend Accelerator: A commercial opcode cache developed by the Zend Engine team. Very reliable and robust. Visit http://zend.com for more information.You will need to test the following open source opcode caches before using them on production servers as their performance and reliability very much depends on the PHP scripts you run.Turcke MMCache: http://turck-mmcache.sourceforge.net/ is no longer maintained. See eAccelerator, which is a branch of mmcache that is actively maintained (Added 28 Feb 2005).Alternative PHP Cache: http://apc.communityconnect.com/PHP Accelerator: http://www.php-accelerator.co.uk/AfterBurner Cache: http://www.bwcache.bware.it/

One of the secrets of high performance is not to write faster PHP code, but to avoid executing PHP code by caching generated HTML in a file or in shared memory. The PHP script is only run once and the HTML is captured, and future invocations of the script will load the cached HTML. If the data needs to be updated regularly, an expiry value is set for the cached HTML. HTML caching is not part of the PHP language nor Zend Engine, but implemented using PHP code. There are many class libraries that do this. One of them is the PEAR Cache, which we will cover in the next section. Another is the Smarty template library.Finally, the HTML sent to a web client can be compressed. This is enabled by placing the following code at the beginning of your PHP script:<?phpob_start(“ob_gzhandler”); ::?>If your HTML is highly compressible, it is possible to reduce the size of your HTML file by 50-80%, reducing network bandwidth requirements and latencies. The downside is that you need to have some CPU power to spare for compression.HTML Caching with PEAR CacheThe PEAR Cache is a set of caching classes that allows you to cache multiple types of data, including HTML and images.The most common use of the PEAR Cache is to cache HTML text. To do this, we use the Output buffering class which caches all text printed or echoed between the start() and end() functions: require_once(“Cache/Output.php”);$cache = new Cache_Output(“file”, array(“cache_dir” => “cache/”) );if ($contents = $cache->start(md5(“this is a unique key!”))) {## aha, cached data returned# print $contents; print “<p>Cache Hit</p>”;} else {## no cached data, or cache expired# print “<p>Don’t leave home without it…</p>”; # place in cache print “<p>Stand and deliver</p>”; # place in cache print $cache->end(10);}Since I wrote these lines, a superior PEAR cache system has been developed: Cache Lite; and for more sophisticated distributed caching, see memcached (Added 28 Feb 2005). The Cache constructor takes the storage driver to use as the first parameter. File, database and shared memory storage drivers are available; see the pear/Cache/Container directory. Benchmarks by Ulf Wendel suggest that the “file” storage driver offers the best performance. The second parameter is the storage driver options. The options are “cache_dir”, the location of the caching directory, and “filename_prefix”, which is the prefix to use for all cached files. Strangely enough, cache expiry times are not set in the options parameter.To cache some data, you generate a unique id for the cached data using a key. In the above example, we used md5(“this is a unique key!”).The start() function uses the key to find a cached copy of the contents. If the contents are not cached, an empty string is returned by start(), and all future echo() and print() statements will be buffered in the output cache, until end() is called.The end() function returns the contents of the buffer, and ends output buffering. The end() function takes as its first parameter the expiry time of the cache. This parameter can be the seconds to cache the data, or a Unix integer timestamp giving the date and time to expire the data, or zero to default to 24 hours.Another way to use the PEAR cache is to store variables or other data. To do so, you can use the base Cache class:<?phprequire_once(“Cache.php”);$cache = new Cache(“file”, array(“cache_dir” => “cache/”) );$id = $cache->generateID(“this is a unique key”);if ($data = $cache->get($id)) { print “Cache hit.<br>Data: $data”;} else { $data = “The quality of mercy is not strained…”; $cache->save($id, $data, $expires = 60); print “Cache miss.<br>”;}?>To save the data we use save(). If your unique key is already a legal file name, you can bypass the generateID() step. Objects and arrays can be saved because save() will serialize the data for you. The last parameter controls when the data expires; this can be the seconds to cache the data, or a Unix integer timestamp giving the date and time to expire the data, or zero to use the default of 24 hours. To retrieve the cached data we use get().You can delete a cached data item using $cache->delete($id) and remove all cached items using $cache->flush().New: A faster Caching class is Cache-Lite. Highly recommended.Using BenchmarksIn earlier section we have covered many performance issues. Now we come to the meat and bones, how to go about measuring and benchmarking your code so you can obtain decent information on what to tune.If you want to perform realistic benchmarks on a web server, you will need a tool to send HTTP requests to the server. On Unix, common tools to perform benchmarks include ab (short for apachebench) which is part of the Apache release, and the newer flood (httpd.apache.org/test/flood). On Windows NT/2000 you can use Microsoft’s free Web Application Stress Tool (webtool.rte.microsoft.com).These programs can make multiple concurrent HTTP requests, simulating multiple web clients, and present you with detailed statistics on completion of the tests.You can monitor how your server behaves as the benchmarks are conducted on Unix using “vmstat 1″. This prints out a status report every second on the performance of your disk i/o, virtual memory and CPU load. Alternatively, you can use “top d 1″ which gives you a full screen update on all processes running sorted by CPU load every 1 second.On Windows 2000, you can use the Performance Monitor or the Task Manager to view your system statistics.If you want to test a particular aspect of your code without having to worry about the HTTP overhead, you can benchmark using the microtime(), which returns the current time accurate to the microsecond as a string. The following function will convert it into a number suitable for calculations. function getmicrotime(){ list($usec, $sec) = explode(” “,microtime()); return ((float)$usec + (float)$sec);}$time = getmicrotime();## benchmark code here# echo “<p>Time elapsed: “,getmicrotime() – $time, ” seconds”; Alternatively, you can use a profiling tool such as APD or XDebug. Also see my article squeezing code with xdebug.Benchmarking Case StudyThis case study details a real benchmark we did for a client. In this instance, the customer wanted a guaranteed response time of 5 seconds for all PHP pages that did not involve running long SQL queries. The following server configuration was used: an Apache 1.3.20 server running PHP 4.0.6 on Red Hat 7.2 Linux. The hardware was a twin Pentium III 933 MHz beast with 1 Gb of RAM. The HTTP requests will be for the PHP script “testmysql.php”. This script reads and processes about 20 records from a MySQL database running on another server. For the sake of simplicity, we assume that all graphics are downloaded from another web server.We used “ab” as the benchmarking tool. We set “ab” to perform 1000 requests (-n1000), using 10 simultaneous connections (-c10). Here are the results: 

# ab   -n1000 -c10 http://192.168.0.99/php/testmysql.phpThis is ApacheBench, Version 1.3Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/Server Software:        Apache/1.3.20Server Hostname:        192.168.0.99Server Port:            80Document Path:          /php/testmysql.phpDocument Length:        25970 bytesConcurrency Level:      10Time taken for tests:   128.672 secondsComplete requests:      1000Failed requests:        0Total transferred:      26382000 bytesHTML transferred:       25970000 bytesRequests per second:    7.77Transfer rate:          205.03 kb/s receivedConnnection Times (ms)              min   avg   maxConnect:        0     9   114Processing:   698  1274  2071Total:        698  1283  2185

While running the benchmark, on the server side we monitored the resource utilization using the command “top d 1″. The parameters “d 1″ mean to delay 1 second between updates. The output is shown below. 

10:58pm  up  3:36,  2 users,  load average: 9.07, 3.29, 1.7974 processes: 63 sleeping, 11 running, 0 zombie, 0 stoppedCPU0 states: 92.0% user,  7.0% system,  0.0% nice,  0.0% idleCPU1 states: 95.0% user,  4.0% system,  0.0% nice,  0.0% idleMem:  1028484K av,  230324K used,  798160K free,      64K shrd,   27196K buffSwap: 2040244K av,       0K used, 2040244K free                   30360K cached  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND 1142 apache    20   0  7280 7280  3780 R    21.2  0.7   0:20 httpd 1154 apache    17   0  8044 8044  3788 S    19.3  0.7   0:20 httpd 1155 apache    20   0  8052 8052  3796 R    19.3  0.7   0:20 httpd 1141 apache    15   0  6764 6764  3780 S    14.7  0.6   0:20 httpd 1174 apache    14   0  6848 6848  3788 S    12.9  0.6   0:20 httpd 1178 apache    13   0  6864 6864  3804 S    12.9  0.6   0:19 httpd 1157 apache    15   0  7536 7536  3788 R    11.0  0.7   0:19 httpd 1159 apache    15   0  7540 7540  3788 R    11.0  0.7   0:19 httpd 1148 apache    11   0  6672 6672  3784 S    10.1  0.6   0:20 httpd 1158 apache    14   0  7400 7400  3788 R    10.1  0.7   0:19 httpd 1163 apache    20   0  7540 7540  3788 R    10.1  0.7   0:19 httpd 1169 apache    12   0  6856 6856  3796 S    10.1  0.6   0:20 httpd 1176 apache    16   0  8052 8052  3796 R    10.1  0.7   0:19 httpd 1171 apache    15   0  7984 7984  3780 S     9.2  0.7   0:18 httpd 1170 apache    16   0  7204 7204  3796 R     6.4  0.7   0:20 httpd 1168 apache    10   0  6856 6856  3796 S     4.6  0.6   0:20 httpd 1377 natsoft   11   0  1104 1104   856 R     2.7  0.1   0:02 top 1152 apache     9   0  6752 6752  3788 S     1.8  0.6   0:20 httpd 1167 apache     9   0  6848 6848  3788 S     0.9  0.6   0:19 httpd    1 root       8   0   520  520   452 S     0.0  0.0   0:04 init    2 root       9   0     0    0     0 SW    0.0  0.0   0:00 keventd

Looking at the output of “top”, the twin CPU Apache server is running flat out with 0% idle time. What is worse is that the load average is 9.07 for the past minute (and 3.29 for the past 5 minutes, 1.79 for the past 15 minutes). The load average is the average number of processes that are ready to be run. For a twin processor server, any load above 2.0 means that the system is being overloaded. You might notice that there is a close relationship between load (9.07) and the number of simultaneous connections (10) that we have defined with ab.Luckily we have plenty of physical memory, with about 798,160 Mb free and no virtual memory used.Further down we can see the processes ordered by CPU utilization. The most active ones are the Apache httpd processes. The first httpd task is using 7280K of memory, and is taking an average of 21.2% of CPU and 0.7% of physical memory. The STAT column indicates the status: R is runnable, S is sleeping, and W means that the process is swapped out. Given the above figures, and assuming this a typical peak load, we can perform some planning. If the load average is 9.0 for a twin-CPU server and assuming each task takes about the same amount of time to complete, then a lightly loaded server should be 9.0 / 2 CPUs = 4.5 times faster. So a HTTP request that used to take 1.283 seconds to satisfy at peak load will take about 1.283 / 4.5 = 0.285 seconds to complete. To verify this, we benchmarked with 2 simultaneous client connections (instead of 10 in the previous benchmark) to give an average of 0.281 seconds, very close to the 0.285 seconds prediction! 

# ab   -n100 -c2 http://192.168.0.99/php/testmysql.php [ some lines omitted for brevity ]Requests per second:    7.10Transfer rate:          187.37 kb/s receivedConnnection Times (ms)              min   avg   maxConnect:        0     2    40Processing:   255   279   292Total:        255   281   332

Conversely, doubling the connections, we can predict that the average connection time should double from 1.283 to 2.566 seconds. In the benchmarks, the actual time was 2.570 seconds.Overload on 40 connectionsWhen we pushed the benchmark to use 40 connections, the server overloaded with 35% failed requests. On further investigation, it was because the MySQL server persistent connects were failing because of “Too Many Connections”.The benchmark also demonstrates the lingering behavior of Apache child processes. Each PHP script uses 2 persistent connections, so at 40 connections, we should only be using at most 80 persistent connections, well below the default MySQL max_connections of 100. However Apache idle child processes are not assigned immediately to new requests due to latencies, keep-alives and other technical reasons; these lingering child processes held the remaining 20+ persistent connections that were “the straws that broke the Camel’s back”.The FixBy switching to non-persistent database connections, we were able to fix this problem and obtained a result of 5.340 seconds. An alternative solution would have been to increase the MySQL max_connections parameter from the default of 100.ConclusionsThe above case study once again shows us that optimizing your performance is extremely complex. It requires an understanding of multiple software subsystems including network routing, the TCP/IP stack, the amount of physical and virtual memory, the number of CPUs, the behavior of Apache child processes, your PHP scripts, and the database configuration.In this case the PHP code was quite well tuned, so the first bottleneck was the CPU, which caused a slowdown in response time. As the load increased, the system slowed down in a near linear fashion (which is a good sign) until we encountered the more serious bottleneck of MySQL client connections. This caused multiple errors in our PHP pages until we fixed it by switching to non-persistent connections.From the above figures, we can calculate for a given desired response time, how many simultaneous HTTP connections we can handle. Assuming two-way network latencies of 0.5 seconds on the Internet (0.25s one way), we can predict:As our client wanted a maximum response time of 5 seconds, the server can handle up to 34 simultaneous connections per second. This works out to a peak capacity of 34/5 = 6.8 page views per second.To get the maximum number of page views a day that the server can handle, multiply the peak capacity per second by 50,000 (this technique is suggested by the webmasters at pair.com, a large web hosting company), to give 340,000 page views a day.Code OptimizationsThe patient reader who is still wondering why so much emphasis is given to discussing non-PHP issues is reminded that PHP is a fast language, and many of the likely bottlenecks causing slow speeds lie outside PHP.Most PHP scripts are simple. They involve reading some session information, loading some data from a content management system or database, formatting the appropriate HTML and echoing the results to the HTTP client. Assuming that a typical PHP script completes in 0.1 seconds and the Internet latency is 0.2 seconds, only 33% of the 0.3 seconds response time that the HTTP client sees is actual PHP computation. So if you improve a script’s speed by 20%, the HTTP client will see response times drop to 0.28 seconds, which is an insignificant improvement. Of course the server can probably handle 20% more requests for the same page, so scalability has improved.The above example does not mean we should throw our hands up and give up. It means that we should not feel proud tweaking the last 1% of speed from our code, but we should spend our time optimizing worthwhile areas of our code to get higher returns.High Return Code OptimizationsThe places where such high returns are achievable are in the while and for loops that litter our code, where each slowdown in the code is magnified by the number of times we iterate over them. The best way of understanding what can be optimized is to use a few examples:Example 1Here is one simple example that prints an array:for ($j=0; $j<sizeof($arr); $j++) echo $arr[$j].”<br>”;This can be substantially speeded up by changing the code to:for ($j=0, $max = sizeof($arr), $s = ”; $j<$max; $j++)$s .= $arr[$j].”<br>”;echo $s; First we need to understand that the expression $j<sizeof($arr) is evaluated within the loop multiple times. As sizeof($arr) is actually a constant (invariant), we move the cache the sizeof($arr) in the $max variable. In technical terms, this is called loop invariant optimization. The second issue is that in PHP 4, echoing multiple times is slower than storing everything in a string and echoing it in one call. This is because echo is an expensive operation that could involve sending TCP/IP packets to a HTTP client. Of course accumulating the string in $s has some scalability issues as it will use up more memory, so you can see a trade-off is involved here.An alternate way of speeding the above code would be to use output buffering. This will accumulate the output string internally, and send the output in one shot at the end of the script. This reduces networking overhead substantially at the cost of more memory and an increase in latency. In some of my code consisting entirely of echo statements, performance improvements of 15% have been observed. ob_start();for ($j=0, $max = sizeof($arr), $s = ”; $j<$max; $j++)echo $arr[$j].”<br>”; Note that output buffering with ob_start() can be used as a global optimization for all PHP scripts. In long-running scripts, you will also want to flush the output buffer periodically so that some feedback is sent to the HTTP client. This can be done with ob_end_flush(). This function also turns off output buffering, so you might want to call ob_start() again immediately after the flush. In summary, this example has shown us how to optimize loop invariants and how to use output buffering to speed up our code.Example 2In the following code, we iterate through a PEAR DB recordset, using a special formatting function to format a row, and then we echo the results. This time, I benchmarked the execution time at 10.2 ms (this excludes the database connection and SQL execution time):function FormatRow(&$recordSet){$arr = $recordSet->fetchRow();return ‘<b>’.$arr[0].’</b><i>’.$arr[1].’</i>’;}for ($j = 0; $j < $rs->numRows(); $j++) {print FormatRow($rs);}From example 1, we learnt that we can optimize the code by changing the code to the following (execution time: 8.7 ms):function FormatRow(&$recordSet){$arr = $recordSet->fetchRow();return ‘<b>’.$arr[0].’</b><i>’.$arr[1].’</i>’;}ob_start();for ($j = 0, $max = $rs->numRows(); $j < $max; $j++) {print FormatRow($rs);}My benchmarks showed me that the use of $max contributed 0.5 ms and ob_start contributed 1 ms to the 1.5 ms speedup.However by changing the looping algorithm we can simplify and speed up the code. In this case, execution time is reduced to 8.5 ms:function FormatRow($arr){return ‘<b>’.$arr[0].’</b><i>’.$arr[1].</i>’;}ob_start();while ($arr = $rs->fetchRow()) {print FormatRow($arr);}One last optimization is possible here. We can remove the overhead of the function call (potentially sacrificing maintainability for speed) to shave off another 0.1 milliseconds (execution time: 8.4 ms):ob_start();while ($arr = $rs->fetchRow()) {print ‘<b>’.$arr[0].’</b><i>’.$arr[1].’</i>’;}By switching to PEAR Cache, execution time dropped again to 3.5 ms for cached data:require_once(“Cache/Output.php”);ob_start();$cache = new Cache_Output(“file”, array(“cache_dir” => “cache/”) );$t = getmicrotime();if ($contents = $cache->start(md5(“this is a unique kexy!”))) {print “<p>Cache Hit</p>”;print $contents;} else {print “<p>Cache Miss</p>”;#### Code to connect and query database omitted## while ($arr = $rs->fetchRow()) {print ‘<b>’.$arr[0].’</b><i>’.$arr[1].’</i>’;} print $cache->end(100);}print (getmicrotime()-$t);We summarize the optimization methods below:

ExecutionTime (ms) Optimization Method
9.9 Initial code, no optimizations, excluding database connection and SQL execution times.
9.2 Using ob_start
8.7 Optimizing loop invariants ($max) and using ob_start
8.5 Changing from for-loop to while-loop, and passing an array to FormatRow()and using ob_start
8.4 Removing FormatRow()and using ob_start
3.5 Using PEAR Cache and using ob_start

From the above figures, you can see that biggest speed improvements are derived not from tweaking the code, but by simple global optimizations such as ob_start(), or using radically different algorithms such as HTML caching.

Optimizing Object-oriented Programming In March 2001, I conducted some informal benchmarks with classes on PHP 4.0.4pl1, and I derived some advice from the results. The three main points are: 1. Initialise all variables before use. 2. Dereference all global/property variables that are frequently used in a method and put the values in local variables if you plan to access the value more than twice. 3. Try placing frequently used methods in the derived classes. Warning: as PHP is going through a continuous improvement process, things might change in the future. More Details I have found that calling object methods (functions defined in a class) are about twice as slow as a normal function calls. To me that’s quite acceptable and comparable to other OOP languages. Inside a method (the following ratios are approximate only):
  1. Incrementing a local variable in a method is the fastest. Nearly the same as calling a local variable in a function.
  2. Incrementing a global variable is 2 times slow than a local var.
  3. Incrementing a object property (eg. $this->prop++) is 3 times slower than a local variable.
  4. Incrementing an undefined local variable is 9-10 times slower than a pre-initialized one.
  5. Just declaring a global variable without using it in a function also slows things down (by about the same amount as incrementing a local var). PHP probably does a check to see if the global exists.
  6. Method invocation appears to be independent of the number of methods defined in the class because I added 10 more methods to the test class (before and after the test method) with no change in performance.
  7. Methods in derived classes run faster than ones defined in the base class.
  8. A function call with one parameter and an empty function body takes about the same time as doing 7-8 $localvar++ operations. A similar method call is of course about 15 $localvar++ operations.Update: 11 July 2004: The above test was on PHP 4.0.4, about 3 years ago. I tested this again in PHP4.3.3 and calling a function now takes about 20 $localvar++ operations, and calling a method takes about 30 $localvar++ operations. This could be because $localvar++ runs faster now, or functions are slower.

Summary of Tweaks

  1. The more you understand the software you are using (Apache, PHP, IIS, your database) and the deeper your knowledge of the operating system, networking and server hardware, the better you can perform global optimizations on your code and your system.
  2. For PHP scripts, the most expensive bottleneck is normally the CPU. Twin CPUs are probably more useful than two Gigabytes of RAM.
  3. Compile PHP with the “configure –-enable-inline-optimization” option to generate the fastest possible PHP executable.
  4. Tune your database and index the fields that are commonly used in your SQL WHERE criteria. ADOdb, the very popular database abstraction library, provides a SQL tuning mode, where you can view your invalid, expensive and suspicious SQL, their execution plans and in which PHP script the SQL was executed.
  5. Use HTML caching if you have data that rarely changes. Even if the data changes every minute, caching can help provided the data is synchronized with the cache. Depending on your code complexity, it can improve your performance by a factor of 10.
  6. Benchmark your most complex code early (or at least a prototype), so you get a feel of the expected performance before it is too late to fix. Try to use realistic amounts of test data to ensure that it scales properly.Updated 11 July 2004: To benchmark with an execution profile of all function calls, you can try the xdebug extension. For a brief tutorial of how i use xdebug, see squeezing code with xdebug. There are commercial products to do this also, eg. Zend Studio.
  7. Consider using a opcode cache. This gives a speedup of between 10-200%, depending on the complexity of your code. Make sure you do some stress tests before you install a cache because some are more reliable than others.
  8. Use ob_start() at the beginning of your code. This gives you a 5-15% boost in speed for free on Apache. You can also use gzip compression for extra fast downloads (this requires spare CPU cycles).
  9. Consider installing Zend Optimizer. This is free and does some optimizations, but be warned that some scripts actually slow down when Zend Optimizer is installed. The consensus is that Zend Optimizer is good when your code has lots of loops. Today many opcode accelerators have similar features (added this sentence 21 Oct 2003).
  10. Optimize your loops first. Move loop invariants (constants) outside the loop.
  11. Use the array and string functions where possible. They are faster than writing equivalent code in PHP.

  12. The fastest way to concatenate multiple small strings into one large string is to create an output buffer (ob_start) and to echo into the buffer. At the end get the contents using ob_get_contents. This works because memory allocation is normally the killer in string concatenation, and output buffering allocates a large 40K initial buffer that grows in 10K chunks. Added 22 June 2004.

  13. Pass objects and arrays using references in functions. Return objects and arrays as references where possible also. If this is a short script, and code maintenance is not an issue, you can consider using global variables to hold the objects or arrays.

  14. If you have many PHP scripts that use session variables, consider recompiling PHP using the shared memory module for sessions, or use a RAM Disk. Enable this with “configure -–with-mm” then re-compile PHP, and set session.save_handler=mm in php.ini.
  15. For searching for substrings, the fastest code is using strpos(), followed by preg_match() and lastly ereg(). Similarly, str_replace() is faster than preg_replace(), which is faster than ereg_replace().
  16. Added 11 July 2004: Order large switch statements with most frequently occuring cases on top. If some of the most common cases are in the default section, consider explicitly defining these cases at the top of the switch statement.
  17. For processing XML, parsing with regular expressions is significantly faster than using DOM or SAX.
  18. Unset() variables that are not used anymore to reduce memory usage. This is mostly useful for resources and large arrays.
  19. For classes with deep hierarchies, functions defined in derived classes (child classes) are invoked faster than those defined in base class (parent class). Consider replicating the most frequently used code in the base class in the derived classes too.
  20. Consider writing your code as a PHP extension or a Java class or a COM object if your need that extra bit of speed. Be careful of the overhead of marshalling data between COM and Java.

Useless Optimizations Some optimizations are useful. Others are a waste of time – sometimes the improvement is neglible, and sometimes the PHP internals change, rendering the tweak obsolete. Here are some common PHP legends: a. echo is faster than print Echo is supposed to be faster because it doesn’t return a value while print does. From my benchmarks with PHP 4.3, the difference is neglible. And under some situations, print is faster than echo (when ob_start is enabled). b. strip off comments to speed up code If you use an opcode cache, comments are already ignored. This is a myth from PHP3 days, when each line of PHP was interpreted in run-time. c. ‘var=’.$var is faster than “var=$var” This used to be true in PHP 4.2 and earlier. This was fixed in PHP 4.3. Note (22 June 2004): apparently the 4.3 fix reduced the overhead, but not completely. However I find the performance difference to be negligible.

Do References Speed Your Code?References do not provide any performance benefits for strings, integers and other basic data types. For example, consider the following code: 
function TestRef(&$a){    $b = $a;    $c = $a;}$one = 1;ProcessArrayRef($one);

And the same code without references: 

function TestNoRef($a){    $b = $a;    $c = $a;}$one = 1;ProcessArrayNoRef($one);

PHP does not actually create duplicate variables when “pass by value” is used, but uses high speed reference counting internally. So in TestRef(), $b and $c take longer to set because the references have to be tracked, while in TestNoRef(), $b and $c just point to the original value of $a, and the reference counter is incremented. So TestNoRef() will execute faster than TestRef(). In contrast, functions that accept array and object parameters have a performance advantage when references are used. This is because arrays and objects do not use reference counting, so multiple copies of an array or object are created if “pass by value” is used. So the following code: function ObjRef(&$o){$a =$o->name;}is faster than:$function ObjRef($o){ $a = $o->name;}Note: In PHP 5, all objects are passed by reference automatically, without the need of an explicit & in the parameter list. PHP 5 object performance should be significantly faster.

Many thanks also to Andrei Zmievski for reviewing this article. (c) 2001-2005 John Lim. No reproduction of this articleis permitted without written permission from the author.

Tags: , , ,
Comment

Top 84 MySQL Performance Tips

Filed Under: Mysql, Performance by Nguyễn Bá Khoa — Leave a comment
11/08/2010

MySQL is a widely used and fast SQL database server. It is a client/server implementation that consists of a server daemon (mysqld) and many different client programs/libraries.

You can check the same tips from here.Here is very useful tips for all mysql DBA’s,Developers these tips are noted from MySQL Camp 2006 suggested by mysql community experts.

1. Kaj (Most Excellent Obvious Facilitator) Index stuff.
2. Ronald Don’t Index Everything
3. Use benchmarking
4. Minimize traffic by fetching only what you need.
1. Paging/chunked data retrieval to limit
2. Don’t use SELECT *
3. Be wary of lots of small quick queries if a longer query can be more efficient
5. Use EXPLAIN to profile the query execution plan
6. Use Slow Query Log (always have it on!)
7. Don’t use DISTINCT when you have or could use GROUP BY
8. Use proper data partitions
1. For Cluster. Start thinking about Cluster *before* you need them
9. Insert performance
1. Batch INSERT and REPLACE
2. Use LOAD DATA instead of INSERT
10. LIMIT m,n may not be as fast as it sounds
11. Don’t use ORDER BY RAND() if you have > ~2K records
12. Use SQL_NO_CACHE when you are SELECTing frequently updated data or large sets of data
13. avoid wildcards at the start of LIKE queries
14. avoid correlated subqueries and in select and where clause (try to avoid in)
15. config params –
16. no calculated comparisons — isolate indexed columns
17. innodb_flush_commit=0 can help slave lag
18. ORDER BY and LIMIT work best with equalities and covered indexes
19. isolate workloads don’t let administrative work interfere with customer performance. (ie backups)
20. use optimistic locking, not pessimistic locking. try to use shared lock, not exclusive lock. share mode vs. FOR UPDATE
21. use row-level instead of table-level locking for OLTP workloads
22. Know your storage engines and what performs best for your needs, know that different ones exist.
1. use MERGE tables ARCHIVE tables for logs
23. Optimize for data types, use consistent data types. Use PROCEDURE ANALYSE() to help determine if you need less
24. separate text/blobs from metadata, don’t put text/blobs in results if you don’t need them
25. if you can, compress text/blobs
26. compress static data
27. don’t back up static data as often
28. derived tables (subqueries in the FROM clause) can be useful for retrieving BLOBs w/out sorting them. (self-join can speed up a query if 1st part finds the IDs and use it to fetch the rest)
29. enable and increase the query and buffer caches if appropriate
30. ALTER TABLE…ORDER BY can take chronological data and re-order it by a different field
31. InnoDB ALWAYS keeps the primary key as part of each index, so do not make the primary key very large, be careful of redundant columns in an index, and this can make the query faster
32. Do not duplicate indexes
33. Utilize different storage engines on master/slave ie, if you need fulltext indexing on a table.
34. BLACKHOLE engine and replication is much faster than FEDERATED tables for things like logs.
35. Design sane query schemas. don’t be afraid of table joins, often they are faster than denormalization
36. Don’t use boolean flags
37. Use a clever key and ORDER BY instead of MAX
38. Keep the database host as clean as possible. Do you really need a windowing system on that server?
39. Utilize the strengths of the OS
40. Hire a MySQL ™ Certified DBA
41. Know that there are many consulting companies out there that can help, as well as MySQL’s Professional Services.
42. Config variables & tips:
1. use one of the supplied config files
2. key_buffer, unix cache (leave some RAM free), per-connection variables, innodb memory variables
3. be aware of global vs. per-connection variables
4. check SHOW STATUS and SHOW VARIABLES (GLOBAL|SESSION in 5.0 and up)
5. be aware of swapping esp. with Linux, “swappiness” (bypass OS filecache for innodb data files, innodb_flush_method=O_DIRECT if possible (this is also OS specific))
6. defragment tables, rebuild indexes, do table maintenance
7. If you use innodb_flush_txn_commit=1, use a battery-backed hardware cache write controller
8. more RAM is good so faster disk speed
9. use 64-bit architectures
43. Know when to split a complex query and join smaller ones
44. Debugging sucks, testing rocks!
45. Delete small amounts at a time if you can
46. Archive old data — don’t be a pack-rat! 2 common engines for this are ARCHIVE tables and MERGE tables
47. use INET_ATON and INET_NTOA for IP addresses, not char or varchar
48. make it a habit to REVERSE() email addresses, so you can easily search domains
49. –skip-name-resolve
50. increase myisam_sort_buffer_size to optimize large inserts (this is a per-connection variable)
51. look up memory tuning parameter for on-insert caching
52. increase temp table size in a data warehousing environment (default is 32Mb) so it doesn’t write to disk (also constrained by max_heap_table_size, default 16Mb)
53. Normalize first, and denormalize where appropriate.
54. Databases are not spreadsheets, even though Access really really looks like one. Then again, Access isn’t a real database
55. In 5.1 BOOL/BIT NOT NULL type is 1 bit, in previous versions it’s 1 byte.
56. A NULL data type can take more room to store than NOT NULL
57. Choose appropriate character sets & collations — UTF16 will store each character in 2 bytes, whether it needs it or not, latin1 is faster than UTF8.
58. make similar queries consistent so cache is used
59. Have good SQL query standards
60. Don’t use deprecated features
61. Use Triggers wisely
62. Run in SQL_MODE=STRICT to help identify warnings
63. Turning OR on multiple index fields (<5.0) into UNION may speed things up (with LIMIT), after 5.0 the index_merge should pick stuff up.
64. /tmp dir on battery-backed write cache
65. consider battery-backed RAM for innodb logfiles
66. use min_rows and max_rows to specify approximate data size so space can be pre-allocated and reference points can be calculated.
67. as your data grows, indexing may change (cardinality and selectivity change). Structuring may want to change. Make your schema as modular as your code. Make your code able to scale. Plan and embrace change, and get developers to do the same.
68. pare down cron scripts
69. create a test environment
70. try out a few schemas and storage engines in your test environment before picking one.
71. Use HASH indexing for indexing across columns with similar data prefixes
72. Use myisam_pack_keys for int data
73. Don’t use COUNT * on Innodb tables for every search, do it a few times and/or summary tables, or if you need it for the total # of rows, use SQL_CALC_FOUND_ROWS and SELECT FOUND_ROWS()
74. use –safe-updates for client
75. Redundant data is redundant
76. Use INSERT … ON DUPLICATE KEY update (INSERT IGNORE) to avoid having to SELECT
77. use groupwise maximum instead of subqueries
78. be able to change your schema without ruining functionality of your code
79. source control schema and config files
80. for LVM innodb backups, restore to a different instance of MySQL so Innodb can roll forward
81. use multi_query if appropriate to reduce round-trips
82. partition appropriately
83. partition your database when you have real data
84. segregate tables/databases that benefit from different configuration variables

This article is meant to be an easy and relatively safe way to enhance mysql performance. It is not meant to be a complete guide to tuning MySQL. Fully optimizing MySQL takes both time and effort since every application has different requirements. The Debian MySQL packages ship with very conservative memory usage settings and the same is probably true for other Linux distributions and the Windows binaries. If you loosen these settings up a little, MySQL will perform much faster under ordinary circumstances.The items are ordered on impact, high to low. Feedback and hints will be greatly appreciated.

Key Buffer
The key buffer holds the indexes of tables in memory and a bigger key buffer results in faster row lookups. Adjust according to your own needs. Bigger is better, but prevent swapping at all costs. A good rule of thumb seems to be to use 1/4 of system memory.

key_buffer = 128M

Query Cache
This is where the magic happens. Well, not magic really, just plain old caching. Keeping the result of queries in memory until they are invalidated by additional writes enhances performance by magnitudes. The query_cache_size, as the name suggests, is the total size of memory available to query caching. The value query_cache_limit is the maximum number of kilobytes one query may be in order to be cached. Setting this value too high might prevent a lot of smaller queries to be cached. Setting it too low will result in bigger queries to never be cached, and the smaller queries not being able to completely fill the cache size, which would be a waste of resources. Adjust according to your own needs and memory available:

query_cache_size = 128MB
query_cache_limit = 4MB

Table Cache
An important variable if your application accesses many tables. It is the number of tables a thread can keep open at the same time. A value of 512 should do no harm.

table_cache = 512

Sort Buffers
sort_buffer_size (the variable previously known as sort_buffer), used for grouping and sorting and is a per-thread buffer. If the buffer can not hold the data to be sorted, a sort is performed on disk. Watch out for making this too large as the buffer is allocated for every thread that needs sorting and with many sorts it can easily consume all your memory.

sort_buffer_size = 32M
myisam_sort_buffer_size = 32M

The InnoDB Engine
Most people do not use the InnoDB engine in MySQL and use MyISAM instead. Since MySQL reserves memory for this engine, you are better off without it. If you need InnoDB, you can find more on its settings in the official MySQL docs.

Add `skip-innodb’ to my.cnf to disable the engine.

Binary Logging
MySQL has a few powerful features. Replicating data changes to a second server is one of them. MySQL keeps a log file of data changes which is used for this purpose. If you do not use replication or use the file as incremental backup, you can disable it. This will save you expensive disk write actions for every change to your data. For applications that have a lot of frequently updated data, this can be quite a performance boost. According to the official docs, this will generally result in just a 1% boost but it’s an easy gain if you do not need the log. Read more about the binary log here. Comment the following line:

log-bin = /var/log/mysql/mysql-bin.log

Temporary Tables
Temporary tables are used for sorting and grouping. The buffer is created on demand so watch out for setting this too high here as well. If the buffer cannot accomodate the data, a temp file is used on disk instead.

tmp_table_size = 64MB

Delayed Writing
This setting can greatly improve writing or updating data to a table. Instead of directly committing data to the disk, MySQL queues writes and returns write queries immediately. Be very very careful with this, because this also means that in case of a power failure or crash, you lose data. You can use this for logging if you don’t mind losing a couple of rows in case of a crash.

delay_key_write = 1

Connection Timeout
This is a little tweak that determines the closing of sleeping connections. The default is one hour and is often too long for practical purposes. I often set this at one minute instead (60).

wait_timeout = 60

The above settings are just to make mysql a little faster in general. You can get much better speed improvements by optimizing the database itself. Setting the correct indexes on tables can be a life-saver.

After 3 months of tweaking here is how I optimized to pull off these numbers with only a single dual Xeon 3.2ghz server with 4GB of memory. This is a very simplified guide and I am very far from being an expert – so this only reflects my personal success with this configuration. At the bottom of the article is a script I use to test performance.Mysql provides a configuration file located in /etc/my.cnf. From here you can set all of the memory, table, and connection limits as well as a host of other options. Before we get started I suggest you get aquainted with the my.cnf file as well as the tuning parameters within it.

Here is the my.cnf I use:

[mysqld]
back_log = 75
skip-innodb
max_connections = 500
key_buffer = 384M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 1800
thread_cache_size = 384
wait_timeout = 7200
connect_timeout = 10
tmp_table_size = 64M
max_heap_table_size = 64M
max_allowed_packet = 64M
max_connect_errors = 1000
read_rnd_buffer_size = 524288
bulk_insert_buffer_size = 8M
query_cache_limit = 4M
query_cache_size =128M
query_cache_type = 1
query_prealloc_size = 65536
query_alloc_block_size = 131072
default-storage-engine = MyISAM[mysqld_safe]
nice = -5
open_files_limit = 8192

[mysqldump]
quick
max_allowed_packet = 16M

[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M

Let’s just look at the important bits.

max_connections = 500 – I use a tool (see below) to check how many current connections I have, and under very heavy load (2000 simultaneous users) I rarely hit 400 concurrent connections to the database. This is because most connections only last for a few milliseconds.

key_buffer = 384M – When tuning a MySQL server, key_buffer_size is very important. This number works well for me and with the mysqlreport script I rarely use 50% of the available memory.

table_cache = 1800 – After key_buffer the next most important variable is your table cache. Again this is set for vBulletin so you may be able to significantly reduce this value depending on the number of tables in your database.

wait_timeout = 7200 – This variable determines the timeout in seconds before mysql will dump a connection. If set to low you will likely receive mySQL server has gone away errors in your log, which in vBulletin’s case is quite common.

max_allowed_packet = 16M – Again if set to low (the default is 8M) users will likely experience errors. 16M has always worked fine for my production environments.

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
skip-locking
#skip-networking
safe-show-database
query_cache_limit=1M
query_cache_size=64M ## 32MB for every 1GB of RAM
query_cache_type=1
max_user_connections=200
max_connections=500
interactive_timeout=10
wait_timeout=20
connect_timeout=20
thread_cache_size=128
key_buffer=128M ## 64MB for every 1GB of RAM
join_buffer=1M
max_connect_errors=20
max_allowed_packet=16M
table_cache=1024
record_buffer=1M
sort_buffer_size=2M ## 1MB for every 1GB of RAM
read_buffer_size=2M ## 1MB for every 1GB of RAM
read_rnd_buffer_size=2M  ## 1MB for every 1GB of RAM
thread_concurrency=2 ## Number of CPUs x 2
myisam_sort_buffer_size=64M
server-id=1
log_slow_queries=/var/log/mysql-slow-queries.log
long_query_time=2
collation-server=latin1_general_ci
old-passwords

[mysql.server]
user=mysql
basedir=/var/lib

[safe_mysqld]
err-log=/var/log/mysqld.log
pid-file=/var/lib/mysql/mysql.pid
open_files_limit=8192

[mysqldump]
quick
max_allowed_packet=16M

[mysql]
no-auto-rehash
#safe-updates

[isamchk]
key_buffer=32M
sort_buffer=32M
read_buffer=16M
write_buffer=16M

[myisamchk]
key_buffer=32M
sort_buffer=32M
read_buffer=16M
write_buffer=16M

[mysqlhotcopy]
interactive-timeout

Tags: ,
Comment

Lấy thông tin “Server Load” bằng PHP

Filed Under: Performance, PHP by Nguyễn Bá Khoa — Leave a comment
11/08/2010

Đôi khi bạn muốn kiểm soát lượng truy cập vào website của mình, thông qua Apache bạn có thể tính toán được thông số lượng truy cập hiện tại.Dựa trên hàm get_server_load(), giả sử server của bạn chỉ cho phép tối đa 1000 lượt truy cập cùng 1 thời điểm, và lượt truy cập thứ 1001 sẽ phải chờ, bạn chỉ cần viết 1 đoạn code sau:

if (get_server_load(true)>1000){echo “Server busy now. Try again later!”;exit(0);}

Dưới đây là hàm tính get_server_load();

function get_server_load($windows = false) {$os = strtolower(PHP_OS);if(strpos($os, “win”) === false) {if(file_exists(“/proc/loadavg”)) {$load = file_get_contents(“/proc/loadavg”);$load = explode(‘ ‘, $load);return $load[0];}elseif(function_exists(“shell_exec”)) {$load = explode(‘ ‘, `uptime`);return $load[count($load)-1];}else {return false;}}elseif($windows) {if(class_exists(“COM”)) {$wmi = new COM(“WinMgmts:\\\\.”);$cpus = $wmi->InstancesOf(”Win32_Processor”);$cpuload = 0;$i = 0;while ($cpu = $cpus->Next()) {$cpuload += $cpu->LoadPercentage;$i++;}$cpuload = round($cpuload / $i, 2);return “$cpuload%”;}else {return false;}}}print_r(get_server_load(true));?>

Tags: ,
Comment

Compress PHP, CSS, JavaScript(JS) & Optimize website performance.

Filed Under: Css, Javascript, Performance, PHP by Nguyễn Bá Khoa — Leave a comment
11/08/2010
Since few days we have been registering heavy traffic spikes on our website. This lead to performance issues. As this site is currently hosted on a shared hosting server, it is very difficult to optimize the performance of the site.We are using WordPress as CMS for this blog, hence we decided to install WP-Super cache plugin for WordPress to improve the performance. This plugin will create static HTML files from your blogs post and other pages and save them on web server. These HTMLs are served to client whenever consecutive requests are made. Hence this greatly improve the performance as it reduce PHP parsing and database connections.Bandwidth control is an important task to be followed when your traffic is increasing. With limited monthly bandwidth hosting, your site may run out of bandwidth and thus result in increase in down time. Hence it is very much advisable to compress your websites response with GZip and then serve it to client. Compressing output can significantly improve your websites performance by reducing the size sometimes upto 80%!So how can you enable GZip compression and compress your websites output? Well there are several ways to achieve this. I have listed out following very useful tricks to enable compression.

GZip compression in Tomcat, JBoss server

You can find a full post explaining this trick here.

GZip using mod_gzip, mod_deflate and htaccess

Apache server supports server level GZip compression with the help of module mod_gzip and mod_deflate. You can use this module and enable GZip compression for your website using htaccess file. First you have to check whether your hosting provider has enabled mod_gzip or mod_deflate module or not? To check this, you can use php_info() function of PHP which prints all the information about server and modules.You can enable compression for text and html by adding following lines in your htaccess file.

# compress all text and html:AddOutputFilterByType DEFLATE text/html text/plain text/xml# Or, compress certain file types by extension:<Files *.html>SetOutputFilter DEFLATE</Files>

You can compress all type of content (image, css, js etc) using mod_deflate. Copy following code in htaccess to do this.

<Location />    SetOutputFilter DEFLATE      SetEnvIfNoCase Request_URI  \        \.(?:gif|jpe?g|png)$ no-gzip dont-vary    SetEnvIfNoCase Request_URI  \        \.(?:exe|t?gz|zip|gz2|sit|rar)$ no-gzip dont-vary</Location>

Also, you can add following code in your htaccess file and enable compression using mod_gzip.

<IfModule mod_gzip.c>    mod_gzip_on       Yes    mod_gzip_dechunk  Yes    mod_gzip_item_include file      \.(html?|txt|css|js|php|pl)$    mod_gzip_item_include handler   ^cgi-script$    mod_gzip_item_include mime      ^text/.*    mod_gzip_item_include mime      ^application/x-javascript.*    mod_gzip_item_exclude mime      ^image/.*    mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*</IfModule>

This technique only works if mod_gzip or mod_deflate modules are loaded in Apache. In our case, these modules were not there and our hosting provider refused to load it as we were using a shared hosting. So following can be another way of enabling compression.

GZip using PHP ob_start() method

If your hosting provider does not support mod_gzip module, ob_start() method can be used to enable compression in PHP file. For this you need to copy following line in top of the PHP file. You may want to add this line in start of the header file that gets included in every php.

<?php	if (substr_count($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip'))		ob_start("ob_gzhandler");	else		ob_start();?>

Above code will check whether your browser supports gzip, if yes, then it send ob_gzhandler method as handle to ob_start method which buffers the output. Thus output is compressed using ob_gzhandler. Only problem with this method is that you have to manually edit all PHP files or should have a header.php file that gets included in all files. There are still ways to achieve this without touching your PHP files. Read following trick.

GZip using php_value directive in htaccess

php_value directive can be used to append/prepend any PHP files in the request of change the output handler. First we will see how we can prepend a PHP file and achieve this. Copy the PHP code that we saw in above example in a file called gzip-enable.php. Now copy following lines in your htaccess file. Thus you need not to modify any of your PHP files can can prepend a PHP file with ob_start() method to all the files.

<FilesMatch "\.(txt|html|htm|php)">    ForceType application/x-httpd-php	php_value auto_prepend_file /the/full/path/gzip-enable.php</FilesMatch>

But what if you don’t want to prepend a PHP file? Still there is a way to specify default output handler using htaccess. Use following line in your htaccess file to tell your apache to register ob_gzhandler handler function as output handler.

    php_value output_handler ob_gzhandler

Compress CSS using htaccess and php_value

CSS Stylesheet files occupy significant size in overall webpage size. It is hence advisable to compress these files before sending them to client. This significantly improve the performance of a webpage. For compressing CSS files, we will first create a PHP file gzip-css.php with following code.

<?php   // initialize ob_gzhandler function to send and compress data   ob_start ("ob_gzhandler");   // send the requisite header information and character set   header ("content-type: text/css; charset: UTF-8");   // check cached credentials and reprocess accordingly   header ("cache-control: must-revalidate");   // set variable for duration of cached content   $offset = 60 * 60;   // set variable specifying format of expiration header   $expire = "expires: " . gmdate ("D, d M Y H:i:s", time() + $offset) . " GMT";   // send cache expiration header to the client broswer   header ($expire);?>

Now add following lines in your htaccess file to enable compression for CSS files.

<FilesMatch "\.(css)">    ForceType application/x-httpd-php	php_value auto_prepend_file "/the/full/path/of/this/file/gzip-css.php"</FilesMatch>

Whenever a http request for .css will come to a server, the type of css will get converted to application/x-httpd-php, thus making them parsed using PHP. Then a file gzip-css.php will be prepend to this request which in turns compress the output using ob_start (”ob_gzhandler”) method.

Compress JavaScript (JS) using htaccess and php_value

Similar to above example for CSS, JavaScript files can also be compressed and sent to client. For this create a PHP file gzip-js.php and copy following code in it.

<?php   // initialize ob_gzhandler function to send and compress data   ob_start ("ob_gzhandler");   // send the requisite header information and character set   header ("content-type: text/javascript; charset: UTF-8");   // check cached credentials and reprocess accordingly   header ("cache-control: must-revalidate");   // set variable for duration of cached content   $offset = 60 * 60;   // set variable specifying format of expiration header   $expire = "expires: " . gmdate ("D, d M Y H:i:s", time() + $offset) . " GMT";   // send cache expiration header to the client broswer   header ($expire);?>

Also add following lines in your htaccess file.

<FilesMatch "\.(js)">    ForceType application/x-httpd-php	php_value auto_prepend_file "/the/full/path/of/this/file/gzip-js.php"</FilesMatch>

Do you know other methods of compressing the output of PHP/JS/CSS files? Let me know, I will try to add them in this tutorial.Happy Compressing :)

Tags: , , , , ,
Comment

Make your pages load faster by combining and compressing js and css

Filed Under: Css, Javascript, Performance by Nguyễn Bá Khoa — 4 Comments
11/08/2010

As some of you may know, I am currently working on a content management system. Although I am not able to share all of the code – it is proprietary after all – I already made one debugging tool public. This tool can be used to test some common techniques which decreases the bandwidth generated by feed consumers. Today I am going to make a second tool public – including source code. It is a method to decrease the loading time of a page by combining all the different css or javascript files and compress them.About six months ago I noticed the pages generated by the content management system were in itself very clean and small, but that these pages still took a long time to load for new visitors. Even on a fast internet connection it took more than 8 seconds to load a basically empty page. The server generated the page in about 350ms, so that wasn’t the problem. The problem turned out to be a combination of two things: each page used more than 12 different css files because each plugin supplied its own css definitions and because the use of the rather large prototype and scriptaculous javascript libraries which also consists of a couple of different files. Now that an article about the same problem featured on the Yahoo! User Interface blog, I decided to make my solution public, so others can benefit from it.The solution turned out the be simple, combine all the different files into a single large file and compress that file using gzip. Unfortunately, if you do this manually you are going to run into maintenance problems. That single compressed file is no longer editable. So after editing one of the original source files you will have to recombine it with the other files and re-compress it.Instead of going for the easy – but hard to maintain – solution I decided to automate the process and thanks to a small PHP script and some clever URL rewriting I now have an easy to maintain method to speed up the loading of pages that use many or large css and javascript files.The idea is that you have one directory for css files and one directory for javascript files on your server. If you rewrite the URLs that point to these directories to a small script that intercepts the requests for those files. The script loads the file from disk and compresses it using gzip. It then sends that compressed file back to the browser. Given that javascript and css files compress really well this will greatly decrease the size of the data that is going to be transferred and thus decrease the time needed to download these files. Because this works completely transparently you do not need to change anything in your existing code.But there is more. Compressing the files will decrease the size of the data that needs to be transferred, it does not solve the problem that the browser can only download a limited number of files at the same time. If you have many different files that need to be loaded the browser will not optimally use the bandwidth it has access to. It will request some files from the server and wait until those files are retrieved before the rest of the files are requested. The solution to this problem is to combine all those different files into one large file. And this is exactly what the script tries to do. You can concatenate different files by simply adding the names of the other files to the URL of the first file.

Take for example the following URLs:
http://www.creatype.nl/javascript/prototype.js
http://www.creatype.nl/javascript/builder.js
http://www.creatype.nl/javascript/effects.js
http://www.creatype.nl/javascript/dragdrop.js
http://www.creatype.nl/javascript/slider.js

You can combine all these files to a single file by simply changing the URL to:
http://www.creatype.nl/javascript/prototype.js,builder.js,effects.js,dragdrop.js,slider.js

The script will intercept the attempt to retrieve something from the javascript directory and will notice that you want to fetch multiple files at once. It will then concatenate the requested files, compress it and send it as one to the browser. Also notice that I include the files that come with scriptaculous manually and I do not use the scriptaculous.js file like you normally would. The reason for this is that scriptaculous.js loads each javascript file individually. If I use the scriptaculous.js file I will get the benefit of compression, but the different files won’t be combined into a single file.Unfortunately I noticed a nasty side effect of the combination of these two methods. If you combine many files the resulting files can be come quite large. Compressing those files takes some time and on a busy server that time will become large enough to negate a significant portion of the improvements you made earlier. But this problem can also be solved by simply adding a cache that stores an already combined and compressed version of the files. The cached version is automatically created the first time that particular combination of files is used and used every time – as long as the files are not changed. The result is that once the cache is created there is almost no overhead and the compressed file is delivered almost instantly.I’ve done some informal testing on my own website and I did get some impressive results. Before this script was added to my website you needed to download 8 javascript files, in total 168 Kb – the prototype and scriptaculous libraries. On average this took about 1905 ms. After installing this script you now need to download only a single file of 37 Kb which only takes around 400 ms. Your results may vary of course, but given that it shaved 1.5 seconds of a total loading time of 3.5 seconds, this script almost cut the time needed to load a page on my weblog in two.Configurating this script is easy. First you need to download and configure the combine.php script. By default this script look in the javascript and css directory in the root of your website, but if you are currently using different directories you can change these values at the top of the combine.php script. Upload the combine.php script to the root of your website. Secondly you need to create a cache directory that is writable by the web server. Again, by default this script will look for the cache directory in the root of the website, but you can change this in the combine.php script. Finally you need to create or modify your .htaccess file. If you do not have a .htaccess file you can create it in the root of your website and add the following lines. If you already have an preexisting .htaccess file you can simply add the following lines to the file:

RewriteEngine On
RewriteBase /
RewriteRule ^css/(.*\.css) /combine.php?type=css&files=$1
RewriteRule ^javascript/(.*\.js) /combine.php?type=javascript&files=$1

Note: if your preexisting .htaccess file already uses URL rewriting you do not need to add the first two lines. You can simply add the last two lines to the bottom of the .htaccess file.

Tags: , , , ,
Comment

Analyze website using online Website Analyzer Tools

Filed Under: Performance by Nguyễn Bá Khoa — 2 Comments
11/08/2010

Website analyzing and optimizing has became one of the key task that a webmaster has to perform in order to make the website run efficiently. By analyzing your website you can identify the problems related to loading time, size, keyword effectiveness etc.A lot of online website analyzing tools are available that can be used freely to analyze your website and which gives you an insight view of the problems and area of improvement. Let us take an example and analyze a website.

Selecting Scapegoat

Let us analyze the BBC website using online website analyzer tool websiteoptimization.com.

Analyze the website

Open websiteoptimization.com, it will ask for the Enter URL to diagnose. Enter any URL that you wants (in my case I am entering http://www.bbc.co.uk). Click on Submit Query button. It will ask for a CAPTCHA to filter spams. Enter the captcha details and hit Continue.

Diagnosis

That’s it. The result will be shown where you can see different sections of report.

Global Statistics

Global Statistics are the figures that gives you details about the total HTTP request fired from the website (Total requests for images, css, javascript etc) and also total size of the webpage.

Object Size Totals

This section provides details about each type of object and its total size. i.e. the size of HTML, JavaScript, CSS, Images etc and total loading time.

Analysis and Recommendations

v
This is the section that one should look and understand. This section will describe each section of the webpage and recommend if the size of any section let say JavaScript or CSS is more than the normal size. You may want to read the tutorial to enable compression for javascript, css, html etc which will reduce the size significantly.

Other online website analyzer tools

Website Checker is one of the similar tool that can be used to analyze the website. Not only does it shows the total loading time of the website and individual objects like html, css, javascript etc, it also provide a keyword suggestion facilities which will be helpful for doing search engine optimization of the site.Submitexpress also provides similar functionality of website analyzing. It will also suggest you whether you have used appropriate meta description or meta keywords in your webpage.

Tags: , ,
Comment

Speed up Apache – how I went from F to A in YSlow

Filed Under: Performance by Nguyễn Bá Khoa — Leave a comment
11/08/2010

I decided to embark on figuring out how to make my site as fast as possible. There were a few tips I was already aware of but decided to grade myself using YSlow. My initial score was bad, an F. I realized I had to do a few things.

  • Compress text/* files using gzip
  • Decrease HTTP requests by combining multiple JavaScript (and CSS) files into single files
  • Add aggressive caching since the site isn’t updated very often (especially images, JavaScript and CSS files)

Here’s what my original scores looked like.



Making fewer HTTP requestsIn order to make fewer HTTP requests I knew that I had to combine my 3 CSS files into one as well as my 7 JavaScript files. I decided to use a little bit of Apache’s mod_rewrite magic. I set up a rewrite rule for any file which didn’t exist inside my js directory. The rule looks like this:

RewriteEngine onRewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)\?*$ compress.php?__args__=$1 [L,QSA]

This let’s me have a script tag that looks like this.

<script src="file1.js|file2.js|file3.js"></script>

I have a php file named compress.php which takes the file names in the query string and concatenates them together. That file looks like this.Note: Thanks to Pascal in the comments for pointing out a security issue in the original version.

  ini_set('include_path', '.');
  header('Content-Type: text/javascript');
  if(!empty($_GET['__args__']))  {
      $files = (array)explode('|',$_GET['__args__']);
      foreach($files as $file)    {
            if(isValidFile($file))      {
                    readfile($file);
                    echo "\n";
            }
    }
  }
  function isValidFile($file)  {
      // add whatever logic here you'd like
      return substr($file, -3) == '.js';
  }

I did the exact same thing for CSS files so I won’t go into great detail with that.Adding expire tags to the HTTP headersThe next step was to add expires headers to the response. I originally thought about doing it with PHP but decided that it would be a much better idea to do it using Apache. I am not going to go into the details of caching (maybe another post someday). I decided to add expiration headers by file type. This can be done using Apache by using mod_expires. You can turn this on in your httpd.conf file by searching for mod_expires and making sure it’s commented out. Once that’s done you can add the following to your configuration files. I did mine inside of my VirtualHost block.

  ExpiresActive On  
  ExpiresByType text/html "A7200"
  ExpiresByType text/javascript "A604800"  
  ExpiresByType text/css "A604800"  
  ExpiresByType image/x-icon "A31536000"  
  ExpiresByType image/gif "A604800"  
  ExpiresByType image/jpg "A604800"  
  ExpiresByType image/jpeg "A604800"  
  ExpiresByType image/png "A604800"   
  Header set Cache-Control "must-revalidate"

You first have to Enable the module and then you can specify when you want specific file types to expire. The syntax for the 2nd parameter is [A|M]seconds. So A3600 would mean one hour since it was last accessed and M60 would mean one minute since it was last modified. I also added the must-revalidate cache control so that all browsers and proxies let me specify exactly how my site handles caching.Compressing files (text/*)This one was easy. I suggest using Apache’s mod_deflate or mod_gzip to do this but I was getting lazy and decided to do it with PHP. It was a one liner for me and I only had to add it in one spot for my entire site to obey it. Doing this also took care of minifying JS listed as #10.

ob_start('ob_gzhandler');

Configuring ETagsI had no idea what ETags were so I had to look them up. ETags are server generated ids for each object. The nice thing about this is that it can be updated programatically to let the browser or proxy know that a new file is available. I decided to have apache handle this for all requests and base it off of the modification time and the file size. I added the following immediately below the cache configurations in my VirtualHost block.

FileETag MTime Size

The results…The results were pretty good. I can do some more tweaking but a few of the remaining issued are due to my usage of Google Analytics and the Photagious API. My final score was an A (92).


Tags: ,
Comment
Powered by WordPress | Theme: by 85ideas. Editor by Khoanguyen