Tuesday, June 21st 2011


Tuning mySQL for Magento
posted @ 10:07 am in [ Apache -Fixing Things -Magento -mySQL -Technology -Web Design ]

Magento loves using lots of mySQL processes, so therefore mySQL needs to finely tuned to achieve this.

We ran some benchmarking tests today using mysqlreport as the benchmarking tool and as you can see from the results below, mySQL caching following some of the tips in the articles below can really make a difference

1. First test

As you can see with basic caching turned on and no massive tuning we get a cache insert to prune rate of 1.31:1 and no real cache hits

query_cache_size = 8M
table_cache = 128
innodb_buffer_pool_size = 8M

__ Query Cache _________________________________________________________
Memory usage 5.89M of 8.00M %Used: 73.58
Block Fragmnt 14.66%
Hits 1.21M 157.7/s
Inserts 389.92k 50.7/s
Insrt:Prune 1.31:1 11.9/s
Hit:Insert

2. Second test with recommendations from mysqltuner.pl and mysqlreport-3.5

Bigger cache better results

query_cache_size = 32M
table_cache = 512
innodb_buffer_pool_size = 256M

__ Query Cache _________________________________________________________
Memory usage 23.79M of 32.00M %Used: 74.33
Block Fragmnt 12.43%
Hits 3.50M 184.3/s
Inserts 681.81k 35.9/s
Insrt:Prune 2.55:1 21.8/s
Hit:Insert 5.13:1

Hit versus Insert of 5.13 to 1 – now that should make the world a much faster place ๐Ÿ™‚

References:

http://www.mysqlperformanceblog.com/2007/11/01/innodb-performance-optimization-basics/
http://www.mysqlperformanceblog.com/2007/11/03/choosing-innodb_buffer_pool_size/
http://www.techiecorner.com/45/turn-on-mysql-query-cache-to-speed-up-mysql-query-performance/
http://webcache.googleusercontent.com/search?q=cache:wVQfPOsS3t0J:www.debianhelp.co.uk/mysqlperformance.htm+mysql+optomise&cd=10&hl=en&ct=clnk&gl=uk&client=firefox-a&source=www.google.co.uk




Friday, August 20th 2010


Magento Site Performance
posted @ 8:57 am in [ Apache -Fixing Things -Hosting -Magento -Media Temple -PHP -Technology -Web Design ]

Magento the nice Community Version available e-commerce platform that we are using at Skywire for a number of our client builds is incredibly feature rich, but with all of these features comes the trade-off that to get any kind of speed out of the system you either need SERIOUS server hardware, or an awful lot of performance tuning.

To be honest it can run like a real dog if you don’t really work at it!

Well we like to make things work hard at Skywire so went on a journey of discovery on how to make Magento fly, and here’s our understandings to share with everyone else.

1. Server software selection and tuning

Lots of articles out there about this around the web, but you can sum it up in a few points really.

– What webserver (Apache vs. Lighttpd vs. Nginx) and how many threads for that webserver you need. Interestingly, against every article out there, Apache 2 was faster for us that Lighttpd and NginX but I think this was to do with the PHP CGI access the other two were using being slower on our Media Temple server

– Fine tune your mySQL database – we found that the two great scripts mysqlreport and mysqltuner are your friends here

– Get rid of any other processes you don’t need that get in the way (xinetd, spam assassin etc.)

2. Turn on lots of Caching

Magento has caching so turn that on for starts, and then get a minify type plugin (there’s lots of them out there but ) to complement that and join all of your CSS and JS into a single compressed file.

Install a PHP Byte Code caching system to cache any code generated by PHP – we used XCache as it was available via yum but eAccellerator gets good reviews too [although it just hung in our environment].

3. Turn on the Page Compilation feature in Magento!

Yes, I know it’s labelled as Beta, and yes I know it falls over most of the time you run it, but if you run it from the command line, as the same user that owns your web files then it works just great creating a new single directory in /includes/src containing flattened files of all your Magento files with the naming format directory_directory_etc_filename.php

This shaved at least 1 second off of every page load for us (amazing but true) however was a job to install as it ignores any modules installed in /app/code/community.

No worries though, you can work around this by just copying the whole module directory to the /app/code/local directory and rerunning the compiler and then it works great.

4. Load you Magento cache directories into a memory filesystem

Sounds a strange thing to do but you can load your /var/cache/ directory into a memory based ‘tmpfs’ which makes it much faster. Also you can move your sessions to your database instead however this slowed things down for us so we left them as files.

Summary:

So once you’ve done this on a mid-spec Media Temple DV server you can reduce page times from about 10 seconds down to just over a second, which believe me seems fast compared to how clunky Magento can be when running. Have fun!

References:

Magento performance and optimization

http://www.magentocommerce.com/group/view/168/

How do I use the inbuilt magento profiler to see bottlenecks?

http://www.magentocommerce.com/group/blog/action/viewpost/1405/group/168/

Magento Compiler – Improve your performance

http://www.magentocommerce.com/group/blog/action/viewpost/1243/group/168/

9 Methods to Speed Up Magento โ€“ A Guide to Making Magento Faster

http://www.blastedthing.com/magento/questions/mag-9-methods-to-speed-up-magento-a-guide-to-making-magento-faster/

Magento performance hosting

http://yoast.com/magento-performance-hosting/

Magento Site Performance Optimization

http://www.yireo.com/tutorials/magento/magento-spo#compress-output-in-general

Performance is Key! – Notes on Magentoโ€™s Performance

http://www.magentocommerce.com/blog/comments/performance-is-key-notes-on-magentos-performance/




Monday, January 18th 2010


Confusing SSL with mixed IP addresses
posted @ 8:21 am in [ Apache -Fixing Things -Hosting -PHP -Technology ]

SSL throws a weird error in that if you have http (port 80) bound to one IP address [say an internal one] and you bind https (port 443) to a different IP address [say an external one] then SSL throws the following very undescriptive error:

SSL received a record that exceeded the maximum permissible length.

(Error code: ssl_error_rx_record_too_long)

To fix, edit your Apace configuration file in /etc/httpd/conf/httpd.conf (or similar) and make sure that both virtual hosts have the same IP address – job done ๐Ÿ™‚

references: here and here




Friday, December 11th 2009


UKFast – Installing cPanel on fresh CentOS x_64
posted @ 10:28 am in [ Apache -Hosting -osCommerce -PHP -Technology -Web2.0 ]

We’re just starting to move one of our clients Merc (http://www.merc.com/) onto a new webserver as their site has been doing so well that it need a faster, more optimised server to cope with the traffic and went with a new Cloud CentOS x_64 server from UK Fast for a very reasonable ยฃ70.00 odd per month.

The server seems lightening quick which is great, but didn’t come with a nice install of Plesk or cPanel or similar so we decided to install cPanel.

It’s actually really easy to do – just follow this really helpful guide here from Network Data Center in the US and you are up and running with a full blown cPanel in a couple of hours.

Job done! ๐Ÿ™‚

References:
https://helpdesk.ndchost.com/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=25




Friday, July 10th 2009


Reading Raw POST Data in PHP
posted @ 3:46 am in [ Apache -PHP -Web Design ]

We’ve been having fun trying to proxy array inputs such as radio fields in PHP, and it turns out that their is a great way to access the POST fields data that PHP receives _before_ PHP does any manipulations to it.

What this means is that if you need to pass data from one server to another, totally unaltered, then using the file_get_contents("php://input"); is the answer.

Here’s the original post I was pointed to about this by the very helpful team on the PHP/Curl Mailing List [curl-and-php at cool.haxx.se]

“Sometimes, albeit rarely, I’ve needed to get at the raw POST data in a PHP file. You can of course get the POST data using $_POST, however, this is after PHP has processed it (i.e. magic quotes may have been applied, thus escaping the data). You can access the raw POST information using the following:

$data = file_get_contents("php://input");

You can access the php://input wrapper as if it were a file, and use the Filesystem functions. More information on the PHP wrappers can be found in the PHP Manual.”

From JellyAndCustard.com




Wednesday, March 26th 2008


Media Temple and PHP5 cooking tips…
posted @ 12:05 pm in [ Apache -PHP -Technology -Web Design ]

I know I know, nobody really wants to run PHP5 on their Media Temple Dedicated Server when PHP4 works just fine – well actually yes we did as one of our team is becoming a dab hand at writing PHP5 dependant classes and then having to find servers to run them.

Media Temple provide their own link on how to upgrade the system (http://kb.mediatemple.net/article.php?id=260) however this was having none of it for us so here’s a quick heads-up on how to do it our way.

1. Install YUM

There’s a few great articles on how to do this on the web (Google for CentOS and YUM) but the trick is to remember when you have it installed is that the packages missing from the central CentOS distribution can still be accessed with YUM you just need to use CentOSPlus.

yum --enablerepo=centosplus install <your package name>

2. Roll your own PHP 5

Download the latest PHP tar ball, unpack and start configuring. Here’s our config file which will help you build in all the dependencies that Media Temple has (zend, mysql, zlib etc.) – you will need to use YUM to get various packages and libraries, and some have to be located on the source websites, but here’s a hints to help.

– installing mcrypt

 wget ftp://mcrypt.hellug.gr/pub/crypto/mcrypt/libmcrypt/libmcrypt-2.5.7.tar.gz

 rpmbuild -ta libmcrypt-2.5.7.tar.gz

 rpm -ivh /usr/src/redhat/RPMS/i386/libmcrypt-2.5.7-1.i386.rpm /usr/src/redhat/RPMS/i386/libmcrypt-devel-2.5.7-1.i386.rpm

 yum --enablerepo=centosplus install php-mcrypt

– installing mhash

 wget http://downloads.sourceforge.net/mhash/mhash-0.9.9.tar.bz2?modtime=1175740797&big_mirror=0

 bunzip2 mhash-0.9.9.tar.bz2

 tar xvf mhash-0.9.9.tar

 cd mhash-0.9.9

 ./configure

 make install

– configuring php5

./configure 

 --with-apxs2=/usr/sbin/apxs 

 --with-pear=/usr/share/pear 

 --build=i686-redhat-linux-gnu 

 --host=i686-redhat-linux-gnu 

 --target=i686-redhat-linux-gnu 

 --prefix=$PHP_PREFIX 

 --sysconfdir=/etc 

 --localstatedir=/var 

 --disable-debug 

 --disable-rpath 

 --disable-ipv6 

 --disable-dba 

 --without-odbc 

 --without-unixODBC 

 --without-gdbm 

 --enable-inline-optimization 

 --enable-exif 

 --enable-ftp 

 --enable-sockets 

 --enable-yp 

 --enable-wddx 

 --enable-memory-limit 

 --enable-calendar 

 --enable-mbstring 

 --enable-soap 

 --enable-bcmath 

 --enable-gd-native-ttf 

 --enable-zip 

 --with-readline 

 --with-libdir=lib 

 --with-config-file-path=/etc 

 --with-exec-dir=/usr/bin 

 --with-freetype-dir=/usr 

 --with-png-dir=/usr 

 --with-jpeg-dir=/usr 

 --with-expat-dir=/usr 

 --with-libxml-dir=/usr 

 --with-mysql=/usr/local/mysql 

 --with-mysql-sock=/var/lib/mysql 

 --with-mysqli 

 --with-curl 

 --with-gettext 

 --with-iconv 

 --with-openssl 

 --with-mcrypt 

 --with-mhash 

 --with-gd 

 --with-png 

 --with-bz2 

 --with-zlib 

 --with-layout=GNU 

 --with-xml 

 --with-xmlrpc 

 --with-tidy

Restart your PSA system (

/usr/local/psa/admin/sbin/websrvmng -a -v

) and you should be cooking on a lovely PHP 5 install with everything else still as it was.




Wednesday, March 12th 2008


Recursive ftp with wget
posted @ 10:49 am in [ Apache -PHP -Technology -Web Design ]

Sometimes when the servers won’t give you ssh access, and you want to copy files from one machine where you don’t have ssh access to one that does, recursive ftp with wget is the answer

wget -r ftp://username:password@servername/directory

For example to get all the files from my server at www.fishcakes.com with username fish and password cakes, in my homedirectory I would use

wget -r ftp://fish:cakes@www.fishcakes.com/~/*

Prolem solved…




Thursday, January 24th 2008


Receiving and Decoding MMS messages using Apache
posted @ 8:02 am in [ Apache -Kannel -MBuni -MMS -Mobile -PHP ]

A quick follow onto the post below of how to use Apache to serve MMS messages, one other interesting thing you can do with Apache is use it to receive MMS messages from a phone.

Basically, this just requires a little bit of lateral thinking, and some clever PHP and then you can see exactly which headers a phone uses to create the MMS messages it sends itself – and in our case use this to work out which fields we should be including in our MMS messages.

Step 1: Configure your phone to send MMS messages to your own Apache server, not the default operator MMSC.

This really depends on what phone you have, but if you dig around in the MMS settings within the phone, you will eventually find a menu option labelled MMS Server Name or similar. On a Motorola Razor V3 for example this is in Messages -> (option menu) -> Message Setup -> Change -> Server Info ->(option menu) -> Edit -> Server Name

Change this to be your own Apache server, for example in our case mms.skywire.co.uk, or an IP address if you have one may be even better if your operator doesn’t support DNS look-ups.

Step 2: Create your MMS on the phone

Step 3: Add you MMS receiving code to your Apache server

There’s lots of good packages out there to do this, including mBlogging software such as b2evolution and Peffisaur however for our decoding we used a nice piece of PHP coding from the following article on creating an MMSC.

Step 4: Send your message from your phone to your server

Step 5: Decode your MMS message

Now you have the MMS message as a binary file, but you really want something handy to decode it for you, so this is where a handy PHP class called MMS Decoder comes in handy. By default this reads from an HTTP stream, but with the following few modifications you can make it read from files instead.

<?php
require_once("mmsdecoder.php");
$filename = "test.mms";

if (is_readable($filename)) {
	$handle = fopen($filename, "r");
	$mmsdata = fread($handle, filesize($filename));
	fclose($handle);
}

// parse MMS
$mms = new MMSDecoder($mmsdata);
$mms->parse();

// The MMS is parsed, so let's get the mms data from the class by the print_r() function,echo(print_r($mms, true));

?>

Voila! All the headers from your phone built MMS ready for you to emulate in your own messages to ensure the phone receives them.




Thursday, January 24th 2008


Serving MMS messages with Apache
posted @ 7:29 am in [ Apache -Kannel -MBuni -MMS -Mobile -PHP ]

Now you would have thought after the number of years I’ve been playing around with MMS messages, using libraries like the Nokia MMS Java library to build them and servers like Kannel (SMS) and MBuni (MMS) to serve them, that we wouldn’t have any problems trying to serve MMS content ourselves using SMS binary push and .MMS encoded files.

Well, basically we did – and it all comes down to one thing – phones needing be specifically told that .MMS messages have a certain MIME type.

Some phones, the Motorola V3 Razor for example, automatically assume that all MMS messages are going to be an acceptable type, no matter what MIME type you serve them as. Therefore using a standard Apache server, a binary created MMS file (try the Nokia MMS Java library for Java, and MMSLib for PHP to build the messages) and a binary SMS push (either made yourself or using HTTP API’s such as Clickatell’s) these phones recieve the messages just fine.

However, a lot of other phones (notably Blackberry’s and Sony Ericsson’s) require specific instructions to treat the .mms file as type application/vnd.wap.mms-message

Therefore for these phones, add into the directory where the .mms files are stored the following .htaccess file and Apache will start serving the content in exactly the right way.

AddType application/vnd.wap.mms-message mms

Job done – and I bet if you’re reading because this is what you’re trying to do, we hopefully just saved you hours of going mad!




Tuesday, April 10th 2007


Running osCommerce on PHP 5.2.1
posted @ 8:03 am in [ Apache -osCommerce -PHP -Technology ]

We use a lot of osCommerce in our client projects as it is one of the most advanced Open Source e-commerce systems available on the market today, and best of all we can share the code with our clients without them having to pay costly licensing contracts or even worse, cough cough, being tied in to a 3 year monthly fee contract like some other vend(or/a)s do in our market.

So, with a recent upgrade to PHP 5.2.1 on one of our development servers, osCommerce ms2.2, the best core release of the platform, refused to play ball having largely been tested and developed for PHP 4. Not to fear though, with a bit of research and a few tweaks we were able to get it up running.

1. Running your PHP as a Module rather than a CGI on Windows

In order to allow PHP local variables to be set, rather than just global ones, you have to run your PHP within Apache as a Module and for mySQL to work download and install the extra libmysql.dll from the mySQL homepages.

As we are now running PHP as a Module, rather than a CGI, it no longer automatically includes the working directory the executable is in (normally c:\program files\php-5.2.1\ or similar) so simply placing this file in that directory, as we used to do, works. You need to either add this directory to the PATH environment in Windows, or copy the libmysql.dll to the Windows/System32/ directory or similar.

Then modify your httpd.conf to include the following

#BEGIN PHP INSTALLER EDITS – REMOVE ONLY ON UNINSTALL
PHPIniDir “C:\\Program Files\\php-5.2.1\\”
LoadModule php5_module “C:\\Program Files\\php-5.2.1\\php5apache2_2.dll”
#END PHP INSTALLER EDITS – REMOVE ONLY ON UNINSTALL

For the php.ini uncomment the mySQL module needed (remember you need to download the extra Module DLLS for PHP 5.2.1 for this to work and path the Module directory correctly)

extension=php_mysql.dll

and

; Directory in which the loadable extensions (modules) reside.
extension_dir =”C:\Program Files\php-5.2.1\ext”

2. Modifying the local environment variables for PHP

In your working directory, create a new .htaccess file and assign the following PHP parameters within it

php_value register_globals on
php_value register_long_arrays on

3. Restart everything

Restart Apache to pick up the new PHP configuration, and then run a quick phpinfo(); in a test PHP file to check that the local values for the two variables above are set.

And.. you should have a running osCommerce MS2.2 on PHP 5.2.1 and Apache 2