Thursday, 29 January 2009

How To Delete Mails From Or To A Specific Email Address From Your Mail Queue (Postfix)

If you get hit by a spam attack that floods your server with hundreds/thousands of emails from the same sender email address or to the same recipient email address, you can clean your mail queue from these emails with one single command before the mail flood takes your server to its knees.

You can check your current mail queue like this:

postqueue -p

To delete all mails from the mail queue that come from falko@example.com or are sent to falko@example.com (the command is the same regardless of if it's the sender or recipient address), you can use this command:

mailq | tail +2 | awk 'BEGIN { RS = "" } / falko@example\.com$/ { print $1 }' | tr -d '*!' | postsuper -d -

Afterwards check your mail queue again:

postqueue -p

It should now be much shorter.

Setting, Changing And Resetting MySQL Root Passwords

This tutorial explains how you can set, change and reset (if you've forgotten the password) MySQL root passwords. Time and again I see problems like mysqladmin: connect to server at 'localhost' failed error: 'Access denied for user 'root'@'localhost' (using password: YES)'. So I thought it's time to remind you how to solve MySQL related password problems. If you are just looking for a quick fix how to reset a MySQL root password you can find that at the bottom of this tutorial.

mysqladmin Command To Change Root Password

Method 1 - Set up root password for the first time

If you have never set a root password for MySQL, the server does not require a password at all for connecting as root. To set up a root password for the first time, use the mysqladmin command at the shell prompt as follows:

$ mysqladmin -u root password newpass

If you want to change (or update) a root password, then you need to use the following command:

$ mysqladmin -u root -p oldpassword newpass

Enter password:

If you get...

mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'root'@'localhost' (using password: YES)'

then follow the instructions below on how to recover your MySQL password.

Change MySQL password for other users

To change a normal user password you need to type:

$ mysqladmin -u user-name -p oldpassword newpass

Method 2 - Update or change password

MySQL stores usernames and passwords in the user table inside the MySQL database. You can directly update a password using the following method to update or change passwords:

1) Login to the MySQL server, type the following command at the shell prompt:

$ mysql -u root -p

2) Use the mysql database (type commands at the mysql> prompt):

mysql> use mysql;

3) Change password for a user:

mysql> update user set password=PASSWORD("newpass") where User='ENTER-USER-NAME-HERE';

4) Reload privileges:

mysql> flush privileges;
mysql> quit

This method you need to use while using PHP or Perl scripting.

Recover MySQL root password

You can recover a MySQL database server password with the following five easy steps:

Step # 1: Stop the MySQL server process.

Step # 2: Start the MySQL (mysqld) server/daemon process with the --skip-grant-tables option so that it will not prompt for a password.

Step # 3: Connect to the MySQL server as the root user.

Step # 4: Set a new root password.

Step # 5: Exit and restart the MySQL server.

Here are the commands you need to type for each step (log in as the root user):

Step # 1 : Stop the MySQL service:

# /etc/init.d/mysql stop

Output:

Stopping MySQL database server: mysqld.

Step # 2: Start the MySQL server w/o password:

# mysqld_safe --skip-grant-tables &

Output:

[1] 5988
Starting mysqld daemon with databases from /var/lib/mysql
mysqld_safe[6025]: started

Step # 3: Connect to the MySQL server using the MySQL client:

# mysql -u root

Output:

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.15-Debian_1-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql>


Step # 4: Set a new MySQL root user password:

mysql> use mysql;
mysql> update user set password=PASSWORD("NEW-ROOT-PASSWORD") where User='root';
mysql> flush privileges;
mysql> quit

Step # 5: Stop the MySQL server:

# /etc/init.d/mysql stop

Output:

Stopping MySQL database server: mysqld
STOPPING server from pid file /var/run/mysqld/mysqld.pid
mysqld_safe[6186]: ended

[1]+ Done mysqld_safe --skip-grant-tables

Start the MySQL server and test it:

# /etc/init.d/mysql start
# mysql -u root -p

Saturday, 10 January 2009

Operating System Usage Over the Next 4 Years

These figures are extrapolated from W3Schools.com. I have predicted data for the next 4 years with a linear scale based on the last 5 years. The vertical axis has a logarithmic scale so the OSX and Linux data is meaningful. The graph shows a very slight decline in Windows usage and a notable increase in Apple OSX and Linux usage.


Obviously there are plenty of unforseen changes that could have an effect on this graph. I see the move towards OSX from Windows in the mainstream as inevitable, and I would imagine the change is more likely to be a sudden surge over a period of months rather than a smooth decline of Windows usage over a number of years.

It is interesting to compare the initial Vista predictions made by Steven York just before Vista’s release with the real stats in hindsight. The obviously low take up of Vista leaves most users on Windows XP, which when you compare to the look and feel of modern environments (for example: Websites, Mobile Phones, Latest Apple OS) it’s very clunky and unintuitive.

4 Simple Apache tweaks to speed up delivery of your web site

Add these simple directives into your Apache configuration and you should notice a considerable increase in speed in the delivery of pages. As a working example, I performed these tweaks on http://www.shortlist.com/. Here are the before and after stats:





The changes are:

  • Homepage on an empty cache from 956.6k to 658.9k, both 133 requests.
  • Homepage on a primed cache from 153.9k and 128 requests, to 25.9k and 8 requests.

Starting point for client side caching
To be added in the directive.

Adds a 1 week client side cache to GIFs, CSS, JS, SWF and JPEGs. Note once in place this cannot be revoked on the client end, the cache is in the user’s web browser. Be sure the site doesn’t overwrite images etc when using this, images need unqiue names.

Requires mod_expires (expires.load)
#Add client side caching for 604800 seconds
ExpiresActive On
ExpiresByType image/gif A604800
ExpiresByType text/css A604800
ExpiresByType application/x-javascript A604800
ExpiresByType image/jpeg A604800
ExpiresByType application/x-shockwave-flash A604800

If needed you can add more or custom MIME types for the ExpiresByType directive to process by editing /etc/mime.types and adding your own lines to the end.

Enable gzip
To be added in the directive.

Should be used on all sites. This used to effect render time, but PCs are plenty fast enough these days.

Requires mod_deflate (deflate.load) and mod_headers (headers.load)

#Add compression on everything except images, with netscape fix and make sure proxys don't override agent
SetOutputFilter DEFLATE
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE[E] !no-gzip !gzip-only-text/html
SetEnvIfNoCase Request_URI \
\.(?:gif|jpe?g|png)$ no-gzip dont-vary
Header append Vary User-Agent env=!dont-vary

Configure ETAGs
To be added in the directive.

This is best used on HTTP load balancers. It prevents the client seeing 2 pages that are identical on 2 servers as different pages and pulling unnecesarily. If this is added to a site on a single web server, HTML will never get cached increasing the load, but this can have good effects as well.

#Disable ETags to hide multiple servers and aid caching
FileETag none

Prefork Configuration
To be modified in the httpd.conf/apache2.conf directive.

The default configuration top’s out way to fast. Any modern server can handle the load this will allow.

ServerLimit 1024

StartServers 8
MinSpareServers 5
MaxSpareServers 20
MaxClients 1024
MaxRequestsPerChild 4000

Further Reading
Yahoo!’s performance rules: http://developer.yahoo.com/performance/rules.html

Installing PHP 5.2.x on RedHat ES5, CentOS 5, etc

I’ve had to follow this tutorial a few times myself now so decided I should share it with the world.

A few of our applications which make use of SOAP get a Segmentation Fault if run with PHP 5.1.x or lower. We believe this is due to a bug in PHP but can’t be sure, regardless it works fine in PHP 5.2.4 and above.

Problem is, RedHat ES5 does not have support at this time for anything higher than 5.1.6, and we didn’t want to break RPM dependancys etc by installing from source.

To install PHP 5.2.5 (Highest in repository at this time) you can make use of a RPM repository maintained by Remi. He has a repository for each distro, but to save you translating for the ES5 one I’ll give you the commands here. Run the following to get up and running:

wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-2.noarch.rpm
wget http://rpms.famillecollet.com/el5.i386/remi-release-5-4.el5.remi.noarch.rpm
rpm -Uvh remi-release-5*.rpm epel-release-5*.rpm

You now have the Remi repository on your system, however it is disabled by default. Obviously you don’t want all of your packages been effected by this repository, however to enable it for a specific package, run the following:

yum --enablerepo=remi update php

You should now have the latest PHP5 installed:

# php -v

PHP 5.2.5 (cli) (built: Nov 10 2007 10:52:30)

Copyright (c) 1997-2007 The PHP Group

Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies

Setting Up a yum Repository

While there are many Linux users who prefer other tools for doing their patch management, yum is designed for RPM-based distributions. Its popularity has grown as Fedora Linux has adapted yum as its primary patch management tool. This chapter will help you set up and configure a yum repository for your Linux distribution.

Configuring a Local yum Server

The skills you need to configure a local yum server are not related to yum. They require knowledge of the FTP, Apache, or NFS services. While this section is not designed to provide a complete guide to any of these services, it provides a description of how you can configure a yum repository server based on the settings described earlier for Fedora Core 4 updates.

Because I personally prefer the efficiency of the FTP and NFS services for sharing files, I’ve covered the configuration steps required only for those services. For completeness, I’ve described how you can configure an Apache server to share files on a RHEL 4 yum repository near the end of this chapter. If you’re configuring a yum server, what you do will vary based on the following factors:

  • Distribution and version

  • Preferred version of a server (configuration steps vary for different FTP and HTTP servers and, to some extent, the way different distributions implement NFS servers)

  • Availability of yum for the distribution (i.e., if it isn’t available, be prepared to compile yum from a source RPM)

As with other network services, yum servers may be sensitive to any firewalls that you may configure. If you have a firewall between the yum server computer and associated clients, you’ll need to make sure traffic can travel through appropriate TCP/IP ports; for example, Apache services require access through TCP/IP port 80. Sure, there are ways to "tunnel" data through other services, such as SSH, but that should not be necessary for updates limited to your internal network. In any case, that level of detail is beyond the scope of this book.

Configuring an FTP yum Server

On current Red Hat/Fedora distributions, the default FTP server is vsFTP. According to its home page at http://vsftpd.beasts.org/, it’s the default FTP server used to share a number of Linux distributions, including Red Hat and Debian. It’s even used to share kernels through ftp.kernel.org.

The default version of vsFTP is configured in the vsftpd RPM. The default installation works well in most cases. The vsFTP configuration file is stored in /etc/vsftpd/vsftpd.conf for Red Hat/Fedora distributions (/etc/vsftpd.conf for SUSE and Debian distributions). In this case, we’re working from the RHEL 4 version of vsFTP.

By default, vsFTP files are stored in the /var/ftp directory. By convention, files that you copy for downloads are stored in the pub/ subdirectory. Therefore, the repository that you create should be in the /var/ftp/pub directory. For the example described earlier, update RPMs are stored in the following directory:

/var/ftp/pub/yum/4/i386/updates

If you’ve used the commands described earlier for Fedora Core 4, you’ll find the update header database in the repodata/ subdirectory.

There is a substantial number of options for vsFTP, most of which you can configure in the vsftpd.conf configuration file. The following is a review of active options in the RHEL 4 version of this file. If you want to check the current defaults for these and other options, read the vsftpd.conf man page associated with your vsFTP RPM.

anonymous_enable=yes

You absolutely want to enable anonymous access for a FTP-based yum server. Anonymous access is normally enabled by default on a vsFTP server.

local_enable=yes

It’s normally best to disable access by regular users to a FTP-based yum server. As it is disabled by default, all you need to do is comment out this option.

write_enable=yes

Naturally, because you do not want anyone (unless authorized) to overwrite (or even add) to a yum-repository, you should disable write access.

local_umask=022

If you do authorize write access (I believe you should not do so on an FTP-based yum server), this option sets the umask for any files created by users who are logged into your FTP server.

dirmessage_enable=yes

If active, this option looks for and reads any .message file that exists in the local directory. This can be useful if you want to send messages to other administrators.

xferlog_enable=yes

If active, this option logs downloads (and uploads) on the vsFTP server in the /var/log/xferlog file. For example, when I downloaded an updated version of yum, the vsFTP server placed the following entry in that file:

Mon Sep 2 17:18:25 2005 1 192.168.0.20 390363 /pub/yum/4/i386/updates/yum-2.4.0-0.fc4.noarch.rpm b _ o a anonymous@ ftp 0 * c

As you can see, this lists the date and time of the transfer, the client IP address, as well as the size and location of the file. This is a standard format shared with the WU-FTP server and can be mined as a database for more information on client computers that connect to your server.

connect_from_port_20=yes

Some FTP clients require this option, which uses TCP/IP port 20 for data transfers.

xferlog_std_format=yes

If you’ve activated the xferlog_enable option noted earlier, this option supports a standard format shared with the WU-FTP server.

pam_service_name=vsftpd

The pam_service_name option defers to Pluggable Authentication Modules to help secure the vsFTP service. This particular option sets rules in /etc/pam.d/vsftpd. One of the key options in this file prohibits users listed in /etc/vsftpd.ftpusers from logging into this vsFTP server.

userlist_enable=YES

As configured, this is redundant with the previous command. When enabled, it makes the vsFTP server read the /etc/vsftpd.user_list file and deny access to all who attempt to connect as one of the users listed in this file. By default, this file contains the same list of users as shown in /etc/vsftpd.users.

listen=YES

By default, Red Hat / Fedora configures the vsFTP server as a standalone service, with a vsftpd activation script in the /etc/rc.d/init.d directory. In contrast, SUSE does not configure vsFTP as a stand-alone service and configures listen=NO by default.

tcp_wrappers=YES

As configured, Red Hat / Fedora configures the vsFTP server for one more level of security, through TCP Wrappers support, which allows you to configure more security related commands in the /etc/hosts.allow and /etc/hosts.deny files.When you’re satisfied with the configuration, you should activate the vsFTP server with the following command (which is not required if you’ve set listen=NO):

/etc/init.d/vsftpd start 

Finally, you can make sure that vsFTP is active the next time you reboot Linux with the following command:

chkconfig vsftpd on

This command activates the vsFTP server whenever you’re in run levels 2, 3, 4, or 5. (If you configure vsFTP in xinetd, it activates it in the /etc/xinetd.d directory.) For the purpose of this chapter, assume the name of this server is yum.example.com.

Configuring a yum Client for an FTP-Based yum Repository

After you’ve configured this FTP yum server, configuring the associated yum client is a straightforward process. As you’ve seen in Chapter 6, yum configuration files that point to yum servers are normally configured in the /etc/yum.repos.d directory. For Fedora Core 4, we will examine the client file that points to the yum Update server: fedora-updates.repo.

For the yum FTP server as configured, all you need to include in the fedora-updates.repo file is the following:

[updates-released]
name=Fedora Core $releasever - $basearch - Released Updates
baseurl=ftp://yum.example.com/pub/yum/4/i386/updates
enabled=1
gpgcheck=1

As described earlier, for a vsFTP server, this means that update RPMs as well as the associated repodata/ subdirectory are stored on the yum.example.com computer in the /var/ftp/pub/yum/4/i386/updates directory.

Configuring an NFS yum Server

On current Red Hat/Fedora distributions, an NFS server is installed by default, courtesy of the nfs-utils RPM. The default installation works well in most cases. You can specify shared NFS directories in the /etc/exports configuration file. In this case, we’re working from the RHEL 4 version of NFS.

You can share directories as configured on an NFS server. For the example described earlier, update RPMs are stored in the following directory:

/var/ftp/pub/yum/4/i386/updates

Therefore, you can share this directory at any level, as long as the mount point on the NFS client is consistent. For example, I’ve added the following line to my /etc/exports configuration file:

/var/ftp/pub  192.168.1.0/24(ro,sync)

This particular configuration command shares the /var/ftp/pub directory. It limits access to clients in the noted IP address range. Clients are allowed read-only (ro) access. Changes are committed to disk before any new requests are made (sync).

For more information, see the exports man page Linux Administration Handbook by Evi Nemeth, Garth Snyder, and Trent Hein (Upper Saddle River, NJ: Prentice Hall, 2002). After you’re satisfied with the configuration in /etc/exports, deactivate the NFS server with the following command:

/etc/init.d/nfs stop 

By default, this should stop any NFS services, quotas, the NFS daemon, as well as the mountd daemon. Next, export the changes to /etc/exports with the following command:

exportfs -a

Now restart the NFS services with the following command:

/etc/init.d/nfs start 

Confirm the exports in from the local list with the following command:

showmount -e

If you’re on another system, you can find the shared NFS directories. For example, you can list those on a server named yum.example.com with the following command:

showmount -e yum.example.com

Finally, you can make sure that NFS is active the next time you reboot Linux with the following command:

chkconfig nfs on

This command activates the NFS server whenever you’re in run levels 2, 3, 4, or 5.

Configuring an NFS yum Client

You’ll need to mount the NFS share on an appropriate local directory, and then configure the associated file in /etc/yum.repos.d to point to that share. Because we’re configuring a share for Fedora updates, we’ll modify the fedora-updates.repo file.

First, on the NFS client, you should confirm your ability to connect to a shared NFS directory. The following command connects to the yum.example.com NFS server to find what shares are available on that server:

showmount -e yum.example.com

You’ll see the shared directories that you configured earlier, including /var/ftp/pub. I’ve mounted it on the local /var/yum directory with the following command:

mount yum.example.com:/var/ftp/pub /var/yum

If the /var/yum directory does not yet exist, you’ll get an error message. Now you can configure your fedora-updates.repo file in your /etc/yum.repos.d directory. For the yum NFS server as configured, all you need in fedora-updates.repo is the following:

[updates-released]
name=Fedora Core $releasever - $basearch - Released Updates
baseurl=file:///var/yum/yum/4/i386/updates
enabled=1
gpgcheck=1

As described earlier, for a vsFTP server, this means that update RPMs as well as the associated repodata/ subdirectory are stored in the yum/4/i386/updates subdirectory, mounted on the /var/yum directory.

Note the syntax associated with the baseurl command. The file: command works in place of ftp: or http:. The triple forward slash (///) is the standard syntax required for mounted directories.

If you’ve configured a Fedora Updates repository on a Fedora Core server, this command may be slightly different. Based on the directory specified earlier, you would substitute the following baseurl command:

baseurl=file:///var/ftp/pub/yum/4/i386/updates 

Naturally, you may want to configure this shared directory as part of the boot process for each NFS client. It’s possible to configure it in the default /etc/fstab configuration file, as well as through the Automounter daemon. I recommend the latter, which avoids hangups when there are network problems. The Automounter daemon is easy to configure; it requires the autofs RPM. After that RPM is installed, here’s how you can configure a NFS client for Fedora updates as configured in this chapter:

  1. Install the autofs RPM; if you’ve configured yum, the simplest way is with the following command. Even if you’re not sure if autofs is installed, this command makes sure that you have the latest version of the Automounter:

    yum install autofs
  2. Configure the Automounter master file to read from /etc/auto.misc. Open the /etc/auto.master configuration file. You’ll see sample commands; activate the following to read from the noted file. The timeout prevents your system from hanging if there’s a problem with your network or the NFS server.

    /misc /etc/auto.misc --timeout=60 

    Automounter shares configured in /etc/auto.misc are configured as subdirectories of /misc.

  3. Configure the Automounter /etc/auto.misc file to read from the shared NFS directory. Based on the shared directory and NFS server name described earlier, add the following line to that file:

    yum  -ro,soft,intr   yum.example.com:/var/ftp/pub
  4. Start the Automounter service with the following command:

    /etc/init.d/autofs start 
  5. Test the result. If your network is connected and the NFS server is running, you should be able to see the shared NFS directory with the following command:

    ls /misc/yum

    Occasionally, you may need to run this command more than once to establish the connection.

  6. Configure the /etc/yum.repos.d/fedora-updates.repo file to point to this directory as shared. Based on the previously shared NFS directory, the baseurl command would be

    baseurl=file:///var/ftp/pub/yum/4/i386/updates 
  7. Test the result with the yum update command. You should see messages similar to a regular yum update from other local or remote servers.

Friday, 9 January 2009

Adding Yum to CentOS 5

I use a lot of VPS and often times, they don’t actually have yum to make my life easier. So here is a quick HOWTO on installing yum on a CentOS box. This assumes that you have rpm and wget already installed. Note: This will only work on CentOS 5.2 while the mirror is still active.

Run the following code in a temporary directory to download all the RPMs.

#!/bin/bash

for file in \
elfutils-0.125-3.el5.i386.rpm \
elfutils-libs-0.125-3.el5.i386.rpm \
expat-1.95.8-8.2.1.i386.rpm \
gmp-4.1.4-10.el5.i386.rpm \
libxml2-2.6.26-2.1.2.1.i386.rpm \
libxml2-python-2.6.26-2.1.2.1.i386.rpm \
m2crypto-0.16-6.el5.2.i386.rpm \
python-2.4.3-21.el5.i386.rpm \
python-elementtree-1.2.6-5.i386.rpm \
python-iniparse-0.2.3-4.el5.noarch.rpm \
python-sqlite-1.1.7-1.2.1.i386.rpm \
python-urlgrabber-3.1.0-2.noarch.rpm \
readline-5.1-1.1.i386.rpm \
rpm-4.4.2-48.el5.i386.rpm \
rpm-libs-4.4.2-48.el5.i386.rpm \
rpm-python-4.4.2-48.el5.i386.rpm \
sqlite-3.3.6-2.i386.rpm \
yum-3.2.8-9.el5.centos.1.noarch.rpm \
yum-metadata-parser-1.1.2-2.el5.i386.rpm
do wget http://mirror.centos.org/centos-5/5.2/os/i386/CentOS/$file;
done

Once you have downloaded the necessary files. Install them all by typing:
# rpm -Uvh *.rpm

Then feel free to # yum -y update to bring your system up to date.