Friday, 18 May 2012

MySQL Error: Client requested master to start replication from impossible position

Error reading packet from server: Client requested master to start replication from impossible position (server_errno=1236)
Got fatal error 1236: 'Client requested master to start replication from impossible position' from master when reading data from binary log 
 
This might happen, for example, if there is a power failure on the master and it is unable to write out all the transactions to the replication log.

The proper way to fix it


Start replication all over.  Follow normal instructions on taking the main database down, record the master position, make a snapshot, copy it over to slave, start things back up, etc.

A more lazy clever approach


Assmption:  Lets assume that your slave was up-to-date with the master when things went sour.  This should usually be the case on a properly-configured not-overly-loaded replication setup.

First, lets look at the master status on the master:

mysql> show master status;
+----------------+----------+--------------+------------------+
| File           | Position | Binlog_do_db | Binlog_ignore_db |
+----------------+----------+--------------+------------------+
| box162-bin.014 | 29510700 |              |                  |
+----------------+----------+--------------+------------------+
1 row in set (0.00 sec)

Okay, lets assume that when you rebooted the master or brought it back up after the power failure, that it will have started this new log, number 014.

My slave was stuck on number 013, and giving the "Client requested master to start replication from impossible position" error.

How to fix?  Well, again, with the big assumption that the slave was generally up-to-date anyway, I simply went to the slave and did:

slave stop;
CHANGE MASTER TO MASTER_HOST='sunabc.net', MASTER_USER='repluser',MASTER_PASSWORD='replPass', MASTER_LOG_FILE='mysql-bin.000005', MASTER_LOG_POS=0
slave start;

And there we have it!  The slave will start replicating on the new file.  When power goes out and you have incomplete replication logs like this, we're really hoping that things are close enough in sync that this works.

Just do a "show slave status" on your slave, and make sure Slave_IO_Running and Slave_SQL_Running both say yes.

If you get errors about duplicate keys or other SQL inconsistency, then sorry, you were not as lucky this time.  Do the normal backup.
 

Monday, 14 May 2012

RAID 0, RAID 1, RAID 5, RAID 10 Explained with Diagrams

RAID stands for Redundant Array of Inexpensive (Independent) Disks.
On most situations you will be using one of the following four levels of RAIDs.
  • RAID 0
  • RAID 1
  • RAID 5
  • RAID 10 (also known as RAID 1+0)
This article explains the main difference between these raid levels along with an easy to understand diagram.

In all the diagrams mentioned below:
  • A, B, C, D, E and F – represents blocks
  • p1, p2, and p3 – represents parity

RAID LEVEL 0

 







Following are the key points to remember for RAID level 0.
  • Minimum 2 disks.
  • Excellent performance ( as blocks are striped ).
  • No redundancy ( no mirror, no parity ).
  • Don’t use this for any critical system.

RAID LEVEL 1

Following are the key points to remember for RAID level 1.
  • Minimum 2 disks.
  • Good performance ( no striping. no parity ).
  • Excellent redundancy ( as blocks are mirrored ).

RAID LEVEL 5

 




Following are the key points to remember for RAID level 5.
  • Minimum 3 disks.
  • Good performance ( as blocks are striped ).
  • Good redundancy ( distributed parity ).
  • Best cost effective option providing both performance and redundancy. Use this for DB that is heavily read oriented. Write operations will be slow.

RAID LEVEL 10



 

Following are the key points to remember for RAID level 10.
  • Minimum 4 disks.
  • This is also called as “stripe of mirrors”
  • Excellent redundancy ( as blocks are mirrored )
  • Excellent performance ( as blocks are striped )
  • If you can afford the dollar, this is the BEST option for any mission critical applications (especially databases).

Friday, 4 May 2012

Install RRDTool on Red Hat Enterprise Linux

RRD is the Acronym for Round Robin Database. RRD is a system to store and display time-series data (i.e. network bandwidth, machine-room temperature, server load average). It stores the data in a very compact way that will not expand over time, and it presents useful graphs by processing the data to enforce a certain data density. It can be used either via simple wrapper scripts (from shell or Perl) or via frontends that poll network devices and put a friendly user interface on it.

Installing RRDTool on RHEL

In order to install RRDTool on Red Hat Enterprise Linux / CentOS Linux 64 bit version you need to install few development tools and libraries.

Step # 1: Install required dependencies

Login as root and type the following command:
# yum install cairo-devel libxml2-devel pango-devel pango libpng-devel freetype freetype-devel libart_lgpl-devel
Sample output:
Loading "rhnplugin" plugin
Loading "security" plugin
rhel-x86_64-server-vt-5   100% |=========================| 1.4 kB    00:00
rhn-tools-rhel-x86_64-ser 100% |=========================| 1.2 kB    00:00
rhel-x86_64-server-5      100% |=========================| 1.4 kB    00:00
Setting up Install Process
Parsing package install arguments
Package libxml2-devel - 2.6.26-2.1.2.1.x86_64 is already installed.
Package libxml2-devel - 2.6.26-2.1.2.1.i386 is already installed.
Package pango - 1.14.9-3.el5.i386 is already installed.
Package pango - 1.14.9-3.el5.x86_64 is already installed.
Package freetype - 2.2.1-20.el5_2.i386 is already installed.
Package freetype - 2.2.1-20.el5_2.x86_64 is already installed.
Resolving Dependencies
--> Running transaction check
---> Package libart_lgpl-devel.x86_64 0:2.3.17-4 set to be updated
---> Package pango-devel.i386 0:1.14.9-3.el5 set to be updated
--> Processing Dependency: libXft-devel for package: pango-devel
--> Processing Dependency: libXrender-devel for package: pango-devel
--> Processing Dependency: libXext-devel for package: pango-devel
--> Processing Dependency: libX11-devel for package: pango-devel
--> Processing Dependency: fontconfig-devel >= 2.0 for package: pango-devel
---> Package pango-devel.x86_64 0:1.14.9-3.el5 set to be updated
---> Package freetype-devel.x86_64 0:2.2.1-20.el5_2 set to be updated
---> Package libpng-devel.i386 2:1.2.10-7.1.el5_0.1 set to be updated
---> Package cairo-devel.x86_64 0:1.2.4-5.el5 set to be updated
---> Package libpng-devel.x86_64 2:1.2.10-7.1.el5_0.1 set to be updated
---> Package cairo-devel.i386 0:1.2.4-5.el5 set to be updated
---> Package libart_lgpl-devel.i386 0:2.3.17-4 set to be updated
--> Processing Dependency: libart_lgpl_2.so.2 for package: libart_lgpl-devel
---> Package freetype-devel.i386 0:2.2.1-20.el5_2 set to be updated
--> Running transaction check
---> Package libXrender-devel.i386 0:0.9.1-3.1 set to be updated
--> Processing Dependency: xorg-x11-proto-devel for package: libXrender-devel
---> Package libXft-devel.i386 0:2.1.10-1.1 set to be updated
---> Package libX11-devel.i386 0:1.0.3-9.el5 set to be updated
--> Processing Dependency: libXdmcp-devel for package: libX11-devel
--> Processing Dependency: libXau-devel for package: libX11-devel
---> Package fontconfig-devel.i386 0:2.4.1-7.el5 set to be updated
---> Package libart_lgpl.i386 0:2.3.17-4 set to be updated
---> Package libXext-devel.i386 0:1.0.1-2.1 set to be updated
--> Running transaction check
---> Package xorg-x11-proto-devel.i386 0:7.1-9.fc6 set to be updated
--> Processing Dependency: mesa-libGL-devel for package: xorg-x11-proto-devel
---> Package libXdmcp-devel.i386 0:1.0.1-2.1 set to be updated
---> Package libXau-devel.i386 0:1.0.1-3.1 set to be updated
--> Running transaction check
---> Package mesa-libGL-devel.i386 0:6.5.1-7.5.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 libart_lgpl-devel       x86_64     2.3.17-4         rhel-x86_64-server-5   21 k
 libart_lgpl-devel       i386       2.3.17-4         rhel-x86_64-server-5   21 k
 pango-devel             i386       1.14.9-3.el5     rhel-x86_64-server-5  280 k
 pango-devel             x86_64     1.14.9-3.el5     rhel-x86_64-server-5  281 k
Installing for dependencies:
 cairo-devel             x86_64     1.2.4-5.el5      rhel-x86_64-server-5  131 k
 cairo-devel             i386       1.2.4-5.el5      rhel-x86_64-server-5  130 k
 fontconfig-devel        i386       2.4.1-7.el5      rhel-x86_64-server-5  168 k
 freetype-devel          x86_64     2.2.1-20.el5_2   rhel-x86_64-server-5  151 k
 freetype-devel          i386       2.2.1-20.el5_2   rhel-x86_64-server-5  151 k
 libX11-devel            i386       1.0.3-9.el5      rhel-x86_64-server-5  665 k
 libXau-devel            i386       1.0.1-3.1        rhel-x86_64-server-5   11 k
 libXdmcp-devel          i386       1.0.1-2.1        rhel-x86_64-server-5  7.6 k
 libXext-devel           i386       1.0.1-2.1        rhel-x86_64-server-5   57 k
 libXft-devel            i386       2.1.10-1.1       rhel-x86_64-server-5   16 k
 libXrender-devel        i386       0.9.1-3.1        rhel-x86_64-server-5  8.9 k
 libart_lgpl             i386       2.3.17-4         rhel-x86_64-server-5   76 k
 libpng-devel            i386       2:1.2.10-7.1.el5_0.1  rhel-x86_64-server-5  182 k
 libpng-devel            x86_64     2:1.2.10-7.1.el5_0.1  rhel-x86_64-server-5  186 k
 mesa-libGL-devel        i386       6.5.1-7.5.el5    rhel-x86_64-server-5  465 k
 xorg-x11-proto-devel    i386       7.1-9.fc6        rhel-x86_64-server-5  247 k
Transaction Summary
=============================================================================
Install     20 Package(s)
Update       0 Package(s)
Remove       0 Package(s)
Total download size: 3.2 M
Is this ok [y/N]:
Downloading Packages:
(1/20): libXext-devel-1.0 100% |=========================|  57 kB    00:00
(2/20): freetype-devel-2. 100% |=========================| 151 kB    00:00
(3/20): libXau-devel-1.0. 100% |=========================|  11 kB    00:00
(4/20): libart_lgpl-devel 100% |=========================|  21 kB    00:00
(5/20): libart_lgpl-2.3.1 100% |=========================|  76 kB    00:00
(6/20): cairo-devel-1.2.4 100% |=========================| 130 kB    00:00
(7/20): libpng-devel-1.2. 100% |=========================| 186 kB    00:00
(8/20): cairo-devel-1.2.4 100% |=========================| 131 kB    00:00
(9/20): fontconfig-devel- 100% |=========================| 168 kB    00:00
(10/20): mesa-libGL-devel 100% |=========================| 465 kB    00:01
(11/20): libXdmcp-devel-1 100% |=========================| 7.6 kB    00:00
(12/20): libpng-devel-1.2 100% |=========================| 182 kB    00:00
(13/20): libX11-devel-1.0 100% |=========================| 665 kB    00:02
(14/20): freetype-devel-2 100% |=========================| 151 kB    00:00
(15/20): libXft-devel-2.1 100% |=========================|  16 kB    00:00
(16/20): pango-devel-1.14 100% |=========================| 281 kB    00:01
(17/20): pango-devel-1.14 100% |=========================| 280 kB    00:01
(18/20): libXrender-devel 100% |=========================| 8.9 kB    00:00
(19/20): libart_lgpl-deve 100% |=========================|  21 kB    00:00
(20/20): xorg-x11-proto-d 100% |=========================| 247 kB    00:01
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: libart_lgpl                  ####################### [ 1/20]
  Installing: freetype-devel               ####################### [ 2/20]
  Installing: fontconfig-devel             ####################### [ 3/20]
  Installing: libpng-devel                 ####################### [ 4/20]
  Installing: libXau-devel                 ####################### [ 5/20]
  Installing: libart_lgpl-devel            ####################### [ 6/20]
  Installing: libart_lgpl-devel            ####################### [ 7/20]
  Installing: libpng-devel                 ####################### [ 8/20]
  Installing: freetype-devel               ####################### [ 9/20]
  Installing: xorg-x11-proto-devel         ####################### [10/20]
  Installing: libX11-devel                 ####################### [11/20]
  Installing: libXrender-devel             ####################### [12/20]
  Installing: libXft-devel                 ####################### [13/20]
  Installing: cairo-devel                  ####################### [14/20]
  Installing: libXext-devel                ####################### [15/20]
  Installing: pango-devel                  ####################### [16/20]
  Installing: pango-devel                  ####################### [17/20]
  Installing: libXdmcp-devel               ####################### [18/20]
  Installing: mesa-libGL-devel             ####################### [19/20]
  Installing: cairo-devel                  ####################### [20/20]
Installed: libart_lgpl-devel.x86_64 0:2.3.17-4 libart_lgpl-devel.i386 0:2.3.17-4 pango-devel.i386 0:1.14.9-3.el5 pango-devel.x86_64 0:1.14.9-3.el5
Dependency Installed: cairo-devel.x86_64 0:1.2.4-5.el5 cairo-devel.i386 0:1.2.4-5.el5 fontconfig-devel.i386 0:2.4.1-7.el5 freetype-devel.x86_64 0:2.2.1-20.el5_2 freetype-devel.i386 0:2.2.1-20.el5_2 libX11-devel.i386 0:1.0.3-9.el5 libXau-devel.i386 0:1.0.1-3.1 libXdmcp-devel.i386 0:1.0.1-2.1 libXext-devel.i386 0:1.0.1-2.1 libXft-devel.i386 0:2.1.10-1.1 libXrender-devel.i386 0:0.9.1-3.1 libart_lgpl.i386 0:2.3.17-4 libpng-devel.i386 2:1.2.10-7.1.el5_0.1 libpng-devel.x86_64 2:1.2.10-7.1.el5_0.1 mesa-libGL-devel.i386 0:6.5.1-7.5.el5 xorg-x11-proto-devel.i386 0:7.1-9.fc6
Complete!

Step # 2: Download latest rrdtool tar ball

Type the following commands:
# cd /opt/
# wget http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.3.1.tar.gz

Untar tar ball, enter:
# tar -zxvf rrdtool-1.3.1.tar.gz

Step #3: Compile and install rrdtool

You need to set PKG_CONFIG_PATH, enter:
# export PKG_CONFIG_PATH=/usr/lib/pkgconfig/
Type the following commands:
# ./configure
Sample output:
config.status: executing default-1 commands
config.status: executing intltool commands
config.status: executing default commands
config.status: executing po/stamp-it commands
checking in... and out again
ordering CD from http://tobi.oetiker.ch/wish .... just kidding ;-)
----------------------------------------------------------------
Config is DONE!
          With MMAP IO: yes
       Static programs: no
          Perl Modules: perl_piped perl_shared
           Perl Binary: /usr/bin/perl
          Perl Version: 5.8.8
          Perl Options: PREFIX=/usr/local/rrdtool-1.3.1 LIB=/usr/local/rrdtool-1.3.1/lib/perl/5.8.8
          Ruby Modules:
           Ruby Binary: no
          Ruby Options: sitedir=$(DESTDIR)NONE/lib/ruby
    Build Tcl Bindings: no
 Build Python Bindings: yes
          Build rrdcgi: yes
       Build librrd MT: yes
     Link with libintl: yes
             Libraries: -lxml2 -lcairo -lcairo -lcairo -lm  -lcairo -lpng12   -lpangocairo-1.0 -lpango-1.0 -lcairo -lgobject-2.0 -lgmodule-2.0 -ldl -lglib-2.0
Type 'make' to compile the software and use 'make install' to
install everything to: /usr/local/rrdtool-1.3.1.
       ... that wishlist is NO JOKE. If you find RRDtool useful
make me happy. Go to http://tobi.oetiker.ch/wish and
place an order.
                               -- Tobi Oetiker 
----------------------------------------------------------------
Now compile and install RRDTool on RHEL:
# make
# make install
# cd /usr/local/
# ln -s rrdtool-1.3.1/ rrdtool/
# cd rrdtool
# ls -l