Friday, 6 February 2009

Linux Convert ext3 to ext4 File system

Some time ago ext4 was released and available for Linux kernel. ext4 provides some additional benefits and perforce over ext3 file system. You can easily convert ext3 to ext4 file system. The next release of Fedora, 11, will default to the ext4 file system unless serious regressions are seen. In this quick tutorial you will learn about converting ext3 to ext4 file system.

ext4 Filesystem Features

The ext4 filesystem has more features and generally better performance than ext3, which is showing its age in the Linux filesystem world. Features include:

Delayed allocation & mballoc allocator for better on-disk allocation

  • Sub-second timestamps
  • Space preallocation
  • Journal checksumming
  • Large (>2T) file support
  • Large (>16T) filesystem support
  • Defragmentation support
WARNING! Once you run following commands, the filesystem will no longer be mountable using the ext3. Please note that ext4 may have some bugs so do not use for production servers (wait for sometime watch Linux kernel mailing list for ext4 bugs). It's recommended that you keep /boot in a ext3 partition for sometime.

You need ext4 patch applied into kernel and compile kernel with ext4 support. Once done type the following command to convert an existing ext3 filesystem to use ext4, type:

# tune2fs -O extents,uninit_bg,dir_index /dev/dev-name

For example convert /dev/sdb1 to ext4, enter:

# cd /; umount /dev/sdb1
# tune2fs -O extents,uninit_bg,dir_index /dev/sdb1

Next run fsck, enter:

# fsck -pf /dev/sdb1

How do I mount ext4 partition?

mount -t ext4 /dev/sdb1 /path
mount -t ext4 /dev/sdb1 /share
mount -t ext4 /dev/disk/by-uuid/YOUR-PARTITION-UUID /share

Use blkid to get UUID.

How do I boot from ext4 (/boot)?

If you have converted /boot file system (or / used for /boot), update /boot/grub.conf (/boot/grub/menu.lst). Open file and find out current kernel config file and append the following:

rootfstype=ext4

Here is sample config (note I've custom kernel names):

title           Ubuntu 8.10, kernel2.6.28.1-vmware-guest-server		

root (hd0,1)
kernel/boot/vmlinuz-2.6.28.1-vmware-guest-serverroot=UUID=8c2da865-13f4-47a2-9c92-2f31738469e8 ro quiet splash rootfstype=ext4
initrd /boot/initrd.img-2.6.28.1-vmware-guest-server
quiet

Save and close the file. And run update-grub:

$ sudo update-grub


Next, update your /etc/fstab file so that it can be mounted as ext4 file system:

UUID=41c22818-fbad-4da6-8196-c816df0b7aa8 	/share	ext4	defaults,errors=remount-ro,relatime 0       1

Finally, reboot the system:

$ sudo reboot

Search for all account without password and lock them

For security, reason it is necessary to disable all account(s) with no password and lock them down. Solaris, Linux and FreeBSD provide account locking (unlocking) facility.

Lock Linux user account with the following command:

passwd -l {user-name}

Solaris UNIX display password status

passwd  -u {user-name}

-l : This option disables an account by changing the password to a value, which matches no possible encrypted value.

Lock FreeBSD user account with the following command:

pw lock {username}

FreeBSD unlocking the account use:

pw unlock {username}

Lock Solaris UNIX user account with the following command:

passwd -l {username}

Lock HP-UX user account with the following command:

passwd -l {username}

For unlocking the HP-UX account you need to edit /etc/passwd file using text editor (or use SAM):

vi /etc/passwd 

However, how will you find out account without password? Again, with the help of 'passwd -s' (status) command you can find out all passwordless accounts.

Linux display password status

passwd -S {user-name}

Where,
-S : Display account status information. The status information consists of total seven fields. The second field indicates the status of password using following format:

  • L : if the user account is locked (L)
  • NP : Account has no password (NP)
  • P: Account has a usable password (P)
# passwd -S radmin

radmin P 10/08/2005 0 99999 7 -1

Solaris UNIX display password status

passwd -s {user-name}

Where,
-s : Display account status information using following format:

  • PS : Account has a usable password
  • LK : User account is locked
  • NP : Account has no password

FreeBSD
I have already written about small awk one line approach to find out all passwords less accounts.

Automated Scripting Solution
However, in real life you write a script and execute it from cron job. Here is small script for Linux:

#!/bin/sh
USERS="$(cut -d: -f 1 /etc/passwd)"
for u in $USERS
do
passwd -S $u | grep -Ew "NP" >/dev/null
if [ $? -eq 0 ]; then
passwd -l $u
fi
done

FreeBSD script:

#!/bin/bash
USERS="$(awk -F: 'NF > 1 && $1 !~ /^[#+-]/ && $2=="" {print $0}'
/etc/master.passwd | cut -d: -f1)"
for u in $USERS
do
pw lock $u
done

Sun Solaris script:

#!/bin/sh
USERS=`passwd -sa | grep -w NP | awk '{ print $1 }'`
for u in $USERS
do
passwd -l $u
done

You can easily add email alert support to script so that when ever scripts finds passwordless account(s) it will send an email alert. See the complete working example of script here.

Wednesday, 4 February 2009

A script to add user in Linux

# Path to files
pfile=/etc/passwd
gfile=/etc/group
sfile=/etc/shells

# Paths to binaries
useradd=/usr/sbin/useradd
chfn=/usr/bin/chfn
passwd=/usr/bin/passwd
chmod=/bin/chmod

# Defaults
defhome=/home
defshell=/bin/bash
defchmod=711 # home dir permissions - may be preferable to use 701, however.
defgroup=users
AGID="audio cdrom floppy plugdev video" # additional groups for desktop users

# Determine what the minimum UID is (for UID recycling)
# (we ignore it if it's not at the beginning of the line (i.e. commented out with #))
export recycleUIDMIN="$(grep ^UID_MIN /etc/login.defs | awk '{print $2}' 2>/dev/null)"
# If we couldn't find it, set it to the default of 1000
if [ -z "$recycleUIDMIN" ]; then
export recycleUIDMIN=1000
fi


# This setting enables the 'recycling' of older unused UIDs.
# When you userdel a user, it removes it from passwd and shadow but it will
# never get used again unless you specify it expliticly -- useradd (appears to) just
# look at the last line in passwd and increment the uid. I like the idea of
# recycling uids but you may have very good reasons not to (old forgotten
# confidential files still on the system could then be owned by this new user).
# We'll set this to no because this is what the original adduser shell script
# did and it's what users expect.
recycleuids=no

# Function to read keyboard input.
# bash1 is broken (even ash will take read -ep!), so we work around
# it (even though bash1 is no longer supported on Slackware).
function get_input() {
local output
if [ "`echo $BASH_VERSION | cut -b1`" = "1" ]; then
echo -n "${1} " >&2 # fudge for use with bash v1
read output
else # this should work with any other /bin/sh
read -ep "${1} " output
fi
echo $output
}

# Function to display the account info
function display () {
local goose
goose="$(echo $2 | cut -d ' ' -f 2-)" # lop off the prefixed argument useradd needs
echo -n "$1 "
# If it's null then display the 'other' information
if [ -z "$goose" -a ! -z "$3" ]; then
echo "$3"
else
echo "$goose"
fi
}

# Function to check whether groups exist in the /etc/group file
function check_group () {
local got_error group
if [ ! -z "$@" ]; then
for group in $@ ; do
local uid_not_named="" uid_not_num=""
grep -v "$^" $gfile | awk -F: '{print $1}' | grep "^${group}$" >/dev/null 2>&1 || uid_not_named=yes
grep -v "$^" $gfile | awk -F: '{print $3}' | grep "^${group}$" >/dev/null 2>&1 || uid_not_num=yes
if [ ! -z "$uid_not_named" -a ! -z "$uid_not_num" ]; then
echo "- Group '$group' does not exist"
got_error=yes
fi
done
fi
# Return exit code of 1 if at least one of the groups didn't exist
if [ ! -z "$got_error" ]; then
return 1
fi
}

#: Read the login name for the new user :#
#
# Remember that most Mail Transfer Agents are case independant, so having
# 'uSer' and 'user' may cause confusion/things to break. Because of this,
# useradd from shadow-4.0.3 no longer accepts usernames containing uppercase,
# and we must reject them, too.

# Set the login variable to the command line param
echo
LOGIN="$1"
needinput=yes
while [ ! -z $needinput ]; do
if [ -z "$LOGIN" ]; then
while [ -z "$LOGIN" ]; do LOGIN="$(get_input "Login name for new user []:")" ; done
fi
grep "^${LOGIN}:" $pfile >/dev/null 2>&1 # ensure it's not already used
if [ $? -eq 0 ]; then
echo "- User '$LOGIN' already exists; please choose another"
unset LOGIN
elif [ ! -z "$( echo $LOGIN | grep "^[0-9]" )" ]; then
echo "- User names cannot begin with a number; please choose another"
unset LOGIN
elif [ ! "$LOGIN" = "`echo $LOGIN | tr A-Z a-z`" ]; then # useradd does not allow uppercase
echo "- User '$LOGIN' contains illegal characters (uppercase); please choose another"
unset LOGIN
elif [ ! -z "$( echo $LOGIN | grep '\.' )" ]; then
echo "- User '$LOGIN' contains illegal characters (period/dot); please choose another"
unset LOGIN
else
unset needinput
fi
done

# Display the user name passed from the shell if it hasn't changed
if [ "$1" = "$LOGIN" ]; then
echo "Login name for new user: $LOGIN"
fi

#: Get the UID for the user & ensure it's not already in use :#
#
# Whilst we _can_ allow users with identical UIDs, it's not a 'good thing' because
# when you change password for the uid, it finds the first match in /etc/passwd
# which isn't necessarily the correct user
#
echo
needinput=yes
while [ ! -z "$needinput" ]; do
_UID="$(get_input "User ID ('UID') [ defaults to next available ]:")"
grep -v "^$" $pfile | awk -F: '{print $3}' | grep "^${_UID}$" >/dev/null 2>&1
if [ $? -eq 0 ]; then
echo "- That UID is already in use; please choose another"
elif [ ! -z "$(echo $_UID | egrep '[A-Za-z]')" ]; then
echo "- UIDs are numerics only"
else
unset needinput
fi
done
# If we were given a UID, then syntax up the variable to pass to useradd
if [ ! -z "$_UID" ]; then
U_ID="-u ${_UID}"
else
# Will we be recycling UIDs?
if [ "$recycleuids" = "yes" ]; then
U_ID="-u $(awk -F: '{uid[$3]=1} END { for (i=ENVIRON["recycleUIDMIN"];i in uid;i++);print i}' $pfile)"
fi
fi

#: Get the initial group for the user & ensure it exists :#
#
# We check /etc/group for both the text version and the group ID number
echo
needinput=yes
while [ ! -z "$needinput" ]; do
GID="$(get_input "Initial group [ ${defgroup} ]:")"
check_group "$GID"
if [ $? -gt 0 ]; then
echo "- Please choose another"
else
unset needinput
fi
done
# Syntax the variable ready for useradd
if [ -z "$GID" ]; then
GID="-g ${defgroup}"
else
GID="-g ${GID}"
fi

#: Get additional groups for the user :#
#
echo "Additional UNIX groups:"
echo
echo "Users can belong to additional UNIX groups on the system."
echo "For local users using graphical desktop login managers such"
echo "as XDM/KDM, users may need to be members of additional groups"
echo "to access the full functionality of removable media devices."
echo
echo "* Security implications *"
echo "Please be aware that by adding users to additional groups may"
echo "potentially give access to the removable media of other users."
echo
echo "If you are creating a new user for remote shell access only,"
echo "users do not need to belong to any additional groups as standard,"
echo "so you may press ENTER at the next prompt."
echo
needinput=yes
while [ ! -z "$needinput" ]; do
history -c
history -s "$AGID"
echo "Press ENTER to continue without adding any additional groups"
echo "Or press the UP arrow to add/select/edit additional groups"
AGID="$(get_input ": " | tr -d '[:punct:]' | tr -s ' ' | sed 's?^ $??g' )"
if [ ! -z "$AGID" ]; then
check_group "$AGID" # check all groups at once (treated as N # of params)
if [ $? -gt 0 ]; then
echo "- Please re-enter the group(s)"
echo
else
unset needinput # we found all groups specified
AGID="-G $(echo $AGID | tr ' ' ,)" # useradd takes comma delimited groups
fi
else
unset needinput # we don't *have* to have additional groups
fi
done

#: Get the new user's home dir :#
#
echo
needinput=yes
while [ ! -z "$needinput" ]; do
HME="$(get_input "Home directory [ ${defhome}/${LOGIN} ]")"
if [ -z "$HME" ]; then
HME="${defhome}/${LOGIN}"
fi
# Warn the user if the home dir already exists
if [ -d "$HME" ]; then
echo "- Warning: '$HME' already exists !"
getyn="$(get_input " Do you wish to change the home directory path ? (Y/n) ")"
if [ "$(echo $getyn | grep -i "n")" ]; then
unset needinput
# You're most likely going to only do this if you have the dir *mounted* for this user's $HOME
getyn="$(get_input " Do you want to chown $LOGIN.$( echo $GID | awk '{print $2}') $HME ? (y/N) ")"
if [ "$(echo $getyn | grep -i "y")" ]; then
CHOWNHOMEDIR=$HME # set this to the home directory
fi
fi
else
unset needinput
fi
done
HME="-d ${HME}"

#: Get the new user's shell :#
echo
needinput=yes
while [ ! -z "$needinput" ]; do
unset got_error
SHL="$(get_input "Shell [ ${defshell} ]")"
if [ -z "$SHL" ]; then
SHL="${defshell}"
fi
# Warn the user if the shell doesn't exist in /etc/shells or as a file
if [ -z "$(grep "^${SHL}$" $sfile)" ]; then
echo "- Warning: ${SHL} is not in ${sfile} (potential problem using FTP)"
got_error=yes
fi
if [ ! -f "$SHL" ]; then
echo "- Warning: ${SHL} does not exist as a file"
got_error=yes
fi
if [ ! -z "$got_error" ]; then
getyn="$(get_input " Do you wish to change the shell ? (Y/n) ")"
if [ "$(echo $getyn | grep -i "n")" ]; then
unset needinput
fi
else
unset needinput
fi
done
SHL="-s ${SHL}"

#: Get the expiry date :#
echo
needinput=yes
while [ ! -z "$needinput" ]; do
EXP="$(get_input "Expiry date (YYYY-MM-DD) []:")"
if [ ! -z "$EXP" ]; then
# Check to see whether the expiry date is in the valid format
if [ -z "$(echo "$EXP" | grep "^[[:digit:]]\{4\}[-]\?[[:digit:]]\{2\}[-]\?[[:digit:]]\{2\}$")" ]; then
echo "- That is not a valid expiration date"
else
unset needinput
EXP="-e ${EXP}"
fi
else
unset needinput
fi
done

# Display the info about the new impending account
echo
echo "New account will be created as follows:"
echo
echo "---------------------------------------"
display "Login name.......: " "$LOGIN"
display "UID..............: " "$_UID" "[ Next available ]"
display "Initial group....: " "$GID"
display "Additional groups: " "$AGID" "[ None ]"
display "Home directory...: " "$HME"
display "Shell............: " "$SHL"
display "Expiry date......: " "$EXP" "[ Never ]"
echo

echo "This is it... if you want to bail out, hit Control-C. Otherwise, press"
echo "ENTER to go ahead and make the account."
read junk

echo
echo "Creating new account..."
echo
echo

# Add the account to the system
CMD="$useradd "$HME" -m "$EXP" "$U_ID" "$GID" "$AGID" "$SHL" "$LOGIN""
$CMD

if [ $? -gt 0 ]; then
echo "- Error running useradd command -- account not created!"
echo "(cmd: $CMD)"
exit 1
fi

# chown the home dir ? We can only do this once the useradd has
# completed otherwise the user name doesn't exist.
if [ ! -z "${CHOWNHOMEDIR}" ]; then
chown "$LOGIN"."$( echo $GID | awk '{print $2}')" "${CHOWNHOMEDIR}"
fi

# Set the finger information
$chfn "$LOGIN"
if [ $? -gt 0 ]; then
echo "- Warning: an error occurred while setting finger information"
fi

# Set a password
$passwd "$LOGIN"
if [ $? -gt 0 ]; then
echo "* WARNING: An error occured while setting the password for"
echo " this account. Please manually investigate this *"
exit 1
fi

# If it was created (it should have been!), set the permissions for that user's dir
HME="$(echo "$HME" | awk '{print $2}')" # We have to remove the -g prefix
if [ -d "$HME" ]; then
$chmod $defchmod "$HME"
fi

echo
echo
echo "Account setup complete."
exit 0