centos 7 raid5 rebuild + grub from live ubuntu 18.04 rescue env

sudo su -
 
modprobe raid5
mdadm --stop /dev/md0
mdadm --stop /dev/md1
stop any other active md device (you can see them in /prod/mdstat)
mdadm --stop /dev/md127
mdadm --stop /dev/md126
check if they are all stopped
cat /proc/mdstat
run a new scan
mdadm --assemble --scan
check if the raid5 is properly detected
cat /proc/mdstat
copy over the partition tables from working disk (sdc here) to the new disk (sdb here)
sfdisk -d /dev/sdc | sfdisk /dev/sdb
add the devices to their corresponding arrays (start with the /boot one if you have a dedicated boot one)
mdadm /dev/md126 -a /dev/sdb2
mdadm /dev/md127 -a /dev/sdb3
now let’s get ready to fix grub on the new disk, in our example md126 is boot, md127 is /
mkdir /mnt-boot
mount /dev/md126 /mnt
mount /dev/md127 /mnt-boot
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys
mount --bind /mnt-boot /mnt/boot
chroot /mnt
you may get an error when doing the chroot if the shell is on a different path. this works for default centos install
chroot /mnt /usr/bin/bash
install grub on all the disks
grub2-install /dev/sda
grub2-install /dev/sdb
grub2-install /dev/sdc
regenerate grub.cfg
grub2-mkconfig > /boot/grub2/grub.cfg
now monitor the progress of the array rebuild process and reboot once completed
watch cat /proc/mdstat

Processing mysql dumps in hurry (convert single insert to extended insert)

Most time there’s little time, sometime there’s NO TIME!

A few days ago I had no time, and had to manipulate a badly exported database (2million+ single myisam insert statements) tuning mysqld was useless, insert delayed useless, increasing buffers useless… and so on… import was taking hours (many hours) on the target box due to impressively high disk io!

So I just fired up a vmware instance with 32gb of ram, 10gb hdd and 8cpu cores (of a xeon L56xx) and did everything in ram.
What was going to take hours on the target box took just 2minutes on the vmware instance…
Then I did a proper “mysqldump –opt” and imported it back into the target box in just 20seconds 😀

yum upgrade -y
wget -q -O - http://www.atomicorp.com/installers/atomic | sh
mkdir -p /var/lib/mysql && mount -v -t tmpfs -o size=24G none /var/lib/mysql
yum install mysql mysql-server -y
nano -w /etc/my.cnf

tune it up a little, in my case

thread_concurrency=16

was enough 🙂

service mysqld restart
mysql_secure_installation

and you are good to go!

import the bad export and after that export it making use of all the proper settings (extended queries, locking and so on) … –opt handles all of them by default 🙂

So yes… sometime I make use of “the cloud” too :O

PS: I do the same (storage on ramdisk) when I’ve to compile a linux kernel.

wget ftp download specific directory content – no recursion

This one command allows you to download the content of a directory to a local directory without doing recuirsive searches

wget -np -N --cut-dirs=1 -A .dem ftp://user:password@host.tld/tf2/orangebox/tf/*

specifically this one downloads all the “.dem” (-A .dem) (team fortress demo files) located into the remote “/tf2/orangebox/tf/” directory.
Files are saved into the current directory (–cut-dirs=1)

Additionally it makes use of timestamping (-N) in order to not download already existing files when doing a subsequent run.

Upgrading munin from 1.4.6 to 1.4.7 – re-enabling lost plugins

Upgrading munin on CentOS/RHEL/Scientific Linux using the rpm from EPEL repositories results in an empty plugin list on the nodes:

ls -al /etc/munin/plugins/

easy fix:

munin-node-configure --suggest --shell | sh ; service munin-node restart

to verify what was detected just run:

ls -al /etc/munin/plugins/

Linux fix wrong date/time hwclock

If your clock is wrong you might have a wrong hwclock set…

yum install ntp
ntpdate -s it.pool.ntp.org
hwclock -w

on a side note to change the timezone on RHEL 6.x adjust the clock settings config file:

nano -w /etc/sysconfig/clock

set it as you wish:

ZONE="Europe/Rome"

then set the localtime

cp /usr/share/zoneinfo/Europe/Rome /etc/localtime

BFD vsftpd script

BFD is an easy to use brute force detection script that plays very nicely when combined with APF…

currently it does support certain daemons out of the box… but vspftd is not one of those 🙁
This a *very basic* (it does not pass the offending username to bfd) script to add VSFTPD support to BFD.

You just need to create a file named “vsftpd” into the BFD ./rules/ directory and paste this content into it:

REQ="/usr/sbin/vsftpd"
 
if [ -f "$REQ" ]; then
 LP="/var/log/vsftpd.log"
 TLOG_TF="vsftpd"
 
 #Mon Mar 28 23:57:38 2011 [pid 9897] [asdasd] FAIL LOGIN: Client "127.0.0.1"
 
 ## VSFTPD
 ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -w 'FAIL LOGIN' | sed -r 's/^.{0,}Client .//' | sed 's/"/:vsftpd/g'`
fi

This script refers to the standard vsftpd rhel/centos installation…
If the logfile is placed elsewhere (vsftpd_log_file) or if the option “syslog_enable” in vsftpd.conf has been enabled it needs to be adjusted 🙂

PS: this was a NON-WORKING test (usernames with a space in it where making it fail):

ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -w 'FAIL LOGIN' | awk '{print $12":"$8}' | tr '[]"'`

Remote CentOS 6 install with VNC

*POST UPDATED – now referring to centos6*
I get in touch with many people related to centos reinstall… I’m aware in certain conditions it’s a pain to have it rolling properly…
I do this for work, just contact me if you want it done for a small fee 😉

There are some ISPs in the dedicated servers market that are offering crappy CentOS installs…

With “crappy” I mean: broken/modifed images…

The worst example I had the opportunity to try was OVH.
OVH is in general a very good dedicated hosting provider offering very nice services at an impressive low price… but they have the bad attitude to install custom operating systems that many times cannot be reverted back to the original kernels 🙁

One of them is their CentOS 5 install… Lucky for us it’s not a big problem as, most times, it’s possible to do a clean CentOS install 😉

Continue reading “Remote CentOS 6 install with VNC”