Category: Linux

lvcreate – Aborting. Failed to wipe start of new LV

root@linux:~ # lvcreate -L 400g -n lv_test vg_test
Volume “vg_test/lv_test” is not active locally.
Aborting. Failed to wipe start of new LV.

Add vg_test to volume_list parameter in /etc/lvm/lvm.conf

vi /etc/lvm/lvm.conf
volume_list = [ “vgo2o”, “vgroot”, “vg_test” ]

Then try again

root@linux:~ # lvcreate -L 400g -n lv_test vg_test
Logical volume “lv_test” created.

Rotate file /var/log/faillog, /var/log/lastlog and /var/log/tallylog

If you are having disk space problems in /var and you found that /var/log/faillog, /var/log/lastlog and /var/log/tallylog are filling up the space and you need to rotate them, you probably don’t need to rotate them

root@linux:~ # df -h /var/log
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vgroot-lv_var_log
4.9G 2.0G 2.8G 42% /var/log

These are sparse files and are occupying minimal disk space. Keep looking for the offender

root@linux:~ # ls -lh /var/log/faillog
-rw——-. 1 root root 258M Apr 25 14:07 /var/log/faillog

root@linux:~ # du -sh /var/log/faillog
624K /var/log/faillog
root@linux:~ # du -h –apparent-size /var/log/faillog
258M /var/log/faillog

root@linux:~ # ls -lh /var/log/lastlog
-rw——-. 1 root root 2.3G May 2 11:05 /var/log/lastlog

root@linux:~ # du -h /var/log/lastlog
348K /var/log/lastlog
root@linux:~ # du -h –apparent-size /var/log/lastlog
2.3G /var/log/lastlog

root@linux:~ # ls -lh /var/log/tallylog
-rw——-. 1 root root 515M May 2 10:42 /var/log/tallylog

root@linux:~ # du -sh /var/log/tallylog
288K /var/log/tallylog
root@linux:~ # du -sh –apparent-size /var/log/tallylog
515M /var/log/tallylog

Source: faillog command create a huge file, like 128GB file (/var/log/faillog)

Why is the /var/log/lastlog file so large?

error while loading shared libraries: libnl.so.1

error while loading shared libraries: libnl.so.1

Check which package provides the library that you want.

On Red Hat Linux you can check running yum whatprovides. On my case, I need to install libnl to solve my problem

root@linux:~ # yum whatprovides */libnl.so.1
Loaded plugins: downloadonly, rhnplugin, security

libnl-1.0-0.10.pre5.4.i386 : Convenience library for kernel netlink sockets
Repo : internalrepository
Matched from:
Filename : /usr/lib/libnl.so.1

libnl-1.0-0.10.pre5.5.i386 : Convenience library for kernel netlink sockets
Repo : internalrepository
Matched from:
Filename : /usr/lib/libnl.so.1

libnl-1.0-0.10.pre5.5.x86_64 : Convenience library for kernel netlink sockets
Repo : internalrepository
Matched from:
Filename : /usr/lib64/libnl.so.1

libnl-1.0-0.10.pre5.4.x86_64 : Convenience library for kernel netlink sockets
Repo : internalrepository
Matched from:
Filename : /usr/lib64/libnl.so.1

Default vsftpd log file location

The default log location for vsftpd is not /var/log/vsftpd.log

It is on /var/log/xferlog

root@linux:~ # cat /etc/vsftpd/vsftpd.conf
# The target log file can be vsftpd_log_file or xferlog_file.
# This depends on setting xferlog_std_format parameter
xferlog_enable=YES

# The name of log file when xferlog_enable=YES and xferlog_std_format=YES
# WARNING – changing this filename affects /etc/logrotate.d/vsftpd.log
xferlog_file=/var/log/xferlog

If syslog_enable is set, vsftpd will log everything to /var/log/messages

UXMON: ServiceGuard: Package(s) are running on ADOPTIVE node(s): infraHP0 (linux02)

UXMON: ServiceGuard: Package(s) are running on ADOPTIVE node(s): infraHP0 (linux02)

Node : linux01.setaoffice.com
Node Type : Intel/AMD x64(HTTPS)
Severity : major
OM Server Time: 2016-02-22 14:54:56
Message : UXMON: ServiceGuard: Package(s) are running on ADOPTIVE node(s): infraHP0 (linux02)
Msg Group : OS
Application : sgmon
Object : cmviewcl
Event Type :
not_found

Instance Name :
not_found

Instruction : Check with cmviewcl -v;

If package was down the customer has to be informed about the downtime.

Do not close this case before it is resolved.
As long as this EWM-case is not resolved or closed, monitoring is disabled
In any case inform technical lead!

This ticket is about the cluster package that it is running on alternate node.

root@linux:~ # cp -p /var/opt/OV/bin/instrumentation/sg_mon.cfg /var/opt/OV/conf/OpC/sg_mon.cfg
root@linux:~ # vi /var/opt/OV/conf/OpC/sg_mon.cfg
# PKG[0]=XYZ Package name 1
# PKG_NODE[0]=ABC Primary node on which the package must run
# Define as * will disable running on adoptive node check
# PKG_SWTCH[0]=1 Set to 1 if Package_switching should be ENABLED
# Set to 0 if Package_switching must not be ENABLED

In our case, it was disabled the node check for the package

root@linux:~ # vi /var/opt/OV/conf/OpC/sg_mon.cfg
PKG[0]=infraHP0; PKG_NODE[0]=*; PKG_SWTCH[0]=0

Received disconnect from host: 2: Too many authentication failures for username

root@linux:~ # ssh emerson@10.32.36.45
Received disconnect from 10.32.36.45: 2: Too many authentication failures for emerson

The problem was that my server 10.32.36.45 tried to authenticate using a method and then gave up.

To fix it, edit file /etc/ssh/sshd_config and increase MaxAuthTries. On my server was just trying 1 time.

root@linux:~ # vi /etc/ssh/sshd_config
#MaxAuthTries 1

You can try to not use ssh keys to login but I commented the parameter to restore the default value and restarted the sshd service

root@linux:~ # service sshd restart
Stopping sshd: [ OK ]
Starting sshd: [ OK ]

systool – Query information about Fibre Channel HBA card on a Linux server

To query information about Fibre Channel HBA cards, use systool

root@linux:~ # which systool
/usr/bin/systool

Install the package sysfsutils using yum install systool if you have a Red Hat based distribution

root@linux:~ # rpm -qf /usr/bin/systool
sysfsutils-2.1.0-7.el6.x86_64

root@linux:~ # rpm -qi sysfsutils
Name : sysfsutils Relocations: (not relocatable)
Version : 2.1.0 Vendor: Red Hat, Inc.
Release : 7.el6 Build Date: Thu 20 Sep 2012 10:11:43 AM BRT
Install Date: Fri 23 May 2014 12:17:31 PM BRT Build Host: x86-012.build.bos.redhat.com
Group : Development/Tools Source RPM: sysfsutils-2.1.0-7.el6.src.rpm
Size : 117100 License: GPLv2
Signature : RSA/8, Fri 09 Nov 2012 09:27:58 AM BRST, Key ID 199e2f91fd431d51
Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla&gt;
URL : http://sourceforge.net/projects/linux-diag/
Summary : Utilities for interfacing with sysfs
Description :
This package’s purpose is to provide a set of utilities for interfacing
with sysfs.

root@linux:~ # cat /etc/*release
LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Red Hat Enterprise Linux Server release 6.5 (Santiago)
Red Hat Enterprise Linux Server release 6.5 (Santiago)

resize2fs: Filesystem has unsupported feature(s) while trying to open

Tried to use resize2fs on a logical volume after expanding it and gave the error that couldn’t find a valid filesystem superblock and filesystem has unsupported features.

root@linux:~ # resize2fs /dev/mapper/vg00-tmpvol
resize2fs 1.39 (29-May-2006)
resize2fs: Filesystem has unsupported feature(s) while trying to open /dev/mapper/vg00-tmpvol
Couldn’t find valid filesystem superblock.

Check if the filesystem is ext4

root@linux:~ # mount | grep tmpvol
/dev/mapper/vg00-tmpvol on /tmp type ext4 (rw)

Use ext4 version of the tools: tune4fs, resize4fs

Linux – command ls or df hangs on /

If you run ls or df -h, these commands will appear that hung

Check if you have NFS shares mounted

root@linux:~ # mount | grep nfs
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
mdmMPC:/export/sapmnt/MPC on /sapmnt/MPC type nfs (rw,nfsvers=3,proto=tcp,noac,soft,sloppy,addr=10.106.10.118)
linuxnfs25:/oracle/HP0/sapdata1/DUMP on /dump type nfs (rw,addr=142.40.81.32)
nfshp0:/export/sapmnt/HP0/exe on /sapmnt/HP0/exe type nfs (rw,nfsvers=3,proto=udp,noac,soft,sloppy,addr=10.106.10.28)
nfshp0:/export/sapmnt/HP0/profile on /sapmnt/HP0/profile type nfs (rw,nfsvers=3,proto=udp,noac,soft,sloppy,addr=10.106.10.28)

I have this share that the server was turned off. So I tried to umount the share

root@linux:~ # umount /dump
umount.nfs: /dump: device is busy
umount.nfs: /dump: device is busy

And even forcing but no luck

root@linux:~ # umount -f /dump
umount2: Device or resource busy
umount.nfs: /dump: device is busy
umount2: Device or resource busy
umount.nfs: /dump: device is busy

umount with -l to do a lazy unmount

root@linux:~ # umount -l /dump
root@linux:~ # df -h /dump
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vgroot-lv_root
7.8G 818M 6.6G 11% /

Disk removed on Linux and then LVM commands giving error: read failed after 0 of 4096 at 0: Input/output error

Some LUNs were removed from the server and now LVM commands give error messages

root@linux:~# pvs
/dev/mapper/350002ac4f691374a: read failed after 0 of 4096 at 0: Input/output error
/dev/mapper/350002ac4f691374a: read failed after 0 of 4096 at 107374116864: Input/output error
/dev/mapper/350002ac4f691374a: read failed after 0 of 4096 at 107374174208: Input/output error
/dev/mapper/350002ac4f691374a: read failed after 0 of 4096 at 4096: Input/output error
/dev/vgHP0ascs/lv_ASCS: read failed after 0 of 4096 at 0: Input/output error
/dev/vgHP0ascs/lv_ASCS: read failed after 0 of 4096 at 10737352704: Input/output error
/dev/vgHP0ascs/lv_ASCS: read failed after 0 of 4096 at 10737410048: Input/output error
/dev/vgHP0ascs/lv_ASCS: read failed after 0 of 4096 at 4096: Input/output error
/dev/vgHP0ascs/lv_NOVELL_RemoteLoader: read failed after 0 of 4096 at 0: Input/output error
/dev/vgHP0ascs/lv_NOVELL_RemoteLoader: read failed after 0 of 4096 at 117374976: Input/output error
/dev/vgHP0ascs/lv_NOVELL_RemoteLoader: read failed after 0 of 4096 at 117432320: Input/output error
/dev/vgHP0ascs/lv_NOVELL_RemoteLoader: read failed after 0 of 4096 at 4096: Input/output error
/dev/vgHP0ascs/lv_sapmnt_global: read failed after 0 of 4096 at 0: Input/output error
/dev/vgHP0ascs/lv_sapmnt_global: read failed after 0 of 4096 at 10737352704: Input/output error
/dev/vgHP0ascs/lv_sapmnt_global: read failed after 0 of 4096 at 10737410048: Input/output error
/dev/vgHP0ascs/lv_sapmnt_global: read failed after 0 of 4096 at 4096: Input/output error
/dev/vgHP0ascs/lv_sapmnt_profile: read failed after 0 of 4096 at 0: Input/output error
/dev/vgHP0ascs/lv_sapmnt_profile: read failed after 0 of 4096 at 2147418112: Input/output error
/dev/vgHP0ascs/lv_sapmnt_profile: read failed after 0 of 4096 at 2147475456: Input/output error
/dev/vgHP0ascs/lv_sapmnt_profile: read failed after 0 of 4096 at 4096: Input/output error
/dev/vgHP0ascs/lv_usrsapHP0: read failed after 0 of 4096 at 0: Input/output error
/dev/vgHP0ascs/lv_usrsapHP0: read failed after 0 of 4096 at 7516127232: Input/output error
/dev/vgHP0ascs/lv_usrsapHP0: read failed after 0 of 4096 at 7516184576: Input/output error
/dev/vgHP0ascs/lv_usrsapHP0: read failed after 0 of 4096 at 4096: Input/output error
/dev/vgHP0ascs/lv_sapmnt: read failed after 0 of 4096 at 0: Input/output error
/dev/vgHP0ascs/lv_sapmnt: read failed after 0 of 4096 at 20971454464: Input/output error
/dev/vgHP0ascs/lv_sapmnt: read failed after 0 of 4096 at 20971511808: Input/output error
/dev/vgHP0ascs/lv_sapmnt: read failed after 0 of 4096 at 4096: Input/output error
/dev/vgHP0ascs/lv_sapmnt_exe: read failed after 0 of 4096 at 0: Input/output error
/dev/vgHP0ascs/lv_sapmnt_exe: read failed after 0 of 4096 at 5368643584: Input/output error
/dev/vgHP0ascs/lv_sapmnt_exe: read failed after 0 of 4096 at 5368700928: Input/output error
/dev/vgHP0ascs/lv_sapmnt_exe: read failed after 0 of 4096 at 4096: Input/output error
PV VG Fmt Attr PSize PFree
/dev/sda2 vgroot lvm2 a– 99.50g 38.74g

Check with the command dmsetup the device that needs to be removed and then remove it. I do it from bottom to top

root@linux:~# dmsetup ls | grep lv_sapmnt_exe
vgHP0ascs-lv_sapmnt_exe (253:21)
root@linux:~# dmsetup remove vgHP0ascs-lv_sapmnt_exe

root@linux:~# dmsetup ls | grep lv_sapmnt
vgHP0ascs-lv_sapmnt_profile (253:18)
vgHP0ascs-lv_sapmnt_global (253:17)
vgHP0ascs-lv_sapmnt (253:20)
root@linux:~# dmsetup remove vgHP0ascs-lv_sapmnt

root@linux:~# dmsetup ls | grep lv_usrsapHP0
vgHP0ascs-lv_usrsapHP0 (253:19)
root@linux:~# dmsetup remove vgHP0ascs-lv_usrsapHP0

root@linux:~# dmsetup ls | grep lv_sapmnt_profile
vgHP0ascs-lv_sapmnt_profile (253:18)
root@linux:~# dmsetup remove vgHP0ascs-lv_sapmnt_profile

root@linux:~# dmsetup ls | grep lv_sapmnt_global
vgHP0ascs-lv_sapmnt_global (253:17)
root@linux:~# dmsetup remove vgHP0ascs-lv_sapmnt_global

root@linux:~# dmsetup ls | grep lv_NOVELL_RemoteLoader
vgHP0ascs-lv_NOVELL_RemoteLoader (253:16)
root@linux:~# dmsetup remove vgHP0ascs-lv_NOVELL_RemoteLoader

root@linux:~# dmsetup ls | grep lv_ASCS
vgHP0ascs-lv_ASCS (253:15)
root@linux:~# dmsetup remove vgHP0ascs-lv_ASCS

root@linux:~# dmsetup ls | grep 350002ac4f691374a
350002ac4f691374a (253:14)
root@linux:~# dmsetup remove 350002ac4f691374a

Then after issuing pvs, it won’t show error messages anymore

root@linux:~# pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 vgroot lvm2 a– 99.50g 38.74g