Problems with a NFS share coming from a Linux server and mounted on Linux client

Running df or ls in a NFS share it simply hangs.

See if df or ls hangs. Run strace df -h or strace ls -l and see if it finishes

root@linux:~ # strace df -h

root@linux:~ # strace ls -l

statfs(“/mnt/NFS”,

Try to umount with -f or -l

-f, –force
Force an unmount (in case of an unreachable NFS system). (Requires kernel 2.1.116 or later.)

Note that this option does not guarantee that umount command does not hang. It’s strongly recommended to use absolute paths without symlinks to avoid unwanted readlink and stat syscalls on unreachable NFS in umount.

-l, –lazy
Lazy unmount. Detach the filesystem from the file hierarchy now, and clean up all references to this filesystem as soon as it is not busy anymore. (Requires kernel 2.4.11 or later.)

NetIQ Remote Loader: JCO_ERROR_LOGON_FAILURE: Initialization of repository destination SAP-GRC_PRIMCONN failed: Password logon no longer possible – too many failed attempts

DirXML: [07/11/16 08:15:03.685]: TRACE: Remote Loader: Sending…
DirXML: [07/11/16 08:15:03.686]: TRACE:

Identity Manager Driver for User Management of SAP Software Novell, Inc.

 

com.novell.nds.dirxml.driver.sapumshim.BapiException: Error authenticating to SAP host: com.sap.conn.jco.JCoException: (103) JCO_ERROR_LOGON_FAILURE: Initialization of repository destination SAP-GRC_PRIMCONN failed: Password logon no longer possible – too many failed attempts on ascsND0.setaoffice.com sysnr 01

Asked SAP team to reset the user IDM_CPIC

NetIQ Remote Loader – Connect to SAP gateway failed

root@linux:/usr/sap/PD0/NOVELL_RemoteLoader # cat trace.log
DirXML: [06/02/16 06:26:00.946]: TRACE: Remote Loader: SubscriptionShim.execute() returned:
DirXML: [06/02/16 06:26:00.947]: TRACE:

Identity Manager Driver for User Management of SAP Software Novell, Inc.

 

com.novell.nds.dirxml.driver.sapumshim.BapiException: Error connecting to SAP host: com.sap.conn.jco.JCoException: (102) JCO_ERROR_COMMUNICATION: Initialization of repository destination SAP-PI_PRIMCONN failed: Connect to SAP gateway failed
Connection parameters: TYPE=A DEST=SAP-PI_PRIMCONN ASHOST=ascsPD0.setaoffice.com SYSNR=27 PCS=1

LOCATION CPIC (TCP/IP) on local host with Unicode
ERROR partner ‘ascsPD0.setaoffice.com:3327’ not reached
TIME Thu Jun 2 06:26:00 2016
RELEASE 721
COMPONENT NI (network interface)
VERSION 40
RC -10
MODULE nixxi.cpp
LINE 3283
DETAIL NiPConnect2: 10.106.136.41:3327
SYSTEM CALL connect
ERRNO 111
ERRNO TEXT Connection refused
COUNTER 96

Provided information provided by SAP team to the user to login to the system.

SAP System ID: PD0
Logon Group:
SAP System id: 06
SAP User client number: 300
Logical System Name: PD0CLNT300
Authentication ID:
Authentication context: ascsPD0.setaoffice.com
Remote Loader connection parameters: hostname=ascsPD0.setaoffice.com port=13041 kmo=sap_dev

Using yum behind a proxy on CentOS 6.3 and showing message Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again

I’m behind a proxy and having problems using the EPEL repository on CentOS 6.3.

After installing it, it shows the message Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again

root@linux:~ # yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again

From what I read, yum is not able to access the repository using HTTPS. So replace https for http

root@linux:~ # mv /etc/yum.repos.d/epel.repo /etc/yum.repos.d/epel.repo.bak
root@linux:~ # sed ‘s/https/http/g’ /etc/yum.repos.d/epel.repo.bak > /etc/yum.repos.d/epel.repo

Bug 486324 – (yum with proxy) and (repo with https uri) / ssl request doesn’t work throught proxy

UXMON: Volume UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 should be mounted on /srv. Please check your

I’m receiving this ticket

ATTENTION, RMC LEVEL 1 AGENT: This ticket will be automatically worked by the Automation Bus. Pls. ensure your Ticket List/View includes the “Assignee” column, monitor this ticket until the user “ABOPERATOR” is no longer assigned, BEFORE you start work on this ticket.
Node : linux.setaoffice.com
Node Type : Intel/AMD x64(HTTPS)
Severity : warning
OM Server Time: 2016-06-30 17:06:04
Message : UXMON: Volume UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 should be mounted on /srv. Please check your vfstab fstab or filesystems file. Please also check: UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8
Msg Group : OS
Application : volmon
Object : LVM
Event Type : NONE
Instance Name : NONE
Instruction : No

Checking the UXMONbroker I see that it shows that UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 is not mounted.

This server is a SUSE Linux 11 SP3

root@linux:~ # cat /etc/*release
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 3
Build no: 1565
Build date: Fri Aug 14 07:53:12 CEST 2015
Kiwi version: 7.02.58
LSB_VERSION=”core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64″

root@linux:~ # /var/opt/OV/bin/instrumentation/UXMONbroker -check volmon
Mon Jul 4 11:52:53 2016 : INFO : UXMONvolmon is running now, pid=33366
connect() failed on local socket: No such file or directory
Internal cluster locking initialisation failed.
WARNING: Falling back to local file-based locking.
Volume Groups with the clustered attribute will be inaccessible.
Finding all volume groups
Finding volume group “vg_log1_dp_14”
Finding volume group “vg_data1_dp_11”
Finding volume group “vg_log1_14”
Finding volume group “vg_log1_dp_12”
Finding volume group “vg_log1_dp_11”
Finding volume group “vg_data1_dp_13”
Finding volume group “vg_data1_dp_14”
Finding volume group “vg_data1_dp_12”
Finding volume group “vg_data1_11”
Finding volume group “vg_log1_dp_13”
Finding volume group “vg_data1_14”
Finding volume group “vg_log1_11”
Finding volume group “vg_data1_12”
Finding volume group “vg_log1_12”
Finding volume group “vg_log1_13”
Finding volume group “vg_data1_13”
Mon Jul 4 11:52:58 2016 : VOLMON: CMA(NONE,NONE) Volume UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 should be mounted on /srv .Please also check: UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8 UUID=c7c47b25-30d8-42bc-8ca8-13f939b5c7b8
mv: `/dev/null’ and `/dev/null’ are the same file
Mon Jul 4 11:52:58 2016 : INFO : UXMONvolmon end, pid=33366

The filesystem is mounted

root@linux:~ # df -h /srv
Filesystem Size Used Avail Use% Mounted on
/dev/dm-16 509G 14G 494G 3% /srv

This server is using btrfs

root@linux:~ # blkid /dev/dm-16
/dev/dm-16: UUID=”c7c47b25-30d8-42bc-8ca8-13f939b5c7b8″ UUID_SUB=”ebe2d68d-0b4f-4586-bd40-6476a824f170″ TYPE=”btrfs”

Suppressing EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full! error in HPOM

Node : linux.setaoffice.com
Node Type : Intel/AMD x64(HTTPS)
Severity : minor
OM Server Time: 2016-06-18 04:38:03
Message : EXT4-fs: warning (device dm-134): ext4_dx_add_entry: Directory index full!
Msg Group : OS
Application : dmsg_mon
Object : EXT4
Event Type :
not_found

Instance Name :
not_found

Instruction : No

Check if the file /var/opt/OV/conf/OpC/dmsg_mon.cfg
This is the error message showing on dmesg

root@linux:~ # dmesg
EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!

I’m going to suppress in HPOM. First check if the file exists and if it doesn’t make a copy from instrumentation directory

root@linux:~ # ls -l /var/opt/OV/conf/OpC/dmsg_mon.cfg
ls: cannot access /var/opt/OV/conf/OpC/dmsg_mon.cfg: No such file or directory

root@linux:~ # cp /var/opt/OV/bin/instrumentation/dmsg_mon.cfg /var/opt/OV/conf/OpC/

Edit the file and add the string complained by dmsg_mon

root@linux:~ # vi /var/opt/OV/conf/OpC/dmsg_mon.cfg
###############################################################################
#@(#) $Id: dmsg_mon.cfg 2132 2014-08-22 06:47:32Z zhaofeif $
#@(#) $Rev: 2132 $
#@(#) $Author: zhaofeif $
#@(#) $Date: 2014-08-22 14:47:32 +0800 (Fri, 22 Aug 2014) $
#@(#) $LastChangedBy: zhaofeif $
###############################################################################

###############################################################################
#
# File: dmsg_mon.
# [disable = yes|no]
# [interval = ]
# disable
#===============
# If set disable to YES (or yes), this module won’t run anytime
#
# interval
#===============
# If the module will allow to run after the interval minutes

# Description: strings listed here don’t generate an ITO message for dmesg
# Syntax: just list the strings, one line for each
# !!! all dmesg lines matching one of the listed strings
# are taken out of monitoring !!!
#
# Example:
#
# hardware path
#
# If the string “hardware path” is listed, all dmesg lines matching (containing)
# the string “hardware path” are ignored for monitoring purposes.
# Still, the dmesg history contains these lines, but no message is generated.
#
###############################################################################

###############################################################################
# End of dmesg_mon.cfg
###############################################################################

EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!

Run UXMONbroker. The module related to the dmsg_mon.cfg template is dmesg

root@linux:~ # /var/opt/OV/bin/instrumentation/UXMONbroker -h

GD UXMON monitoring package
Broker utility

usage: /var/opt/OV/bin/instrumentation/UXMONbroker [-h | –help] [-x ] [ -d ] [-l ] [-c ] [-f]
[ –col -p param ]
[ -t ] [ -p ] [-b ]
[ –os ]
[ -v ]

-h : this (help) message
–help : this (help) message

-x : triggers the execution of the module passed as parameter
-d : Allows execution activating debug
-check : Same as -x but output is redirected standard output, no logfile used
-l : output the logfile used by the module passed as parameter
-c : output the preferred config file used by the module

-t : output the TEMPORAL folder to be used if needed
-b : output the folder where the commands or instrumentation are located
-perl : output the perl runtime to be used

–col : Execute the collecting information of module
-p : Parameter passed to the recollection
–os : Show the OS name

-v : Version of UXMON package
-f : force the execution of the module bypass interval setting
supported modules are:
actmon, sshdmon, uxmon, nfsmon, selfcheck, swapmon, evm, mpmon, mdmon, cronmon, bondmon, rcmon, volmon, scmon, loopmon, dmesg, advfsmon, ntpmon, hwmon, bootmon, nicmon, perfmon, psmon, lpmon, vcmon, ktsmon, sgmon, dfmon.
This is the interface to the OVO templates. Templates will call this command
to get executed the different modules available, or retrieve configuration
information about the UXMON and the platform

Run UXMONbroker with dmesg

root@linux:~ # /var/opt/OV/bin/instrumentation/UXMONbroker -d dmesg
>>Debug mode activated
>>Opened the logfile: /var/opt/OV/log/OpC/dmsg_mon.log
>>logfile: /var/opt/OV/log/OpC/dmsg_mon.log
>>history file: /var/opt/OV/log/OpC/dmsg_mon.hist[1-5]
>>config line: EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!
>>Module interval setting: 0
>>process info: 108389 /opt/OV/nonOV/perl/a/bin/perl -I/var/opt/OV/bin/instrumentation /var/opt/OV/bin/instrumentation/UXMONdmsg -d -c /var/opt/OV/conf/OpC/dmsg_mon.cfg -l /var/opt/OV/log/OpC/dmsg_mon.log
>>configuration file: /var/opt/OV/conf/OpC/dmsg_mon.cfg
>>lock file: /var/opt/OV/tmp/OpC/UXMONdmsg_dmsg_mon.lock
>>get lock to read /var/opt/OV/tmp/OpC/UXMONdmsg_dmsg_mon.lock
>>run the process now
>>Logged this info…:Wed Jun 22 14:50:56 2016 : INFO : UXMONdmsg is running now, pid=108389
>>There are the following lines in actual dmesg buffer program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!
EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!
EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!

>>This is the old dmesg buffer program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!
EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
program bma is using a deprecated SCSI ioctl, please convert it to SG_IO
EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!
EXT4-fs warning (device dm-134): ext4_dx_add_entry: Directory index full!

>>Difference between old and new dmesg buffers
>>The following is assigned to dmesg for processing
>>ReleaseLock: 108389 1466617856 0 stopped 108389 /opt/OV/nonOV/perl/a/bin/perl -I/var/opt/OV/bin/instrumentation /var/opt/OV/bin/instrumentation/UXMONdmsg -d -c /var/opt/OV/conf/OpC/dmsg_mon.cfg -l /var/opt/OV/log/OpC/dmsg_mon.log
>>Logged this info…:Wed Jun 22 14:50:56 2016 : INFO : UXMONdmsg end, pid=108389

HPE Fibreutils

Fibreutils is a set of utility scripts for Linux that make certain operations, such as information gathering, easier. The following sections describe commands contained in the Fibreutils
package.

adapter_info

The adapter_info command lists information about the Fibre Channel adapters.

–I Lists all FC adapters in this system.
–L Lists all LUNs for each adapter.
–t Lists all targets seen by each adapter.
-m Lists each adapter’s model.
–v Verbose
–h Prints the help menu.
–d Prints LUN and verbose information for a specific device. The specific device should be a SCSI host number such as 0 or 6.

lssd

The lssd command lists all bound /dev/sd* devices.

–c Prints cached data instead of scanning.
–h Prints help message.
–l Prints inquiry page 0 x 83 UUID of devices.
–w Prints WWNN of devices.

lssg

The lssg command lists all bound /dev/sg* devices.

–c Prints cached data instead of scanning.
–h Prints help message.
–l Prints inquiry page 0 x 83 UUID of devices.
–w Print WWNN of devices.

hp_rescan

The hp_rescan command rescans LUNs on Hewlett Packard Enterprise supported Fibre Channel
adapters.

–a Rescans all adapters.
–h Prints help message
–i Rescans a specific adapter instance. The specific device should be a SCSI host number such as 0 or 6.
–l Lists all FC adapters.
–n Do not perform scsi remove-single-device when executing probe-luns.

Locating adapter_info

root@linux:~ # which adapter_info
/usr/bin/adapter_info

It is provided by the package fibreutils

root@linux:~ # rpm -qf /usr/bin/adapter_info
fibreutils-3.2-6

root@linux:~ # rpm -qi fibreutils-3.2-6
Name : fibreutils Relocations: (not relocatable)
Version : 3.2 Vendor: Hewlett-Packard Company
Release : 6 Build Date: Tue 18 Dec 2012 02:38:23 AM BRST
Install Date: Wed 26 Mar 2014 10:42:41 AM BRT Build Host: localhost.localdomain
Group : Applications/System Source RPM: fibreutils-3.2-6.src.rpm
Size : 77657 License: Proprietary
Signature : RSA/SHA1, Tue 18 Feb 2014 04:44:32 PM BRT, Key ID b070680a5ce2d476
Packager : Hewlett-Packard Company
URL : http://www.hp.com
Summary : Provides complimentary programs and scripts for HP supported fibre channel HBAs
Description :
This RPM has the following components:

* Miscellaneous scripts and programs to compliment HP supported FC drivers:

lssd
lssg
adapter_info
hp_rescan
scsi_info

Help message for adapter_info

root@linux:~ # /usr/bin/adapter_info -h
NAME

adapter_info

DESCRIPTION

Prints information about Fibre Channel HBAs/CNAs.

OPTIONS

-d, –device – Prints all information for a specific SCSI host adapter
-h, –help – Prints this help message
-i, –versioninfo – Prints driver version information for all HBAs
-l, –luns – Prints the device information for all HBAs
-m, –model – Prints the HBA model for all HBAs
-p, –pciids – Prints the PCI IDs for all HBAs
-r, –remoteports – Prints the attached remote ports for all HBAs
-v, –verbose – Prints all information except device and LUN information

Checking HBA card WWNN WWPN and state

root@linux:~ # /usr/bin/adapter_info
/sys/class/scsi_host/host3: wwnn=0x2000001b32001275 wwpn=0x2100001b32001275 state=Link Up – F_Port
/sys/class/scsi_host/host4: wwnn=0x2000001b3200c270 wwpn=0x2100001b3200c270 state=Link Up – F_Port
/sys/class/scsi_host/host1: wwnn=0x20000000c97e3572 wwpn=0x10000000c97e3572 state=Online
/sys/class/scsi_host/host2: wwnn=0x20000000c97e3573 wwpn=0x10000000c97e3573 state=Linkdown

Source: https://h20565.www2.hpe.com/hpsc/doc/public/display?sp4ts.oid=461311&docId=emr_na-c01487184&docLocale=en_US

Red Hat Linux 5: qla2xxx 0000:05:00.0: scsi(3:4:0): Abort command issued — 1 40e 2002

The server had two single port HBA cards that were receiving a lot of reset commands

root@linux:~ # lspci | grep -i fibre
05:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
08:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
0b:00.0 Fibre Channel: Emulex Corporation Zephyr-X LightPulse Fibre Channel Host Adapter (rev 02)
0b:00.1 Fibre Channel: Emulex Corporation Zephyr-X LightPulse Fibre Channel Host Adapter (rev 02)

Jun 12 04:11:47 linnux kernel: qla2xxx 0000:08:00.0: Performing ISP error recovery – ha= ffff8102abac04f8.
Jun 12 04:11:47 linnux kernel: qla2xxx 0000:05:00.0: scsi(3:4:0): Abort command issued — 1 40e 2002.
Jun 12 04:11:47 linnux kernel: qla2xxx 0000:05:00.0: scsi(3:4:0): LOOP RESET ISSUED.
Jun 12 04:11:48 linnux kernel: qla2xxx 0000:05:00.0: qla2xxx_eh_bus_reset: reset succeeded
Jun 12 04:11:48 linnux kernel: qla2xxx 0000:08:00.0: LIP reset occured (f700).
Jun 12 04:11:48 linnux kernel: qla2xxx 0000:08:00.0: LIP occured (f700).
Jun 12 04:11:48 linnux kernel: qla2xxx 0000:08:00.0: LIP reset occured (f7f7).
Jun 12 04:11:48 linnux kernel: qla2xxx 0000:08:00.0: LOOP UP detected (4 Gbps).
Jun 12 04:11:50 linnux kernel: qla2xxx 0000:08:00.0: qla2xxx_eh_host_reset: reset succeeded
Jun 12 04:11:54 linnux kernel: lpfc 0000:0b:00.1: 1:(0):0713 SCSI layer issued Device Reset (11, 0) return x2002
Jun 12 04:12:01 linnux kernel: scsi 1:0:5:0: scsi: Device offlined – not ready after error recovery
Jun 12 04:12:01 linnux kernel: scsi 1:0:5:0: timing out command, waited 22s

We had opened a ticket with Red Hat and they said to check the hardware.

The BUR team was asked to check the tape library and storage team was asked to check the SAN switch. No problems were found.

So we logged a ticket with HP to replace the system board. Two different HBA cards in different PCI slots were having problems.

Replaced system board and riser card.

Jun 16 12:02:49 linnux kernel: lpfc 0000:0b:00.0: 0:(0):0713 SCSI layer issued Device Reset (3, 0) return x2002
Jun 16 12:02:59 linnux kernel: lpfc 0000:0b:00.0: 0:(0):0714 SCSI layer issued Bus Reset Data: x2002
Jun 16 12:03:20 linnux kernel: lpfc 0000:0b:00.0: 0:3172 SCSI layer issued Host Reset Data: x2002
Jun 16 12:03:20 linnux kernel: lpfc 0000:0b:00.0: 0:1303 Link Up Event x1 received Data: x1 xf7 x10 x9 x0 x0 0
Jun 16 12:03:40 linnux kernel: scsi 1:0:3:0: scsi: Device offlined – not ready after error recovery
Jun 16 12:03:40 linnux kernel: scsi 1:0:3:0: timing out command, waited 22s
Jun 16 12:03:55 linnux kernel: lpfc 0000:0b:00.0: 0:(0):0713 SCSI layer issued Device Reset (4, 0) return x2002

But the problem still persisted.

Some parameters in the SAN switch were changed and the problem was solved.

cmrunpkg: Unable to start some package or package instances.

Checking Serviceguard cluster packages

root@linux01:~ # cmviewcl

CLUSTER STATUS
cluster_virtual_scc_004 up

NODE STATUS STATE
linux01 up running
linux02 up running

PACKAGE STATUS STATE AUTO_RUN NODE
infraWP0 up running enabled linux02
ascsWP0 up running enabled linux02
wdpWP0 up running enabled linux02
infraSCP up running enabled linux02
scsSCP up running enabled linux02

UNOWNED_PACKAGES

PACKAGE STATUS STATE AUTO_RUN NODE
wdpSCP down failed enabled unowned

When trying to start the cluster package wdpSCP on server linux02 it fails because it says the node is not eligible

root@linux01:~ # cmrunpkg -n linux02 wdpSCP
Checking for license………
Found Valid Advanced License
Number of Advanced licenses:1
Unable to run package wdpSCP on node linux02. Node is not eligible.
cmrunpkg: Unable to start some package or package instances.

Checking detailed statistics about the cluster package wdpSCP says that it is disabled to run on node linux02

root@linux01:~ # cmviewcl -v -p wdpSCP

UNOWNED_PACKAGES

PACKAGE STATUS STATE AUTO_RUN NODE
wdpSCP down failed enabled unowned

Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual

Script_Parameters:
ITEM STATUS NODE_NAME NAME

Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled linux01
Alternate up disabled linux02

Dependency_Parameters:
DEPENDENCY_NAME NODE_NAME SATISFIED
infraSCP_dep linux01 no
infraSCP_dep linux02 yes

Other_Attributes:
ATTRIBUTE_NAME ATTRIBUTE_VALUE
Style modular
Priority no_priority

Enable the cluster package to run on node linux02

root@linux01:~ # cmmodpkg -e -v -n linux02 wdpSCP
Checking for license………
Found Valid Advanced License
Number of Advanced licenses:1
Enabling node linux02 for switching of package wdpSCP
Successfully enabled package wdpSCP to run on node linux02
cmmodpkg: Completed successfully on all packages specified

HP-UX HPOM The module VOLMON has detected an inconsistence between the number of LV and the number of current LV. Please make some UX expert verify this inconsistence due there is a risk of data corruption

Node : hpux.setaoffice.com
Node Type : Itanium 64/32(HTTPS)
Severity : major
OM Server Time: 2015-12-01 23:50:35
Message : UXMON: The number of Open LV and Current LV is different for VG: /dev/vgEP0_bc.
Msg Group : OS
Application : volmon
Object : LV
Event Type : NONE
Instance Name : NONE
Instruction : The module VOLMON has detected an inconsistence between the number of LV and the number of current LV. Please make some UX expert verify this inconsistence due there is a risk of data corruption

Checking the volume group

root@hp-ux:~ # vgdisplay vgEP0_bc
— Volume groups —
VG Name /dev/vgEP0_bc
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 27
Open LV 26
Max PV 40
Cur PV 15
Act PV 11
Max PE per PV 50000
VGDA 22
PE Size (Mbytes) 16
Total PE 91253
Alloc PE 91253
Free PE 0
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0
VG Version 1.0
VG Max Size 31250g
VG Max Extents 2000000

Checking all logical volumes from the volume group vgEP0_bc

root@hp-ux:~ # vgdisplay -v vgEP0_bc | grep “LV Name” | awk ‘{print “lvdisplay -v “$3” | grep -v disk”‘}
lvdisplay -v /dev/vgEP0_bc/lv11202 | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvDVEBMGS00 | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvNOVELL_RemoteLoader | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvSCS01 | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvSYS | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvconfig | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvdesenvolvimentos | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvdrlocalfs | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvinterf | grep -v disk -> Problemas
lvdisplay -v /dev/vgEP0_bc/lvinterface | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvol26 | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvora10264 | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvora11204 | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvoraarch | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvoracle | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvoracli | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvput | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvreorg | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvsapmnt | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvsapmntWDP | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvstage | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvtrans | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvtransARCH | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvusrsapWDP | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvusrsapdaa | grep -v disk
lvdisplay -v /dev/vgEP0_bc/lvusrsaptmp | grep -v disk

Checking the logical volume with -v to display more verbose I checked that PE1 column is showing question marks. The disk needs to be replaced and this logical volume recreated

root@hp-ux:~ # lvdisplay -v /dev/vgEP0_bc/lvinterf | grep -v disk | more
— Logical volumes —
LV Name /dev/vgEP0_bc/lvinterf
VG Name /dev/vgEP0_bc
LV Permission read/write
LV Status available/syncd
Mirror copies 0
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 614400
Current LE 38400
Allocated PE 38400
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation strict
IO Timeout (Seconds) default

— Distribution of logical volume —
PV Name LE on PV PE on PV

— Logical extents —
LE PV1 PE1 Status 1
08750 ??? 15952 current
08751 ??? 15953 current
08752 ??? 15954 current
08753 ??? 15955 current
08754 ??? 15956 current
08755 ??? 15957 current
08756 ??? 15958 current

Follow

Get every new post delivered to your Inbox.

Join 3,011 other followers

%d bloggers like this: