Tag: lsof

Linux – umount: /filesystem: device is busy. No processes found using lsof and fuser

root@linux:~ # df -hP | grep SCR
/dev/mapper/vgSAPlocal-lv_sapmnt_SCR 15G 2.0G 12G 15% /sapmnt/SCR
scsscr:/export/sapmnt/SCR/exe 4.2G 3.5G 473M 89% /sapmnt/SCR/exe
scsscr:/export/sapmnt/SCR/global 4.2G 362M 3.6G 9% /sapmnt/SCR/global
scsscr:/export/sapmnt/SCR/profile 2.0G 3.0M 1.9G 1% /sapmnt/SCR/profile

I unmounted the filesystems under /sapmnt/SCR

root@linux:~ # umount /sapmnt/SCR/exe /sapmnt/SCR/global /sapmnt/SCR/profile
root@linux:~ #

But I was unable to unmount /sapmnt/SCR

root@linux:~ # umount /sapmnt/SCR
umount: /sapmnt/SCR: device is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))

After using fuser and lsof, it didn’t find any processes.
I was able to unmount it after restarting the autofs service

root@linux:~ # service autofs restart
Stopping automount: [ OK ]
Starting automount: [ OK ]

Advertisement

Installing lsof on Solaris 11 from OpenCSW

I’m doing an offline install of CSWlsof for Solaris 11

You need 3 packages, the package itself and the dependencies
lsof
common
isaexec

Download from one of the mirrors
https://reflector.westga.edu/repos/OpenCSW/stable/sparc/5.11/

Install the packages

common

root@solaris:/tmp/lsof_solaris # pkgadd -d common-1.5\,REV\=2010.12.11-SunOS5.8-sparc-CSW.pkg

The following packages are available:
1 CSWcommon common – common files and dirs for CSW packages
(sparc) 1.5,REV=2010.12.11

Select package(s) you wish to process (or ‘all’ to process
all packages). (default: all) [?,??,q]: 1

Processing package instance from </tmp/lsof_solaris/common-1.5,REV=2010.12.11-SunOS5.8-sparc-CSW.pkg>

common – common files and dirs for CSW packages(sparc) 1.5,REV=2010.12.11
http://www.opencsw.org/ packaged for CSW by Philip Brown
## Executing checkinstall script.
## Processing package information.
## Processing system information.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

Installing common – common files and dirs for CSW packages as

## Installing part 1 of 1.
/opt/csw/bin/sparc
/opt/csw/doc
/opt/csw/info
/opt/csw/lib/32
/opt/csw/lib/64
/opt/csw/lib/locale
/opt/csw/lib/sparc
/opt/csw/lib/sparcv8
/opt/csw/man
/opt/csw/share/locale/locale.alias
[ verifying class ]

Installation of was successful.

isaexec

root@solaris:/tmp/lsof_solaris # pkgadd -d isaexec-0.2\,REV\=2009.03.26-SunOS5.8-all-CSW.pkg

The following packages are available:
1 CSWisaexec isaexec – sneaky wrapper around Sun isaexec
(all) 0.2,REV=2009.03.26

Select package(s) you wish to process (or ‘all’ to process
all packages). (default: all) [?,??,q]: 1

Processing package instance from </tmp/lsof_solaris/isaexec-0.2,REV=2009.03.26-SunOS5.8-all-CSW.pkg>

isaexec – sneaky wrapper around Sun isaexec(all) 0.2,REV=2009.03.26
This interesting packaging hack is copyrighted by Philip Brown,
phil@bolthole.com
You may copy, install, and otherwise redistribute this package as you like,
without having to pay any fee, so long as it remain unchanged, and this notice
remain intact as the copyright notice.
## Processing package information.
## Processing system information.
2 package pathnames are already properly installed.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with super-user
permission during the process of installing this package.

Do you want to continue with the installation of [y,n,?] y

Installing isaexec – sneaky wrapper around Sun isaexec as

## Installing part 1 of 1.
[ verifying class ]
## Executing postinstall script.
Copying /usr/lib/isaexec to /opt/csw/bin/isaexec

Installation of was successful.

lsof

root@solaris:/tmp/lsof_solaris # pkgadd -d lsof-4.87\,REV\=2013.03.28-SunOS5.11-sparc-CSW.pkg

The following packages are available:
1 CSWlsof lsof – List open files
(sparc) 4.87,REV=2013.03.28

Select package(s) you wish to process (or ‘all’ to process
all packages). (default: all) [?,??,q]: 1

Processing package instance from </tmp/lsof_solaris/lsof-4.87,REV=2013.03.28-SunOS5.11-sparc-CSW.pkg>

lsof – List open files(sparc) 4.87,REV=2013.03.28
Please see /opt/csw/share/doc/lsof/license for license information.
## Processing package information.
## Processing system information.
## Verifying package dependencies.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

The following files are being installed with setuid and/or setgid
permissions:
/opt/csw/bin/sparcv8plus/lsof
/opt/csw/bin/sparcv9/lsof

Do you want to install these as setuid/setgid files [y,n,?,q] y

Installing lsof – List open files as

## Installing part 1 of 1.
/opt/csw/bin/sparcv8plus/lsof
/opt/csw/bin/sparcv9/lsof
/opt/csw/share/doc/lsof/license
/opt/csw/share/man/man8/lsof.8
[ verifying class ]
/opt/csw/bin/lsof

Installation of was successful.

Creating a link on /usr/sbin to the binary that was installed under /opt/csw

root@solaris:/usr/sbin # ln -s /opt/csw/bin/lsof lsof

Deleted file in Linux but didn’t reclaim space in filesystem

The reason that the space in the filesystem wasn’t reclaimed is because the file was opened and in use by another application

root@linux:~ # lsof /bkpcvrd | grep deleted
dsmc 21215 root 11r REG 253,9 4268860136 1097761 /bkpcvrd/pbh020/export/pbh020_20111001.dmp.gz (deleted)
dsmc 30379 root 8r REG 253,9 4268860136 1097761 /bkpcvrd/pbh020/export/pbh020_20111001.dmp.gz (deleted)
dsmc 32691 root 9r REG 253,9 4268860136 1097761 /bkpcvrd/pbh020/export/pbh020_20111001.dmp.gz (deleted)

Check the directory where the file descriptors for the PID is

root@linux:/proc/21215/fd # ls -l
total 12
l-wx—— 1 root root 64 2011-10-04 10:09 0 -> /dev/null
l-wx—— 1 root root 64 2011-10-04 10:09 1 -> /root/nohup.out
lrwx—— 1 root root 64 2011-10-04 10:09 10 -> socket:/[32923523]
lr-x—— 1 root root 64 2011-10-04 10:09 11 -> /bkpcvrd/pbh020/export/pbh020_20111001.dmp.gz (deleted)
l-wx—— 1 root root 64 2011-10-04 10:09 2 -> /dev/null
l-wx—— 1 root root 64 2011-10-04 10:09 3 -> /opt/tivoli/tsm/client/ba/bin/dsmerror.log
l-wx—— 1 root root 64 2011-10-04 10:09 4 -> /opt/tivoli/tsm/client/ba/bin/dsmsched.log
lrwx—— 1 root root 64 2011-10-04 10:09 5 -> socket:/[31319408]
lrwx—— 1 root root 64 2011-10-04 10:09 6 -> socket:/[31319409]
lrwx—— 1 root root 64 2011-10-04 10:09 7 -> socket:/[33998182]
lrwx—— 1 root root 64 2011-10-04 10:09 8 -> socket:/[33083683]
lr-x—— 1 root root 64 2011-10-04 10:09 9 -> /usr/oradata/orapbh020/tbd1/LENEL_DATA.D001

You previously saw that the there is a link and in the end it says deleted. Type > in the number that was shown in that line

root@linux:/proc/21215/fd # > 11

List the other file descriptors

root@linux:/proc/30379/fd # ls -l
total 11
lrwx—— 1 root root 64 2011-10-04 10:09 0 -> /dev/console
l-wx—— 1 root root 64 2011-10-04 10:09 1 -> /dev/null
lrwx—— 1 root root 64 2011-10-04 10:09 10 -> socket:/[33912163]
lr-x—— 1 root root 64 2011-10-04 10:09 11 -> /usr/oradata/orapbh020/tbd1/LENEL_DATA.D001
l-wx—— 1 root root 64 2011-10-04 10:09 2 -> /dev/null
l-wx—— 1 root root 64 2011-10-04 10:09 3 -> /opt/tivoli/tsm/client/ba/bin/dsmerror.log
l-wx—— 1 root root 64 2011-10-04 10:09 4 -> /opt/tivoli/tsm/client/ba/bin/dsmsched.log
lrwx—— 1 root root 64 2011-10-04 10:09 5 -> socket:/[21055400]
lrwx—— 1 root root 64 2011-10-04 10:09 6 -> socket:/[31339242]
lrwx—— 1 root root 64 2011-10-04 10:09 7 -> socket:/[33078656]
lrwx—— 1 root root 64 2011-10-04 10:09 9 -> socket:/[32744402]

root@linux:/proc/32691/fd # ls -l
total 11
l-wx—— 1 root root 64 2011-10-04 10:09 0 -> /dev/null
l-wx—— 1 root root 64 2011-10-04 10:09 1 -> /opt/tivoli/tsm/client/ba/bin/nohup.out
lr-x—— 1 root root 64 2011-10-04 10:09 10 -> /bkpcvrd/pbh020/export/pbh020_20111002.dmp.gz
l-wx—— 1 root root 64 2011-10-04 10:09 2 -> /dev/null
l-wx—— 1 root root 64 2011-10-04 10:09 3 -> /opt/tivoli/tsm/client/ba/bin/dsmerror.log
l-wx—— 1 root root 64 2011-10-04 10:09 4 -> /opt/tivoli/tsm/client/ba/bin/dsmsched.log
lrwx—— 1 root root 64 2011-10-04 10:09 5 -> socket:/[33364746]
lrwx—— 1 root root 64 2011-10-04 10:09 6 -> socket:/[33364747]
lrwx—— 1 root root 64 2011-10-04 10:09 7 -> socket:/[33365181]
lrwx—— 1 root root 64 2011-10-04 10:09 8 -> socket:/[33365142]
lr-x—— 1 root root 64 2011-10-04 10:09 9 -> /bkpcvrd/pbh020/export/pbh020_20111003.dmp.gz

The space is now reclaimed

root@linux:~ # df -h /bkpcvrd
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rootvg-bkpcvrdlv
17G 8.2G 7.3G 53% /bkpcvrd

Listing the open files in the filesystem to see if there is still an open file with the deleted status

root@linux:~ # lsof /bkpcvrd | grep deleted
root@linux:~ #