Advertisements

Solaris – UXMON: [ID 377184 daemon.error] SUNW-MSG-ID: ZFS-8000-D3, TYPE: Fault, VER: 1, SEVERITY: Major

UXMON: [ID 377184 daemon.error] SUNW-MSG-ID: ZFS-8000-D3, TYPE: Fault, VER: 1, SEVERITY: Major

Node : solaris.setaoffice.com
Node Type : Sun SPARC (HTTPS)
Severity : major
OM Server Time: 2017-08-12 10:27:31
Message : UXMON: [ID 377184 daemon.error] SUNW-MSG-ID: ZFS-8000-D3, TYPE: Fault, VER: 1, SEVERITY: Major
Msg Group : OS
Application : SOL_mon
Object : FMT
Event Type :
not_found

Instance Name :
not_found

Instruction : “The Fault Management agent has identified a HW or OS related problem with the severity presented by the ticket.
The problem(s) can be viewed and managed with the command – fmdump
To get a better understanding of the problem and on how to resolve it, locate the event that generated
the ticket in the syslog file /var/adm/messages, a URL will be found (http://sun.com/msg/xxx-nnnn-yy),
follow the link using your Oracle portal account for instructions.”

After running fmadm faulty, we see that there is a problem with a zpool. Run zpool status -x and then we see pool prd027_software is having problems

root@solaris:~ # zpool status prd027_software
pool: prd027_software
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
Run ‘zpool status -v’ to see device specific details.
see: http://support.oracle.com/msg/ZFS-8000-8A
scan: none requested
config:

NAME STATE READ WRITE CKSUM
prd027_software ONLINE 0 0 14.7K
c0t600507680191818C1000000000000BE9d0 ONLINE 0 0 0
c0t600507680191818C1000000000000BEAd0 ONLINE 0 0 0
c0t600507680191818C1000000000000BEBd0 ONLINE 0 0 0
c0t600507680191818C1000000000000BECd0 ONLINE 0 0 0

errors: 3 data errors, use ‘-v’ for a list

Run zpool scrub prd027_software

root@solaris:~ # zpool scrub prd027_software

root@solaris:~ # zpool status -xv
pool: prd027_software
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://support.oracle.com/msg/ZFS-8000-8A
scan: scrub in progress since Wed Dec 31 21:00:00 1969
50.7M scanned out of 1.08T at 25.3M/s, 12h25m to go
0 repaired, 0.00% done
config:

NAME STATE READ WRITE CKSUM
prd027_software ONLINE 0 0 14.7K
c0t600507680191818C1000000000000BE9d0 ONLINE 0 0 0
c0t600507680191818C1000000000000BEAd0 ONLINE 0 0 0
c0t600507680191818C1000000000000BEBd0 ONLINE 0 0 0
c0t600507680191818C1000000000000BECd0 ONLINE 0 0 0

errors: Permanent errors have been detected in the following files:

/zones/prd027/root/usr/software/best1/Patrol3/Solaris-2-10-sparc-64/best1/7.4.00/bgs/monitor/log/prd027-bgsagent_6767.als
prd027_software/software027:<0x198c996>
prd027_software/software027:<0x198de4e>

After the pool is scanned, check if there is still a problem

root@solaris:~ # zpool status -xv
all pools are healthy

Repairing fmadm entries

root@solaris:~ # fmadm faulty|grep “Aug”
Aug 12 11:23:22 82fe93a5-8120-657b-9e61-e33252b84d30 ZFS-8000-D3 Major
Aug 12 11:22:01 74c61e33-7c56-4aca-d707-a32ce06a9bd8 ZFS-8000-CS Major

root@solaris:~ # fmadm repair 82fe93a5-8120-657b-9e61-e33252b84d30
fmadm: recorded repair to 82fe93a5-8120-657b-9e61-e33252b84d30

root@solaris:~ # fmadm repair 74c61e33-7c56-4aca-d707-a32ce06a9bd8
fmadm: recorded repair to 74c61e33-7c56-4aca-d707-a32ce06a9bd8

root@solaris:~ # fmadm faulty
root@solaris:~ #

Advertisements
%d bloggers like this: