Tag: svcadm

How to clear fmadm faulty entries in Solaris 10

Clear fmadm log

root@sc02-app04:~ # fmadm faulty
————— ———————————— ————– ———
TIME EVENT-ID MSG-ID SEVERITY
————— ———————————— ————– ———
Feb 06 10:17:04 0ad5260a-f9e0-ef7a-c91f-efcc76b9b164 ZFS-8000-HC Major

Host : sc02-app04
Platform : ORCL,SPARC-T5-8 Chassis_id :
Product_sn :

Fault class : fault.fs.zfs.io_failure_wait
Affects : zfs://pool=prd171
faulted but still in service
Problem in : zfs://pool=prd171
faulted but still in service

Description : The ZFS pool has experienced currently unrecoverable I/O
failures.

Response : No automated response will be taken.

Impact : Read and write I/Os cannot be serviced.

Action : Use ‘fmadm faulty’ to provide a more detailed view of this event.
Make sure the affected devices are connected, then run ‘zpool
clear’. Please refer to the associated reference document at
http://sun.com/msg/ZFS-8000-HC for the latest service procedures
and policies regarding this diagnosis.

————— ———————————— ————– ———
TIME EVENT-ID MSG-ID SEVERITY
————— ———————————— ————– ———
Feb 06 10:17:03 fe2302d7-99c9-c0c8-d54b-92495cc94fc9 ZFS-8000-D3 Major

Host : sc02-app04
Platform : ORCL,SPARC-T5-8 Chassis_id :
Product_sn :

Fault class : fault.fs.zfs.device
Affects : zfs://pool=prd171/vdev=5d2cdc446e947471
faulted and taken out of service
Problem in : zfs://pool=prd171/vdev=5d2cdc446e947471
faulted and taken out of service

Description : A ZFS device failed.

Response : No automated response will occur.

Impact : Fault tolerance of the pool may be compromised.

Action : Run ‘zpool status -x’ for more information. Please refer to the
associated reference document at http://sun.com/msg/ZFS-8000-D3
for the latest service procedures and policies regarding this
diagnosis.

root@sc02-app04:~ # fmadm repair 0ad5260a-f9e0-ef7a-c91f-efcc76b9b164
fmadm: recorded repair to 0ad5260a-f9e0-ef7a-c91f-efcc76b9b164
root@sc02-app04:~ # fmadm repair fe2302d7-99c9-c0c8-d54b-92495cc94fc9
fmadm: recorded repair to fe2302d7-99c9-c0c8-d54b-92495cc94fc9

Clear ereports and resource cache

root@sc02-app04:~ # cd /var/fm/fmd
root@sc02-app04:/var/fm/fmd # rm e* f* c*/eft/* r*/*

Clearing out FMA files with no reboot needed

root@sc02-app04:~ # svcadm disable -s svc:/system/fmd:default
root@sc02-app04:~ # cd /var/fm/fmd
root@sc02-app04:/var/fm/fmd # find /var/fm/fmd -type f -exec ls {} \;
/var/fm/fmd/topo/90ab82b5-08eb-6f9f-9a9a-af2975a2808b/hc-topology.xml
/var/fm/fmd/topo/6b4eba63-3576-e155-ac3b-8f6609f0b968/hc-topology.xml
/var/fm/fmd/topo/1badc01d-82b9-6203-9440-9dd440aedaca/hc-topology.xml
/var/fm/fmd/topo/f32b13d0-63a1-4b5a-e811-bfda6bddcba1/hc-topology.xml
/var/fm/fmd/topo/c4824832-ced3-672a-ec69-a9490f94d2c0/hc-topology.xml
/var/fm/fmd/ckpt/etm/etm
/var/fm/fmd/ckpt/zfs-diagnosis/zfs-diagnosis
root@sc02-app04:/var/fm/fmd # find /var/fm/fmd -type f -exec rm {} \;
root@sc02-app04:/var/fm/fmd # svcadm enable svc:/system/fmd:default

Checking fmadm faulty

root@sc02-app04:~ # fmadm faulty
root@sc02-app04:~ #

Reset the fmd serd modules

root@sc02-app04:~ # fmadm reset cpumem-diagnosis
fmadm: failed to reset module cpumem-diagnosis: specified module is not loaded in fault manager
root@sc02-app04:~ # fmadm reset cpumem-retire
fmadm: cpumem-retire module has been reset
root@sc02-app04:~ # fmadm reset eft
fmadm: eft module has been reset
root@sc02-app04:~ # fmadm reset io-retire
fmadm: io-retire module has been reset

Source: https://saifulaziz.com/2011/12/26/how-to-clear-fmadm-log-or-fma-faults-log/

UXMON: Service /application/pkg/zones-proxy-client status is maintenance, check with svcs -xv

ATTENTION, RMC LEVEL 1 AGENT: This ticket will be automatically worked by the Automation Bus. Pls. ensure your Ticket List/View includes the “Assignee” column, monitor this ticket until the user “ABOPERATOR” is no longer assigned, BEFORE you start work on this ticket.
Node : localzone.setaoffice.com
Node Type : Sun SPARC (HTTPS)
Severity : minor
OM Server Time: 2017-09-24 22:32:27
Message : UXMON: Service /application/pkg/zones-proxy-client status is maintenance, check with svcs -xv
Msg Group : OS
Application : svcsmon
Object : svcs
Event Type :
not_found

Instance Name :
not_found

Instruction : The svcsmon has detected solaris service status

Please, for details, browse the /var/opt/OV/log/OpC/svcs_mon.log
The configuration file uses to be /var/opt/OV/conf/OpC/svcs_mon.cfg

On global zone

root@globalzone:~ # pkg publisher
PUBLISHER TYPE STATUS P LOCATION
solaris origin online F file:///net/192.168.252.12/export/IPS-repos/solaris11/repo/
solaris origin online F file:///var/scmuidrs/idr2142.1.p5p/
solaris origin online F file:///var/scmuidrs/idr2160.2.p5p/
solaris origin online F file:///var/scmuidrs/idr2193.2.p5p/
solaris origin online F file:///var/scmuidrs/idr2194.1.p5p/
solaris origin online F file:///var/scmuidrs/idr2238.1.p5p/
exa-family origin online F file:///net/192.168.252.12/export/IPS-repos/exafamily/repo/
ha-cluster origin online F file:///net/192.168.252.12/export/IPS-repos/osc4/repo/

root@globalzone:~ # svcadm disable zones-proxyd system-repository; svcadm enable system-repository zones-proxyd; sleep 30
root@globalzone:~ #

On local zone

root@localzone:~# svcs -a | grep /application/pkg/zones-proxy-client
maintenance 8:08:15 svc:/application/pkg/zones-proxy-client:default

root@localzone:~# svcadm disable svc:/application/pkg/zones-proxy-client:default

root@localzone:~# svcadm enable svc:/application/pkg/zones-proxy-client:default

root@localzone:~# svcs -a | grep /application/pkg/zones-proxy-client
online 14:09:45 svc:/application/pkg/zones-proxy-client:default

Oracle SMF Oracle Configuration Manager (OCM) svc:/system/ocm:default

Listing status of ocm service

root@solaris:~ # svcs svc:/system/ocm:default
STATE STIME FMRI
disabled Jul_17 svc:/system/ocm:default

Listing SMF

root@solaris:~ # svcs -l svc:/system/ocm:default
fmri svc:/system/ocm:default
name Oracle Configuration Manager (OCM)
enabled false
state disabled
next_state none
state_time Mon Jul 17 04:48:06 2017
logfile /var/svc/log/system-ocm:default.log
restarter svc:/system/svc/restarter:default
contract_id
manifest /etc/svc/profile/generic.xml
manifest /lib/svc/manifest/system/ocm.xml
dependency require_all/none svc:/milestone/multi-user-server:default (online)
dependency require_all/error svc:/milestone/network:default (online)
dependency require_all/none svc:/system/cryptosvc (online)

Oracle Configuration Manager is used to collect client configuration information and upload it to the Oracle repository
When enabling OCM, it goes to maintenance mode

root@solaris:~ # svcadm enable svc:/system/ocm:default

root@solaris:~ # svcs -v svc:/system/ocm:default
STATE NSTATE STIME CTID FMRI
maintenance – 10:58:25 1081435 svc:/system/ocm:default

There are two errors in this installation. There is no proxy setup and it is missing user ocm

root@solaris:~ # cat /var/svc/log/system-ocm:default.log
[ Aug 28 10:58:09 Enabled. ]
[ Aug 28 10:58:09 Executing start method (“/lib/svc/method/svc-ocm start”). ]
/lib/svc/method/svc-ocm: starting…
OCM not registered
Collector running in connected mode
Begin anonymous registration…
Starting response file generation…
Can not create response file: Unknown Host: ccr.oracle.com: unknown error
Failed to create response file…
Failed to generate anonymous response file…
Unable to contact ccr.oracle.com. Please set your system proxy
in order to allow this system to contact Oracle for better
serviceability. See the configCCR(1M) manual page on home to set
the proxy server for Oracle Configuration Manager.

svc:/system/ocm:default has been temporarily disabled.

[ Aug 28 10:58:20 Method “start” exited with status 0. ]
[ Aug 28 10:58:20 Stopping because service disabled. ]
[ Aug 28 10:58:20 Executing stop method (“/lib/svc/method/svc-ocm stop”). ]
Stopping scheduler…
su: Unknown id: ocm
[ Aug 28 10:58:25 Method “stop” exited with status 95. ]

UXMON: Service /system/webconsole:console (java web console) status is maintenance.

Node : solaris10.setaoffice.com
Node Type : Sun SPARC (HTTPS)
Severity : minor
OM Server Time: 2015-05-11 19:47:56
Message : UXMON: Service /system/webconsole:console (java web console) status is maintenance.
Msg Group : OS
Application : svcsmon
Object : svcs
Event Type :
not_found

Instance Name :
not_found

Instruction : The svcsmon has detected solaris service status

Please, for details, browse the /var/opt/OV/log/OpC/svcs_mon.log
The configuration file uses to be /var/opt/OV/conf/OpC/svcs_mon.cfg

Verify the service status

root@solaris10:~ # svcs -xv svc:/system/webconsole:console
svc:/system/webconsole:console (java web console)
State: maintenance since Tue May 12 07:24:23 2015
Reason: Restarting too quickly.
See: http://sun.com/msg/SMF-8000-L5
See: man -M /usr/share/man -s 1M smcwebserver
See: /var/svc/log/system-webconsole:console.log
Impact: This service is not running.

After disabling the service, it is not completely disabled

root@solaris10:~ # svcs svc:/system/webconsole:console
STATE STIME FMRI
offline* 10:26:43 svc:/system/webconsole:console

Run this command as shown below and then run svcadm clear svc:/system/webconsole:console

root@solaris10:~ # /usr/share/webconsole/private/bin/wcremove -i console

Stopping console process…

Disabling console service…

Removing console instance configuration files…

root@solaris10:~ # svcadm clear svc:/system/webconsole:console

It shouldn’t be a problem anymore

root@solaris10:~ # svcs svc:/system/webconsole:console
STATE STIME FMRI
disabled 10:27:35 svc:/system/webconsole:console

If you still have problems, check the process of the svc-webconsole script and then kill it. First try to kill with kill and if it not works, use kill -9

root@solaris10:~ # ps -ef | grep web
root 15623 9 0 10:27:06 ? 0:00 /sbin/sh /lib/svc/method/svc-webconsole start
root 15646 15623 0 10:27:07 ? 0:00 /bin/sh /usr/share/webconsole/private/bin/sjwcx start -i console -u noaccess —
root 15649 15646 0 10:27:07 ? 0:03 /usr/java/bin/java -cp /usr/share/webconsole/private/lib/console_config.jar:/us

root@solaris10:~ # kill -9 15623

Check again. It should be disabled

root@solaris10:~ # svcs svc:/system/webconsole:console
STATE STIME FMRI
disabled 10:27:52 svc:/system/webconsole:console