Wednesday, June 25, 2014

Map ASM disks to physical ones

Once ASM is configured and ASM disks are created several structures are created:

[root@dba-test2 ~]# ls -l /dev/oracleasm
total 0
drwxr-xr-x 1 root   root     0 Sep 29 04:28 disks
drwxrwx--- 1 oracle oinstall 0 Sep 29 03:40 iid
 
On the /dev/oracleasm/disks directory are located the ASMLib mappings of ASM devices to physical devices

[root@dba-test2 ~]# ls -l /dev/oracleasm/disks/
total 0
brw-rw---- 1 grid asmadmin 202,  33 Sep 29 04:28 ASMDISK01
brw-rw---- 1 grid asmadmin 202,  49 Sep 29 04:28 ASMDISK02
brw-rw---- 1 grid asmadmin 202,  65 Sep 29 04:28 ASMDISK03
brw-rw---- 1 grid asmadmin 202,  81 Sep 29 04:28 ASMDISK04
brw-rw---- 1 grid asmadmin 202,  97 Sep 29 04:28 ASMDISK05
brw-rw---- 1 grid asmadmin 202, 193 Sep 29 04:28 ASMDISK06

The major and minor numbers permit to map the ASM Disks to the physical devices

[root@dba-test2 ~]# ls -l /dev/* |grep 202, |egrep " 33 | 49 | 65 | 81 | 97 | 193 "
brw-r-----  1 root disk     202,  33 Sep 29 04:28 /dev/xvdc1
brw-r-----  1 root disk     202,  49 Sep 29 04:28 /dev/xvdd1
brw-r-----  1 root disk     202,  65 Sep 29 04:28 /dev/xvde1
brw-r-----  1 root disk     202,  81 Sep 29 04:28 /dev/xvdf1
brw-r-----  1 root disk     202,  97 Sep 29 04:28 /dev/xvdg1
brw-r-----  1 root disk     202, 193 Sep 29 04:28 /dev/xvdm1
 
A faster way to do it would be to use the short perl script I wrote for this :

#!/usr/bin/perl

my @asmmajmin = qx 'ls -l /dev/oracleasm/disks/';
my @alldevsmajmin = qx 'ls -l /dev/*';
my %mapasmmin = my %mapdiskmin = ();

foreach (@asmmajmin) {
        if (/^b.*\s+(\d+,)\s+(\d+).*\d+:\d+\s+(.*$)/) {
                $mapasmmin{$2} = $3;
                $maj = $1;
        }
}
my $major = $maj;

foreach (@alldevsmajmin) {
        if (/^b.*$major\s+(\d+).*\d+:\d+\s+(.*$)/) {
                $mapdiskmin{$1} = $2;
        }
}

foreach (sort (keys %mapasmmin)) {
  chomp;
  print "The asmdisk $mapasmmin{$_} maps to $mapdiskmin{$_}\n";
}
 
Another way to do this is to issue the following command provided that you are the owner of the /dev/xvd* devices : 
 
[root@dba-test2 ~]# /etc/init.d/oracleasm querydisk /dev/xvd*1
Device "/dev/xvda1" is not marked as an ASM disk
Device "/dev/xvdb1" is not marked as an ASM disk
Device "/dev/xvdc1" is marked an ASM disk with the label "ASMDISK01"
Device "/dev/xvdd1" is marked an ASM disk with the label "ASMDISK02"
Device "/dev/xvde1" is marked an ASM disk with the label "ASMDISK03"
Device "/dev/xvdf1" is marked an ASM disk with the label "ASMDISK04"
Device "/dev/xvdg1" is marked an ASM disk with the label "ASMDISK05"
Device "/dev/xvdh1" is marked an ASM disk with the label "ASMDISK08"
Device "/dev/xvdi1" is marked an ASM disk with the label "ASMDISK07"
Device "/dev/xvdj1" is not marked as an ASM disk
Device "/dev/xvdk1" is not marked as an ASM disk
Device "/dev/xvdl1" is not marked as an ASM disk
Device "/dev/xvdm1" is marked an ASM disk with the label "ASMDISK06"
 

Thursday, October 15, 2009

Am sters /etc/name_to_major ! belea mare dar nu e totul pierdut

In cazul in care ai sters sau te-ai ales cu un /etc/name_to_major corupt si daca nu ai nici backup (snaphot sau pe banda sau ... ceva/orice ) in primul rand ar trebui sa te gandesti sa-ti incerci norocul intr-un alt domeniu de activitate. Inainte insa de a schimba/parasi job-ul trebuie sa stii ca nu e totul pierdut.
FOARTE IMPORTANT !! - sa nu restartezi serverul pentru ca:

- Nu o sa mai porneasca
- Poti folosi kernelul care ruleaza deja pentru a recrea fisierul pierdut
(pe x86 gasesti informatia asta si in boot_archive)

Iata cum se poate reface fisierul:

#!/bin/ksh
i=0
while ((i < 1000 ))
do
print "0t$i::major2name" | mdb -k | read x && echo $x $i
let i=i+1
done


Redirectezi output-ul de la comanda de mai sus intr-un fisier pe care-l copiezi peste /etc/name_to_major . Schimbi ownerul si permisiunile si ar trebui sa fie ok .

chown root:sys /etc/name_to_major
chmod 644 /etc/name_to_major

Friday, June 19, 2009

Driver Configuration File to Force the Sun GigaSwift Ethernet Adapter's Parameters

For ease of administration it is advisable to put the configuration file in the same directory as the driver itself.There is a 64-bit driver located in the /platform/sun4u/kernel/drv/sparcv9 directory and a 32-bit driver in the /platform/sun4u/kernel/drv directory.

Example 1

As a first example let's look at the devices on a Sun Fire 480R. If we look at the path_to_inst file on this type of system we may see something similar to the following.

# grep '"ce"' /etc/path_to_inst
"/pci@9,700000/network@2" 0 "ce"
"/pci@9,600000/network@1" 1 "ce"

Each line in the file contains the following three fields as described in the man page for path_to_inst(4).

"physical-name" instance-number "driver-binding-name"

So, we see two instances of the device, 0 and 1, which will be seen as ce0 and ce1 respectively in the output from the ifconfig -a command if the interfaces are plumbed.

To force ce0 to run at 100 Mbit/s full duplex and ce1 to run at 1 Gbit/s full duplex we would need to place the following two lines in the ce.conf file.

name="ce" parent="/pci@9,700000" unit-address="2" adv_autoneg_cap=0 adv_1000fdx_cap=0
adv_1000hdx_cap=0 adv_100fdx_cap=1 adv_100hdx_cap=0 adv_10fdx_cap=0 adv_10hdx_cap=0;
name="ce" parent="/pci@9,600000" unit-address="1" adv_autoneg_cap=0 adv_1000fdx_cap=1
adv_1000hdx_cap=0 adv_100fdx_cap=0 adv_100hdx_cap=0 adv_10fdx_cap=0 adv_10hdx_cap=0;

The important points to note here are:

  1. The name, parent and unit-address parameters should be enclosed in double quotes. Should you wish to force all ce interfaces to the same speed and duplex you do not need to include these parameters.
    • The "name" parameter is the driver binding name for the device and will always be "ce". You may have seen previous revisions of this document which state that it should be "pci108e,abba". While this works you should use the driver binding name (ce) due to its simplicity and the fact that it does not tie the configuration to a specific type of PCI card as defined in the /etc/driver_aliases file.
    • The "parent" parameter is that part of the device path (or physical-name) prior to the "/network@" part. In a Sun Cluster environment you may see the device path prefixed with "node@<n>" where <n> is replaced by a digit. This prefix should not be included in the parent parameter.
    • The "unit-address" parameter is the number immediately after the "network@" part in the device path, it is not the device instance.
  2. We have defined more than the minimum set of parameters to effect this change since we are being deliberately verbose for the sake of completeness and to reduce the scope for error.
  3. Each entry in the file should be on a separate line and terminated with a semi-colon. While the example above may at first sight appear to be spread over several lines there are in fact only two lines, one for each instance of the device.
  4. These parameters will take effect following a system reboot, only the ndd command may be used to effect dynamic changes.

Example 2

As a second example let's look at the interfaces on a Sun Fire 4800 domain where the /etc/path_to_inst file contains the following entries.

"/ssm@0,0/pci@18,600000/pci@1/network@0" 0 "ce"
"/ssm@0,0/pci@18,600000/pci@1/network@1" 1 "ce"
"/ssm@0,0/pci@19,700000/pci@3/network@0" 2 "ce"
"/ssm@0,0/pci@1c,600000/pci@1/network@0" 3 "ce"
"/ssm@0,0/pci@1c,600000/pci@1/network@1" 4 "ce"
"/ssm@0,0/pci@1d,700000/network@3" 5 "ce"

In order to force ce5 to 1000 Mbit/s full duplex and ce0 and ce3 to 100Mbit/s full duplex the ce.conf file should contain the following three lines.

name="ce" parent="/ssm@0,0/pci@1d,700000" unit-address="3" adv_autoneg_cap=0 adv_1000fdx_cap=1
adv_1000hdx_cap=0 adv_100fdx_cap=0 adv_100hdx_cap=0 adv_10fdx_cap=0 adv_10hdx_cap=0;
name="ce" parent="/ssm@0,0/pci@18,600000/pci@1" unit-address="0" adv_autoneg_cap=0
adv_1000fdx_cap=0 adv_1000hdx_cap=0 adv_100fdx_cap=1 adv_100hdx_cap=0 adv_10fdx_cap=0 adv_10hdx_cap=0;
name="ce" parent="/ssm@0,0/pci@1c,600000/pci@1" unit-address="0" adv_autoneg_cap=0
adv_1000fdx_cap=0 adv_1000hdx_cap=0 adv_100fdx_cap=1 adv_100hdx_cap=0 adv_10fdx_cap=0 adv_10hdx_cap=0;

In order to clarify point 1 above, if we wanted to force all six interfaces to operate at 1Gbit/s full duplex we would only need to place the following single line in the file.

adv_autoneg_cap=0 adv_1000fdx_cap=1 adv_1000hdx_cap=0 adv_100fdx_cap=0 adv_100hdx_cap=0 adv_10fdx_cap=0

To check the interface settings:

netstat -k ce0

To get the NIC's capabilities:

kstat -p ce:0::"/^cap_/"

To get advertised capabilities:

ndd -set /dev/ce instance
ndd -get /dev/ce adv_autoneg_cap
ndd -get /dev/ce \? | grep "adv_"

Parameter Name Value Meaning
adv_autoneg_cap 0 The interface has been forced directly into a particular operating mode without regard to auto-negotiation

1 The interface is currently auto-negotiating
adv_1000fdx_cap 0 The interface is not capable of operating at 1000 Mbit/s full duplex

1 The interface is capable of operating at 1000 Mbit/s full duplex
adv_1000hdx_cap 0 The interface is not capable of operating at 1000 Mbit/s half duplex

1 The interface is capable of operating at 1000 Mbit/s half duplex
adv_100fdx_cap 0 The interface is not capable of operating at 100 Mbit/s full duplex

1 The interface is capable of operating at 100 Mbit/s full duplex
adv_100hdx_cap 0 The interface is not capable of operating at 100 Mbit/s half duplex

1 The interface is capable of operating at 100 Mbit/s half duplex
adv_10fdx_cap 0 The interface is not capable of operating at 10 Mbit/s full duplex

1 The interface is capable of operating at 10 Mbit/s full duplex
adv_10hdx_cap 0 The interface is not capable of operating at 10 Mbit/s half duplex

1 The interface is capable of operating at 10 Mbit/s half duplex


To get the actual link mode:

kstat -p ce:0::"/^link_/"

Parameter Name Value Meaning
link_up 0 The link is down

1 The link is up
link_speed 1000 The link is operating at 1000 Mbit/s

100 The link is operating at 100 Mbit/s

10 The link is operating at 10 Mbit/s
link_duplex 0 The link is down

1 The link is operating in half duplex mode

2 The link is operating in full duplex mode


To get the link partner capabilities:

kstat -p ce:0::"/^lp_cap_/"

Parameter Name Vlaue Meaning< /b>
lp_cap_autoneg 0 The link partner operational parameters have been forced or the link partner simply does not support auto-negotiation

1 Auto-negotiation information has been received from the link partner
lp_cap_1000fdx 0 The link partner appears to be incapable of operating at 1000 Mbit/s full duplex

1 The link partner is capable of operating at 1000 Mbit/s full duplex
lp_cap_1000hdx 0 The link partner appears to be incapable of operating at 1000 Mbit/s half duplex

1 The link partner is capable of operating at 1000 Mbit/s half duplex
lp_cap_100fdx 0 The link partner appears to be incapable of operating at 100 Mbit/s full duplex

1 The link partner is capable of operating at 100 Mbit/s full duplex
lp_cap_100hdx 0 The link partner appears to be incapable of operating at 100 Mbit/s half duplex

1 The link partner is capable of operating at 100 Mbit/s half duplex
lp_cap_10fdx 0 The link partner appears to be incapable of operating at 10 Mbit/s full duplex

1 The link partner is capable of operating at 10 Mbit/s full duplex
lp_cap_10hdx 0 The link partner appears to be incapable of operating at 10 Mbit/s half duplex

1 The link partner is capable of operating at 10 Mbit/s half duplex


Methods to determine link status of ethernet interfaces

- check /var/adm/messages (including when IPMP is used)

---------------------
checking link status with ndd

# ndd -set /dev/qfe instance 0 (instance set to "0" checks qfe0 status; "1" for qfe1,etc...)
# ndd /dev/qfe link_status
1
( 1 -up, 0 = down )

--------------------

Checking link status for Gigaswift Ethernet(ce) is different from other Ethernet interfaces.   netstat
-k is used:

netstat -k ce0

link_up (1=up,0=down)
-------------

checking with kstat (netstat -k)

# kstat qfe:0::link_up
module: qfe instance: 0
name: qfe0 class: net
link_up 1

# kstat eri |grep link_up
link_up 1

# netstat -k qfe0 |grep link_up
...... link_up 1

-------------

examining LED states

newer network interface adapters have light indicators.

on "qfe" quad fast ethernet and "ge" Sun Gigabit Ethernet 2.0 adapters

     LED           Status                 Description
--------------------------------------------------------------
LINK Off Port not connected or Link partner down
Green Steady Link detected & active
on "ce" Gigaswift:
--------------------------------------------------------------
LINK QUALITY Purple Gigabit link quality good
PHY 1000X Green Link 1Gbit/s up
PHY 100X Green Link 100Mbit/s up
PHY 10X Green Link 10Mbit/s up
-------------

checking with SNMP ( requires snmp Software)
./snmpwalk netlab11 public interfaces.ifTable

...
interfaces.ifTable.ifEntry.ifDescr.1 = lo0
interfaces.ifTable.ifEntry.ifDescr.2 = ce3
interfaces.ifTable.ifEntry.ifDescr.3 = hme0
interfaces.ifTable.ifEntry.ifDescr.4 = qfe0
...
interfaces.ifTable.ifEntry.ifOperStatus.1 = up(1)
interfaces.ifTable.ifEntry.ifOperStatus.2 = up(1)
interfaces.ifTable.ifEntry.ifOperStatus.3 = up(1)
interfaces.ifTable.ifEntry.ifOperStatus.4 = down(2)



Ethernet auto-negotiation

Use the following commands to check that the link partner (switch / link / router) has auto-negotiation capability enabled.

for ge   # ndd /dev/ge lp_1000autoneg_cap
for hme # ndd /dev/hme lp_autoneg_cap
for qfe # ndd /dev/qfe lp_autoneg_cap
for eri # ndd /dev/eri lp_autoneg_cap
for ce # kstat ce:::lp_cap_autoneg
for dmfe # ndd /dev/dmfe0 lp_autoneg_cap

If the lp_autoneg_cap is 0 (ie, auto-negotiate is disabled on the link partner), action should be taken to either enable autoneg_cap on the link partner, or disable autoneg_cap on the host.

Note: the kstat command is only available in Solaris 8 and later. For previous releases the information is included in the "netstat -k" command (see the ce relevant section there).

Wednesday, May 13, 2009

Xclock ca resursa failover in SUN cluster

Asta e raspunsul pe care l-am dat unuia pe sunmanagers pentru a seta Xclock ca resursa in SUN cluster .... si cica i-a mers :-)

On any of the two nodes:

first you need to create the resorce type
the command to do this should look smth like this:

scdscreate -k -V SUN -T XCLOCK

Then you should create the resource package
the command to do this should look smth like this:

scdsconfig -s "/path_to_application_start_script" /
-t "/path_to_application_stop_script" /
-m "/path_to_application_monitor_script"
It's a goo idea to keep the rtconfig file (mv rtconfig SUNXCLOCK/ )
Now in SUNXCLOCK/pkg you should have got
a package for your resource

Install the obtained package on both nodes !!

pkgadd -d name_of_the_package

Register the resource in the cluster (to be done on only one node)

scrgadm -a -t SUN.XCLOCK

Create the resource group and then the resorce itself in the cluster (to be done on only one node):

scrgadm -a -g resorce-group-name

scrgadm -a -j resource-name -g resource-group-name -t SUN.XCLOCK

Activate the resource

scswitch -e -j resource-name

Good luck !!

SSH pe Solaris 6

# Se instaleaza openssl, openssh, zlib,libgcc, prngd
mkdir /var/empty
chown root:sys /var/empty
chmod 755 /var/empty
groupadd sshd
useradd -g sshd -c 'sshd privsep' -d /var/empty -s /bin/false sshd
cat /var/log/* > /etc/prngd-seed
/usr/local/sbin/prngd /var/run/egd-pool
ssh-keygen -t rsa1 -f /usr/local/etc/ssh_host_key -N ""
ssh-keygen -t rsa1 -f /usr/local/etc/ssh_host_key -N ""
ssh-keygen -t dsa -f /usr/local/etc/ssh_host_dsa_key -N ""
ssh-keygen -t rsa -f /usr/local/etc/ssh_host_rsa_key -N ""
ln -s /etc/init.d/sshd /etc/rc2.d/S98sshd
ln -s /etc/init.d/prngd /etc/rc2.d/S97sshd
chmod ug+x /etc/init.d/sshd
chown root /etc/init.d/sshd
chgrp sys /etc/init.d/sshd
/etc/init.d/sshd start

# /etc/init.d/sshd contine:
#!/bin/sh

pid=`/usr/bin/ps -e | /usr/bin/grep sshd | /usr/bin/sed -e 's/^ *//' -e 's/ .*//'`
case $1 in
'start')
/usr/local/sbin/sshd
;;
'stop')
if [ "${pid}" != "" ]
then
/usr/bin/kill ${pid}
fi
;;
*)
echo "usage: /etc/init.d/sshd {start|stop}"
;;
esac

############################################

# /etc/init.d/prngd contine:
#!/bin/sh

pid=`/usr/bin/ps -e | /usr/bin/grep prngd | /usr/bin/sed -e 's/^ *//' -e 's/ .*//'`
case $1 in
'start')
/usr/local/sbin/prngd /var/spool/prngd/pool
;;
'stop')
if [ "${pid}" != "" ]
then
/usr/bin/kill ${pid}
fi
;;
*)
echo "usage: /etc/init.d/prngd {start|stop}"
;;
esac