While applying this procedure and modifying text file, the server remote access crashed so I would recommend that you do this on the local console!

First be sure you got /sbin/ifenslave installed if not install it with “yum install iputils”. Next you will obviously need a acces to the server, it can be remote (if you intend to reboot)  but have a direct access ready in case you are adventurous.

Configuration start by adding the next line into /etc/modprobe.d/bond0.conf  (in previous version it was /etc/modprobe.conf) its necessary to define bond0 as the logical or virtual interface name.

/etc/modprobe.d/bond0.conf :

alias netdev-bond0 bonding
alias bond0 bonding

Note: Using the bond0 issues a deprecated warning as of the latest 2.6.32.xx kernel – so use the recommended “netdev-bond0″ instead. Red_Hat_Enterprise_Linux/6/html/Deployment_Guide is wrong on that matter.

cd /etc/sysconfig/network-scripts/
vi ifcfg-bond0

/etc/sysconfig/network-scripts/ifcfg-bond0 :

DEVICE="bond0"
ONBOOT=yes
BOOTPROTO=dhcp
NAME="bond0"
BONDING_OPTS="mode=4 miimon=100"

• Note 1: if eth0 was dhcp configured, you may want to use the same Ethernet MAC address for bond0. You will then keep the same IP.
• Note 2: BONDING_OPTS mode=0 (balance-rr) means Round-robin while. Mode=0  transmits packets in sequential order from the first slave through the last. This mode provides load balancing (almost twice the bandwidth) and fault tolerance in any case of cable or card failure. This mode require switch configuration as opposed to  mode=6 (balance-alb is active load balancing) which does not. There are 7 existing mode. See Redhat documentation.
• mode=4 is  IEEE 802.1ax Link Aggregation Control Protocol (LACP) which will set the same MAC address to all bounded interfaces.
• Note 3: miimon=100 means monitor ARP every 1/10 of a second or 100/1000 milliseconds.

/etc/sysconfig/network-scripts/ifcfg-eth0 :

DEVICE="eth0"
NM_CONTROLLED="no"
ONBOOT=yes
HWADDR=AA:F3:FC:DA:74:AA
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
NAME="System eth0"
UUID=5fb06bd0-0bb0-7ffb-45f1-00000000000000
SLAVE=yes
MASTER=bond0

/etc/sysconfig/network-scripts/ifcfg-eth1 :

DEVICE="eth1"
HWADDR="BB:F3:FC:DA:83:CC"
NM_CONTROLLED="no"
ONBOOT="yes"
BOOTPROTO=none
SLAVE=yes
MASTER=bond0

You should now be able to reboot and see if it does work. Beware not to loose access to the server if you logged in remotely.

• After a reboot if using DHCP and the hostname is localhost.localdomain you must have an error and DHCP is not working.
• After a reboot if not using DHCP and the hostname localhost.localdomain, it may be because you did not add your actual hostname and IP address into /etc/hosts file. Simply add this line to /etc/hosts like this :

192.168.100.1  myserver  myserver.mydomain.com

Atfer reboot you may see messages entering command “dmesg” or from the file /var/log/messages

Here is the output of ifconfig :

% ifconfig -a
bond0     Link encap:Ethernet  HWaddr AA:F3:FC:DA:74:BB
          inet addr:192.168.108.30  Bcast:192.168.108.255  Mask:255.255.255.0
          inet6 addr: fe80::5ef3:fcff:feda:74a4/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:12169 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15745 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2794755 (2.6 MiB)  TX bytes:2601601 (2.4 MiB)

eth0      Link encap:Ethernet  HWaddr AA:F3:FC:DA:74:BB
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:9778 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10352 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2075894 (1.9 MiB)  TX bytes:1684361 (1.6 MiB)
          Interrupt:28 Memory:92000000-92012800

eth1      Link encap:Ethernet  HWaddr BB:F3:FC:DA:83:CC
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:2559 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5709 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:750032 (732.4 KiB)  TX bytes:957292 (934.8 KiB)
          Interrupt:100 Memory:c0000000-c0012800

• Note: I have seen issue where eth1 had a different address. I change NM_CONTROLLED=”no”
• Note: You may also setup bonding manually. It won’t stick after reboot. it very simple and may be useful to explorer different modes

ifconfig bond0 up
ifenslave bond0 eth0 eth1
ifconfig bond0 192.168.100.1 netmask 255.255.255.0 up
route add default gw 192.168.1.1 bond0

According to RHEL doc you can explore modes, mode 0, 1, 2, 3, 4, 5, 6, by modifying files into /sys/class/net/ and  modify values. However I was not able to get expected result. Given exemple was :

echo balance-alb > /sys/class/net/bond0/bonding/mode

Validate the aggregation is working by checking the status. Consider the switch not to be configured when you got no partner key or Mac.

% cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
        Aggregator ID: 1
        Number of ports: 2
        Actor Key: 17
        Partner Key: 43
        Partner Mac Address: 00:d0:03:56:74:00

Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: AA:f3:fc:da:67:BB
Aggregator ID: 1
Slave queue ID: 0

Slave Interface: eth2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: AA:f3:fc:da:85:CC
Aggregator ID: 1
Slave queue ID: 0

hope this help someone, anyone, even me for next config
Rejean.

Other variables that can also influence this.:

On the Windows PC side in a command prompt (cmd)
set TERM pcansi

On the Unix server side, depending on you configuration of software called termcap on you os.
setenv TERMINFO /usr/share/lib/terminfo
setenv TERMINFO /usr/local/share/terminfo

% zpool status mypool
  pool: mypool
 state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
        Sufficient replicas exist for the pool to continue functioning in a
        degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
        repaired.
 scrub: scrub in progress for 0h3m, 0.01% done, 447h11m to go
config:

        NAME        STATE     READ WRITE CKSUM
        mypool      DEGRADED     0     0     0
          raidz1-0  DEGRADED     0     0     0
            c0t2d0  ONLINE       0     0     0
            c0t3d0  ONLINE       0     0     0
            c0t4d0  ONLINE       0     0     0
            c0t5d0  ONLINE       0     0     0
            c0t6d0  ONLINE       0     0     0
            c0t7d0  FAULTED      0     0     0  too many errors

errors: No known data errors

Physically replace hard disk. If you got an hot swap controller (I think most Sata3 and SAS are) just try it no special commands required, if you still can’t see the disk afterward just reboot. Go ahead, try it and replace the disk.

After this issue a zpool replace command should be issued…  however the system may not see your device right away. In some cases it work in other a reboot is required.

% zpool replace mypool c0t7d0 c0t7d0

Notes: Yes it’s the same device name twice (old and new will have the same address location).

% zpool status mypool
  pool: mypool
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
 scrub: resilver in progress for 0h0m, 0.00% done, 545h13m to go
config:

        NAME              STATE     READ WRITE CKSUM
        mypool            DEGRADED     0     0     0
          raidz1-0        DEGRADED     0     0     0
            c0t2d0        ONLINE       0     0     0
            c0t3d0        ONLINE       0     0     0
            c0t4d0        ONLINE       0     0     0
            c0t5d0        ONLINE       0     0     0
            c0t6d0        ONLINE       0     0     0
            replacing-5   DEGRADED     0     0     0
              c0t7d0s0/o  FAULTED      0     0     0  too many errors
              c0t7d0      ONLINE       0     0     0  5.24M resilvered

After completion old disk will be removed. Errors will be cleared. Pool will be reset to optimal performances. Here is a status.

% zpool status
  pool: mypool
 state: ONLINE
 scrub: resilver completed after 50h43m with 0 errors on Sat Oct 15 18:49:54 2011
config:

        NAME        STATE     READ WRITE CKSUM
        mypool      ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            c0t2d0  ONLINE       0     0     0
            c0t3d0  ONLINE       0     0     0
            c0t4d0  ONLINE       0     0     0
            c0t5d0  ONLINE       0     0     0
            c0t6d0  ONLINE       0     0     0
            c0t7d0  ONLINE       0     0     0  645G resilvered

errors: No known data errors

A very good and detailled guide is with sysinternals.

A lot of people are upset at Oracle with all the bad they are doing to Sun’s legacy. I would love to complain about Oracle to them but they won’t even listen! I now thing they just don’t care. What are they doing to Solaris support? I can’t do much except blog about it because none of the local Oracle staff care or can do much.

Oracle discontinue Sunsolve, I hate that you simply killed it; I hate your support policies; I hate you for that!

Ok ok the fact is that I cannot get any king of security patches without a contract. This is a harsh (since even Microsoft give them for free, not talking about all Linux distributions) but I can kind of understand it, they are the worst capitalist in the world and they are trying to make more money… after all they are not there to contribute to a better world, as Sun did in the past but to make money.

On top of my head Sun gaved the world OpenSolaris, NFS, NIS, ZFS, OpenOffice, mySQL and several protocol… While Oracle is not even giving nothing and they are even taking back by killing what was given like OpenSolaris…

I used Solaris for 2 decade. I had the complete Solaris Admin guides suites thrown to recycling because it was all online at docs.Sun.com (ok Solaris 5.7 was also a bit obsolete). The only reason I am keeping with Solaris is because of ZFS (the newest and best file system of all). ZFS Snapshot capable allowing all users to see they snapshot of all they files without any administrator interactions.

Know, what’s pissed me the most is having a problem with ZFS and not been able to read what the error is. It actually directs me to a Sun web site for detailed explanation about the error but it’s no longer accessible without an Oracle support contract.

--------------- ------------------------------------ -------------- ---------
TIME EVENT-ID MSG-ID SEVERITY
--------------- ------------------------------------ -------------- ---------
Jan 29 08:37:29 0f45250a-9966-4bdf-a0a6-85007a46d576 ZFS-8000-HC Major

Host : server01
Platform : PowerEdge-R510 Chassis_id : xxxdellxxxservicexxxtagxxx
Product_sn :

Fault class : fault.fs.zfs.io_failure_wait
Affects : zfs://pool=mypool
faulted but still in service
Problem in : zfs://pool=mypool
faulted but still in service

Description : The ZFS pool has experienced currently unrecoverable I/O
failures. Refer to http://sun.com/msg/ZFS-8000-HC for more
information.

Response : No automated response will be taken.

Impact : Read and write I/Os cannot be serviced.

Action : Make sure the affected devices are connected, then run
‘zpool clear’.

Well… fallowing the link gets me to sign-in and the a 503 service unavailable, while sing-in in with somebody else’s who got an account display the full detail of the error. The technical detail are irrelevant but let says that the actual detail explain on the page was useless. while zpool status -x reported that all pool where ok and zpool clear mypool did not clear much I am still waiting for a reboot to hopefully fix this

Why am I so cheap, and Why don’t I buy a contract one could say? Well I have looked into this as well. First support is for a University and Canadian University are poor AND THEN…

“Oracle Solaris Premier Subscription for Non-Oracle Hardware (1-4 socket server)” is 1000$ per socket, so it’s in my case 2 socket 2000$ per year… then 4000$ or 6000$ for a 2 or 3 year contract. This is in fact more than the price of the actual server running Solaris… I do not need software support I need patches or security patches access. And tell me why the hell should I pay more if I got 2 CPU? Are they giving me more support because I got 2 CPU? Why don’t you charge for the memory amount while you’re at it!

I manage several servers… Linux, Solaris sparc and intel and Windows… and you know what? I guess I will go and migrate those Solaris into Linux and hope that BTRFS (a clone to ZFS) is out very soon… but I could use LVM and ext4 (as already discussed here is quite good too) but then it means that I don’t have snapshot and going back to a full backup painful procedure or to rsync between servers.

I don’t expect BTRFS soon… why would it be? Development of BTRFS was originally started by Oracle itself and now that Oracle owns ZFS with the buying of Sun there is really no rush on completing BTRFS… unless the community does complete it.

In the end Oracle is killing me softly but surely. The End of Oracle Solaris is close for me. The only possible future for academic research is Linux.

RIP SUN.

Even I need to re-read my posts whenever I compile Samba… it never seem to work as it should… and here I am writing a new post about it!

Those are my configure options:


./configure --prefix=/usr/local/opt/samba-3.5.6 \
--with-privatedir=/usr/local/opt/samba-3.5.6/etc \
--with-configdir==/usr/local/opt/samba-3.5.6/etc \
--datadir=/usr/local/opt/samba-3.5.6/doc \
--with-statedir=/usr/local/opt/samba-3.5.6/var/locks \
--with-cachedir=/usr/local/opt/samba-3.5.6/var/locks \
--with-automount \
--enable-socket-wrapper \
--with-quotas \
--with-sys-quotas \
--with-acl-support \
--without-pam \
--with-aio-support \
--enable-shared \
--enable-cups \
--enable-swat \
--with-automount \
--without-ldap \
--enable-socket-wrapper \
--with-quotas \
--enable-nss-wrapper \
--without-pam

And this is the result of a gnu make (note that sun “make” does not work use gnu make) :

Compiling smbd/statcache.c
Compiling smbd/seal.c
Compiling smbd/posix_acls.c
Compiling lib/sysacls.c
Compiling smbd/process.c
smbd/vfs.c: In function ‘smb_vfs_call_mkdir’:
smbd/vfs.c:1210:46: error: macro “mkdir” passed 3 arguments, but takes just 2
smbd/vfs.c:1210:2: warning: return makes integer from pointer without a cast
Compiling smbd/service.c
Compiling smbd/error.c
The following command failed:
gcc -I. -I/mypool/install/solx/samba-3.5.6/source3 -I/mypool/install/solx/samba-3.5.6/source3/iniparser/src -Iinclude -I./include -I. -I. -I./../lib/replace -I./../lib/tevent -I./libaddns -I./librpc -I./.. -I./../lib/talloc -I../lib/tdb/include -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DSUNOS5 -I/mypool/install/solx/samba-3.5.6/source3/lib -I.. -I../source4 -D_SAMBA_BUILD_=3 -D_SAMBA_BUILD_=3 -fPIC -c smbd/vfs.c -o smbd/vfs.o
Compiling printing/printfsp.c
Compiling lib/sysquotas.c
Compiling lib/sysquotas_linux.c
Compiling lib/sysquotas_xfs.c
Compiling lib/sysquotas_4A.c
Compiling smbd/change_trust_pw.c
Compiling smbd/fake_file.c
Compiling smbd/quotas.c
Compiling smbd/ntquotas.c
Compiling lib/afs.c
Compiling smbd/msdfs.c
gmake: *** [smbd/vfs.o] Error 1
gmake: *** Waiting for unfinished jobs….
Compiling smbd/aio.c
Compiling lib/afs_settoken.c
Compiling smbd/statvfs.c

NOTE: Since we used -j17 to accelerate the compilation so much on our 16 processor system, we will have to do a simple gmake (without -j) to see where and what was the actual error… below in bold.

% gmake
Using CFLAGS     = -I. -I/mypool/install/samba-3.5.6/source3 -I/mypool/install/solx/samba-3.5.6/source3/iniparser/src -Iinclude -I./include  -I. -I. -I./../lib/replace -I./../lib/tevent -I./libaddns -I./librpc -I./.. -I./../lib/talloc -I../lib/tdb/include -DHAVE_CONFIG_H  -D_LARGEFILE_SOURCE -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DSUNOS5  -I/mypool/install/solx/samba-3.5.6/source3/lib -I.. -I../source4 -D_SAMBA_BUILD_=3 -D_SAMBA_BUILD_=3
PICFLAG    = -fPIC
LIBS       = -lsendfile -lresolv -lrt -lnsl -lsocket -liconv -laio
LDFLAGS    = -pie  -Wl,-z,ignore -lthread -L./bin
DYNEXP     =
LDSHFLAGS  = -fPIC -shared  -Wl,-z,ignore -lthread -L./bin -lc -Wl,-z,defs
SHLIBEXT   = so
SONAMEFLAG = -Wl,-h,
Linking bin/net

Undefined first referenced
symbol in file libintl_gettext utils/net.o (symbol belongs to implicit dependency /usr/local/lib/gcc/i386-pc-solaris2.10/3.4.6/../../../libintl.so.8)
libintl_textdomain  utils/net.o (symbol belongs to implicit dependency /usr/local/lib/gcc/i386-pc-solaris2.10/3.4.6/../../../libintl.so.8)
libintl_bindtextdomain utils/net.o (symbol belongs to implicit dependency /usr/local/lib/gcc/i386-pc-solaris2.10/3.4.6/../../../libintl.so.8)
ld: fatal: Symbol referencing errors. No output written to bin/net

collect2: ld returned 1 exit status
gmake: *** [bin/net] Error 1

Q: But why is it complaining about libintl ?
A: Edit the Makefile line 25 and add -lintl on LIBS=

LIBS=-lsendfile -lresolv -lrt -lnsl -lsocket -liconv -laio -lintl

After the configure is done… AND THIS IS THE TRICK… I had to go and modify the created Makefile and here is what I did :

1. edit Makefile
2. search for LIBS= add -lintl at the end
3. search for PERL= to see if correct path is set
4. search for SHELL and replace /bin/sh by /bin/bash
5. other key previously discussed in the other post is no longer necessary.

Next, here is another error… again in bold below…

% gmake
Using CFLAGS = -I. -I/mypool/install/solx/samba-3.5.6/source3 -I/mypool/install/solx/samba-3.5.6/source3/iniparser/src -Iinclude -I./include -I. -I. -I./../lib/replace -I./../lib/tevent -I./libaddns -I./librpc -I./.. -I./../lib/talloc -I../lib/tdb/include -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DSUNOS5 -I/mypool/install/solx/samba-3.5.6/source3/lib -I.. -I../source4 -D_SAMBA_BUILD_=3 -D_SAMBA_BUILD_=3
PICFLAG = -fPIC
LIBS = -lsendfile -lresolv -lrt -lnsl -lsocket -liconv -laio -lintl
LDFLAGS = -pie -Wl,-z,ignore -lthread -L./bin
DYNEXP =
LDSHFLAGS = -fPIC -shared -Wl,-z,ignore -lthread -L./bin -lc -Wl,-z,defs
SHLIBEXT = so
SONAMEFLAG = -Wl,-h,
Linking non-shared library bin/libtalloc.a
Linking shared library bin/libtalloc.so.2
Linking non-shared library bin/libtdb.a
Linking shared library bin/libtdb.so.1
Linking shared library bin/libwbclient.so.0
Linking non-shared library bin/libaddns.a
Linking non-shared library bin/libnetapi.a
Linking shared library bin/libnetapi.so.0
Linking non-shared library bin/libsmbclient.a
Linking shared library bin/libsmbclient.so.0
Linking non-shared library bin/libsmbsharemodes.a
Linking shared library bin/libsmbsharemodes.so.0
Compiling smbd/vfs.c
smbd/vfs.c: In function ‘smb_vfs_call_mkdir’:
smbd/vfs.c:1210:46: error: macro “mkdir” passed 3 arguments, but takes just 2
smbd/vfs.c:1210:2: warning: return makes integer from pointer without a cast

The following command failed:
gcc -I. -I/mypool/install/solx/samba-3.5.6/source3 -I/mypool/install/solx/samba-3.5.6/source3/iniparser/src -Iinclude -I./include -I. -I. -I./../lib/replace -I./../lib/tevent -I./libaddns -I./librpc -I./.. -I./../lib/talloc -I../lib/tdb/include -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DSUNOS5 -I/mypool/install/solx/samba-3.5.6/source3/lib -I.. -I../source4 -D_SAMBA_BUILD_=3 -D_SAMBA_BUILD_=3 -fPIC -c smbd/vfs.c -o smbd/vfs.o
gmake: *** [smbd/vfs.o] Error 1

This is weird… The way I fixed it is not clear… I simply change the configure option, I removed –without-ldap and added –with-ldap which is the default anyways… in earlier release I had issue with ldap libs and didn’t used it.

If this post does not fix your error, check the previous post mentioned earlier. For me it’s another version of Samba working on Solaris 10.

Well hope this help. post some comments.
Rejean

First be sure you got /sbin/ifenslave installed if not you may not be using RHEL 5.4 or 5.5. Then you will need to add the next line into /etc/modprobe.conf its necessary to define bond0 as the logical or virtual interface name.

/etc/modprobe.conf :

alias bond0 bonding

Note: balance-rr means Round-robin. It transmits packets in sequential order from the first slave through the last. This mode provides load balancing (almost twice the bandwidth) and fault tolerance in any case of cable or card failure. This mode require switch configuration as opposed to balance-alb which does not. There are 7 existing mode. See this link.

/etc/sysconfig/network-scripts/ifcfg-bond0 :

DEVICE=bond0
IPADDR=192.168.100.1
NETMASK=255.255.255.0
NETWORK=192.168.100.0
BROADCAST=192.168.100.255
ONBOOT=yes
BOOTPROTO=static
USERCTL=no
BONDING_OPTS="mode=0 miimon=100"

/etc/sysconfig/network-scripts/ifcfg-eth0 :

DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

/etc/sysconfig/network-scripts/ifcfg-eth1 :

DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

if the command hostname return localhost.localdomain, and you are not using dhcp, it may be because you did not add your actual hostname and IP address into /etc/hosts file. Simply add this line to /etc/hosts

192.168.100.1  myserver  myserver.mydomain.com

Atfer reboot you may see messages like those entering command: dmesg or /var/log/messages

Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008)
bonding: Warning: either miimon or arp_interval and arp_ip_target module parameters must be specified, otherwise bonding will not detect link failures! see bonding.txt for details.
bonding: bond0: setting mode to balance-rr (0).
bonding: bond0: Setting MII monitoring interval to 100.
ADDRCONF(NETDEV_UP): bond0: link is not ready
bonding: bond0: Adding slave eth0.
bnx2: eth0: using MSIX
bonding: bond0: enslaving eth0 as an active interface with a down link.
bonding: bond0: Adding slave eth1.
bnx2: eth1: using MSIX
bonding: bond0: enslaving eth1 as an active interface with a down link.
bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex
bonding: bond0: link status definitely up for interface eth0.
ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex
bonding: bond0: link status definitely up for interface eth1.

Here is the output of ifconfig :

% ifconfig -a
bond0     Link encap:Ethernet  HWaddr A4:BA:DB:FF:FA:KE
          inet addr:192.168.100.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a6ba:dbff:fe10:301e/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:572665 errors:0 dropped:0 overruns:0 frame:0
          TX packets:145041 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:320433098 (305.5 MiB)  TX bytes:11531560 (10.9 MiB)

eth0      Link encap:Ethernet  HWaddr A4:BA:DB:FF:FA:KE
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:286461 errors:0 dropped:0 overruns:0 frame:0
          TX packets:87853 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:160503391 (153.0 MiB)  TX bytes:6747616 (6.4 MiB)
          Interrupt:106 Memory:da000000-da012800 

eth1      Link encap:Ethernet  HWaddr A4:BA:DB:FF:FA:KE
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:286204 errors:0 dropped:0 overruns:0 frame:0
          TX packets:57188 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:159929707 (152.5 MiB)  TX bytes:4783944 (4.5 MiB)
          Interrupt:114 Memory:dc000000-dc012800

Note: You can also setup bonding manually but it won’t stick after reboot. it very simple and may be useful to test with modes : mode 0 (balance-rr) , mode 1, 2, 3, 4, 5 … and mode 6 (balance-alb).

modprobe bonding mode=balance-rr miimon=100
ifconfig bond0 192.168.100.1 netmask 255.255.255.0 up
ifenslave bond0 eth0 eth1
route add default gw 192.168.1.1 bond0

And while at it if you configure static ip you will have to add a default gateway in /etc/sysconfig/network add this line: GATEWAY=192.168.1.1

hope this help.
Rejean

1. First, I would like to aggregate two network interfaces so it will become one. More bandwidth will be available and a fail-over will be in place.
2. Second, I want to add virtual interfaces so I would have IP address dedicated to some services (examples: several web servers, a NFS server, a Samba server, DNS, Sendmail, …).

Aggregation

To aggregate my two on board interfaces bnx0 and bnx1 which speed are at 1 gigabyte each and will become aggregated to be as one big 2 gigabyte interfaces having only one IP.

Show all interface on the system with dladm because. This can’t be done with ifconfig -a because the second interface bnx1 is not yet connected. It will need to be plumb before we see it.

Note-1: that plumb come from plumber who plumb some pipes, it needs to be plumbed before water can go in the pipe. So here interface needs to be plumb before network bit can go in the interface.

Note-2: Often people ask to know: How do we see the speed and duplex of Solaris network interfaces. This is how.

% dladm show-dev

bnx0            link: up        speed: 1000  Mbps       duplex: full
bnx1            link: up        speed: 0     Mbps       duplex: unknown
bnxe0           link: up        speed: 10000 Mbps       duplex: full

% dladm show-link

bnx0            type: non-vlan  mtu: 1500       device: bnx0
bnx1            type: non-vlan  mtu: 1500       device: bnx1
bnxe0           type: non-vlan  mtu: 1500       device: bnxe0

We can now see that bnx1 is available on the system, let’s plumb it in so we can play with it.

% ifconfig bnx1 plumb

% ifconfig -a

lo0: flags=2001000849 mtu 8232 index 1
       inet 127.0.0.1 netmask ff000000
bnx0: flags=1000843 mtu 1500 index 2
       inet 192.168.108.15 netmask ffffff00 broadcast 132.207.108.255
       ether 84:2b:2b:6:c1:99
bnx1: flags=1000802 mtu 1500 index 4
        inet 0.0.0.0 netmask 0
        ether 84:2b:2b:6:c1:9a
bnxe0: flags=1000843 mtu 1500 index 3
        inet 192.168.108.2 netmask ffffff00 broadcast 192.168.108.255
        ether 0:10:18:30:fc:fa

And then unplumb both interfaces so we can setup aggregation on them. Interface need to be free to be reconfigured for aggregation.

Note: Be aware that when doing this all network connections to bnx0 interfaces will be lost!!! If you are using ssh remotely you are down. So you will need to be on the console or in my case I got a direct connection to the server thru bnxe0. This interface is a 10 gigabytes interface directly connected from one server to a second identical server. Both servers will be replicated on the other via this dedicated high speed 10 gig link using cross-over cable category CAT-6A.

% ifconfig bnx0 unplumb
% ifconfig bnx1 unplumb

This next line will create the link aggregation and the next one will set it up with the IP address we want.

% dladm create-aggr -d bnx0 -d bnx1 1
% ifconfig -a

lo0: flags=2001000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
bnxe0: flags=1000843 mtu 1500 index 3
        inet 192.168.108.2 netmask ffffff00 broadcast 192.168.108.255
        ether 0:10:18:30:fc:fa

% ifconfig  aggr1  plumb  netmask 255.255.255.0  192.168.108.15 up
% ifconfig -a

lo0: flags=2001000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
bnxe0: flags=1000843 mtu 1500 index 3
        inet 192.168.108.2 netmask ffffff00 broadcast 192.168.108.255
        ether 0:10:18:30:fc:fa
aggr1: flags=1000843 mtu 1500 index 6
        inet 192.168.108.15 netmask ffff0000 broadcast 132.207.255.255
        ether 84:2b:2b:6:c1:99

Voila, that’s it. One big interface offering more bandwidth and a fail-over is in place. Any of the two cables can be unplug and re-plug at will or almost twice the bandwidth is now available.

% dladm show-aggr

key: 1 (0x0001) policy: L4      address: 84:2b:2b:6:c1:99 (auto)
           device       address                 speed           duplex  link    state
           bnx0         84:2b:2b:6:c1:99          1000  Mbps    full    up      attached
           bnx1         84:2b:2b:6:c1:9a          1000  Mbps    full    up      attached

If you want you can check the aggregation statistics for each interfaces.
% dladm show-aggr -s

key:1           ipackets   rbytes       opackets   obytes       %ipkts  %opkts
        Total   4038       524867       847        324837
        bnx0    2310       345467       765        317395         57.2    90.3
        bnx1    1728       179400       82         7442           42.8     9.7

Question: Are those modifications permanent to the system? No! Not yet.

If you reboot your system will not like it and you won’t have running network access. To make it permanent you will need to remove your previous interface name’s in /etc/hostname.bnx0. Assuming you want to keep the same name you had before, simply move the file to its new name.

% mv /etc/hostname.bnx0 /etc/hostname.aggr1

Virtual interfaces

You can plumb reel interfaces or you can plumb a virtual interface on top of existing interfaces, which will add a “:1″ and be called aggr1:0  or number from 0 to 255. Because there is 256 possible virtual interface on Solaris.

For real interfaces bnx0, ce0, hme0, bnxe0, …

% ifconfig bnx0:1 plumb
% ifconfig bnx0:1 192.168.1.80 netmask 255.255.255.0 up

and for aggregated interfaces on a single command:

% ifconfig aggr1:1 plumb netmask 255.255.255.0  192.168.1.80  up

**updated**
Question: Is this permanent? 
No. You will also need to create the hostname file in /etc/hostname.aggr1 with the ip address for the aggregation, and remove /etc/hostname.*** (from the previous interface).

To delete aggregation, you can remove one network interface called bnx0

dladm remove-aggr -d bnx0 1

to completely remove aggregation from the system:

dladm delete-aggr  1

To stop using aggregation you will also need to rename /etc/hostaname.aggr1 to /etc/hostaname.bnx0 and any other virtual interface you defined (hostaname.aggr1:1 hostaname.aggr1:2 …).

Hope this help.
Rejean

Many people still are inevitably and blindly attracted to open source [libraries, packages and tools] available from a single mouse click. For students working on thesis, for companies working on internal tools, this may look fine. It does not depend only on licenses but on your final intention. Licenses are complicated. Please visit the wikipedia GNU_General_Public_License page to briefly understand the deep complexity of this domain. Intention is simple: do you go commercial with your project or not? For a student, this might mean to file a patent in; for a company, this might be to make the internal tool suddenly available to client. Not always clear at first sight.

For all of you who plan to go out commercial, look at products available under multi-license terms. That is, offering different terms and conditions for different packages. For instance, one may freely use a software library for technological exploration, and pay only when the exploration switches from R&D to production. This model is available for the µC/OS RTOS. Others like FreeRTOS offer a free basic version, but a payable fully-featured version. Dual licensing is a new trend for a sustainable open source market, producing high quality products for us to try, and buy. Recent successful models include QT/Trolltech, acquired by Nokia and MySQL, bought by Sun in Q1-2008.

Should you come from the hardware side, you may know opencores.org which offers many IP under different GPL-like licenses. Obviously, payable products usually come with support, and free products don’t, unless their designers intend to live at piece rate. Companies like ours which are committed to delivering fast results use such support to avoid losing too much time.

At the end, this dual-model is good for everyone. Those who don’t have the money: students at first, startups next and finally bigger companies working on R&D projects can use these free solutions. This leads people to make technology-based decisions rather than to focus on pricing considerations. For those who have the money, the buying decision encourages the suppliers to deliver quality solutions, fully documented and fully supported.

Luc Filion, http://www.nuumdesign.com

Special Thanks to Miro Samek, CEO of Quantum Leaps, for his views.

Using the start option the script will call itself only to start the daemon as a limited right user called flexlm, it will (switch users) with su. While the startsu will simply start it as the actual current user.

The things you might need to change in that script are:

• PATH=   # you local path
• APPS=    # the name of your software
• LIC=        # path to your license_file.dat  (eys you can name it whatever you want to)
• PORT=    # tcp port number you want
• USER=    # user other than root who should start the service as
• LOG=       # where ever you want to put your log file

So this is the start-up script I am using for many license server. I simply adjust some tweak.

#!/bin/sh
# Copyright (c) Rejean Lepage.
#
#ident  “@(#)daemon, server 3.5  2010/05/27 RL”

PATH=/usr/bin:/usr/sbin:/usr/sfw/bin:/usr/local/bin
LD_LIBRARY_PATH=/usr/lib:/usr/openwin/lib:/usr/ccs/lib:/usr/dt/lib:/usr/local/lib

APPS=ANY-SOFT
LIC=/usr/local/flexlm/dat/SOFT_licenses.dat
LOG=/tmp/${APPS}.$$.log
PORT=52800
SRV_PATH=/usr/local/flexlm/bin
SRV=”${SRV_PATH}/lmgrd -2 -p ”
USER=flexlm

#
# Start/stop $APPS server on port $PORT
#

case “$1″ in

‘start’)

if [ -f $LIC ] ; then
ulimit -n 1024
ulimit -H -n 1024

# start me with startsu option
/usr/bin/su $USER -c “$0 startsu”
else
echo “** File not found $LIC for $APPS on port $PORT **”
fi
;;

‘startsu’)

if [ "$USER" = "root" ] ; then
echo “Do not run flexlm as root.”
fi

if [ -f $LIC ] ; then
ulimit -n 1024
ulimit -H -n 1024

# Those 3 LD vars should allows to run Solaris 10 daemon on Solaris 9
LD_NOVERSION=1
LD_NOVERSION_32=1
LD_NOVERSION_64=1
export LD_NOVERSION LD_NOVERSION_32 LD_NOVERSION_64

echo “Staring server $APPS on port $PORT with process number $$ log = $LOG
umask 022
$SRV -c $LIC -local -l $LOG

else
echo “** File not found $LIC for $APPS on port $PORT **”
fi
;;

‘stop’)

echo Arret du serveur $APPS
echo y | ${SRV_PATH}/lmdown -c $LIC
;;

‘status’)

echo Etat de $APPS deamon
${SRV_PATH}/lmstat -c $LIC
;;

‘fullstatus’)

echo Etat de $APPS deamon
${SRV_PATH}/lmstat -a -c $LIC
;;

‘reread’)

echo Lecture des licenses de $APPS deamon
${SRV_PATH}/lmreread -c $LIC
;;

env)

echo PATH=$PATH
echo LD_LIBRARY_PATH=$LD_LIBRARY_PATH
;;

*)

echo “Usage: /etc/init.d/$0 { start | stop | status | reread }”
;;

esac

Hopefully this might be useful, it’s a good wrapper I do use it for all license server I got.

Finally I can point out the latest version flexlm (lmgrd, lmutil) is 11.8 and is available for download on the Flexera site.

Also here is two small script to create or remove soft links for each flexlm sub commands.

#!/bin/sh
#
# Create links to make use of the FlexLM tools easier.
#
PATH=/usr/ucb:/usr/bin:/bin; export PATH

if [ -f lmhostid ]; then
echo “The links appear to exist already!”
exit 1
fi

ln -s lmutil lmhostid
ln -s lmutil lmver
ln -s lmutil lmcksum
ln -s lmutil lmdown
ln -s lmutil lmremove
ln -s lmutil lmreread
ln -s lmutil lmswitchr
ln -s lmutil lmstat
ln -s lmutil lmdiag
ln -s lmutil lminstall
ln -s lmutil lmborrow
ln -s lmutil lmnewlog
ln -s lmutil lmpath
ln -s lmutil lmswitch

echo “Links now made”

and to remove links…

#!/bin/sh
#
# Remove links
#

PATH=/usr/ucb:/usr/bin:/bin; export PATH

if [ ! -f lmhostid ]; then
echo “The links do not exist!”
exit 1
fi

rm -f lmhostid
rm -f lmver
rm -f lmcksum
rm -f lmdown
rm -f lmremove
rm -f lmreread
rm -f lmswitchr
rm -f lmstat
rm -f lmdiag
rm -f lminstall
rm -f lmborrow
rm -f lmnewlog
rm -f lmpath
rm -f lmswitch

echo “Links removed”

Rejean.

According to the ld.so.1 manual, by disabling version checking it might be possible to not having to recompile. This is useful when you get a binary from elsewhere… a flexlm daemon file for example.

This specific issue arises when I tried to get the latest version of the Cadence cdslmd flexlm license daemon file running on Solaris 9… That daemon was provided and compiled only for Solaris 10.
A quick fix to that issue would be to set LD so it won’t check for its version this is achieved by setting those variables within the environment starting the daemon. In a sh like shell:

LD_NOVERSION=1
LD_NOVERSION_32=1
LD_NOVERSION_64=1
export LD_NOVERSION LD_NOVERSION_32 LD_NOVERSION_64

This is it. So in some cases Solaris 10 binaries should now work on Solaris 9… but of course, this should have some limitations whenever LD try to use any new function defined in libc.so.newest version it may not work.

I could write another post about securely starting flexlm… I got several nice scripts that I can provide. Just ask… you shall receive for free here.

Rejean

Special thanks to Roberto Muscedere University of Windsor for the idea.