Java Mailing List Archive

http://www.junlu.com/

Home » Home (12/2007) » JBoss User Help »

[jboss-user] [Clustering/JBoss] - Not removing failed node from
 partition

scott.stark@jboss.org

2007-07-20


Hi All,

I'm having some problems configuring jgroups failure detection. I'm using jBoss JBoss [Zion] 4.0.5.GA (build: CVSTag=Branch_4_0 date=200610162339) and jgroups 2.2.9 beta2.

My setup consists of 2 nodes, each with 8 interfaces, paired into 4 redundant groups per node.


          |        |
|    | App 1        | App 1
|    |        |
|  -----------------    -----------------
|  App 3  |    |  Mgmt  |      | App3
|  ------|   NODE 1  |---------------|   NODE 2   |-------
|  |    |    |    |
|  -----------------    -----------------
|    |        |
|    |        |
|    |     Database    |
|    ---------------------------------

I've used the -b option to bind all traffic to the management lan, and tweaked my application to bind only to the App 1 LAN. What I'm now trying to do is have the healthcheck/failure detection run explicitly over the Database LAN.

I've editted my cluster-service.xml file, which looks like this (editted for brevity):

    <Config>
|         <UDP mcast_addr="${jboss.partition.udpGroup:228.1.2.3}" mcast_port="45566"
|           ip_ttl="${jgroups.mcast.ip_ttl:8}" ip_mcast="true"
|           mcast_recv_buf_size="2000000" mcast_send_buf_size="640000"
|           ucast_recv_buf_size="2000000" ucast_send_buf_size="640000"
|           loopback="false"/>
|         <PING timeout="2000" num_initial_members="3"
|           up_thread="true" down_thread="true"/>
|         <MERGE2 min_interval="10000" max_interval="20000"/>
|         <FD_SOCK srv_sock_bind_addr="192.168.104.56" down_thread="false" up_thread="false"/>
|         <VERIFY_SUSPECT timeout="3000" num_msgs="3"
|           up_thread="true" down_thread="true"/>
|         <pbcast.NAKACK gc_lag="50" retransmit_timeout="300,600,1200,2400,4800"
|           max_xmit_size="8192"
|           up_thread="true" down_thread="true"/>
|         <UNICAST timeout="300,600,1200,2400,4800" down_thread="true"/>
|         <pbcast.STABLE desired_avg_gossip="20000" max_bytes="400000"
|           up_thread="true" down_thread="true"/>
|         <FRAG frag_size="8192"
|           down_thread="true" up_thread="true"/>
|         <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
|           shun="true" print_local_addr="true"/>
|         <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>
|       </Config>

This is the config on Node 2 - 192.168.104.56 is this nodes VIP for the database interface.

I have removed the standard FD algorithm, as it was failing to detect a failure when the database connection was broken (Based on what I read here http://www.redhat.com/docs/manuals/jboss/jboss-eap-4.2/doc/Server_Configuration_Guide/Failure_Detection_Protocols-FD.html - "Regular traffic from a node counts as if it is a live. So, the are-you-alive messages are only sent when there is no regular traffic to the node for sometime." - I assumed it must have seen other jGroups traffic across the management LAN and so counted it as alive).

The problem is that now the second node detects a failure correctly and sets itself to the master node, but getCurrentView() is still returning two nodes:

Fri Jul 20 14:34:17 BST 2007 MasterNode=true 192.168.105.158:1099 192.168.105.155:1099

(192.168.105.158 & 192.168.105.155 are the Management interfaces for Nodes 1 & 2 respecively)

What's interesting is when I plugged Node 1 back in, Node 2 breifly removed it from the partition before merging it back in:

Fri Jul 20 14:35:05 BST 2007 MasterNode=true 192.168.105.158:1099 192.168.105.155:1099
| Fri Jul 20 14:35:11 BST 2007 MasterNode=true 192.168.105.158:1099 192.168.105.155:1099
| Fri Jul 20 14:35:16 BST 2007 MasterNode=true 192.168.105.155:1099
| Fri Jul 20 14:35:22 BST 2007 MasterNode=true 192.168.105.155:1099 192.168.105.158:1099
| Fri Jul 20 14:35:29 BST 2007 MasterNode=true 192.168.105.155:1099 192.168.105.158:1099

Any advice or assistance would be greatly appreciated!

Kind Regards,

Neil Saunders.



View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4066233#4066233

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4066233
_______________________________________________
jboss-user mailing list
jboss-user@(protected)
https://lists.jboss.org/mailman/listinfo/jboss-user
©2008 junlu.com - Jax Systems, LLC, U.S.A.