Testsuite: org.apache.catalina.tribes.group.interceptors.TestTcpFailureDetector Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.006 sec ------------- Standard Output --------------- testTcpMcastFail() Channel-2:member added[Channel-1] Channel-1:member added[Channel-2] testTcpFailureMemberAdd() Channel-2:member added[Channel-1] testTcpSendFailureMemberDrop() Channel-2:member added[Channel-1] Channel-1:member added[Channel-2] Channel-1:member disappeared[Channel-2] ------------- ---------------- --------------- ------------- Standard Error ----------------- Feb 21, 2024 11:57:18 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Feb 21, 2024 11:57:18 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4012] Feb 21, 2024 11:57:18 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Feb 21, 2024 11:57:18 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1113, securePort=-1, UDP Port=-1, id={11 -22 29 -91 114 9 70 -98 -92 74 -106 -67 -81 -98 0 71 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1107, securePort=-1, UDP Port=-1, id={-97 65 -103 -96 114 -102 67 -128 -109 112 103 64 -76 54 80 111 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=1002, securePort=-1, UDP Port=-1, id={77 -122 99 -38 78 7 74 55 -109 -45 -115 96 102 -22 23 118 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=1028, securePort=-1, UDP Port=-1, id={6 -47 9 -41 -21 -40 71 107 -114 56 -28 92 -23 41 -70 40 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=1009, securePort=-1, UDP Port=-1, id={-17 -35 -96 127 -126 59 78 -84 -122 69 -52 8 120 -30 89 -22 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4003,{127, 0, 0, 1},4003, alive=1095, securePort=-1, UDP Port=-1, id={69 44 -104 5 -21 90 66 99 -96 127 48 -82 19 -21 39 26 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1007, securePort=-1, UDP Port=-1, id={85 -4 -39 -1 45 42 77 -71 -90 -77 -128 -71 -16 -67 -29 51 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=1021, securePort=-1, UDP Port=-1, id={-64 -41 -39 -1 -69 7 69 26 -109 68 -7 50 -127 27 100 -116 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=1018, securePort=-1, UDP Port=-1, id={84 55 118 3 98 -53 70 40 -81 -117 80 85 50 -24 -80 8 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=1006, securePort=-1, UDP Port=-1, id={-43 -88 27 -115 126 -117 71 -87 -107 113 65 -16 110 66 98 -65 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4011,{127, 0, 0, 1},4011, alive=1011, securePort=-1, UDP Port=-1, id={-96 -120 18 83 -17 -97 65 59 -75 -80 24 -94 14 -27 -47 82 }, payload={}, command={}, domain={-65 -26 -94 -3 -55 18 75 8 -77 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=1034, securePort=-1, UDP Port=-1, id={100 -117 -53 -43 -101 100 65 -117 -123 -6 -12 10 24 -11 85 -75 }, payload={}, command={}, domain={-65 -26 -94 -3 -55 18 75 8 -77 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Feb 21, 2024 11:57:19 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Feb 21, 2024 11:57:20 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Feb 21, 2024 11:57:20 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Feb 21, 2024 11:57:20 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4007] Feb 21, 2024 11:57:20 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Feb 21, 2024 11:57:20 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Feb 21, 2024 11:57:20 AM org.apache.catalina.tribes.io.BufferPool getBufferPool INFO: Created a buffer pool with max size:[104857600] bytes of type: [org.apache.catalina.tribes.io.BufferPool] Feb 21, 2024 11:57:20 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor messageReceived WARNING: Received message from cluster[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=2040, securePort=-1, UDP Port=-1, id={77 -122 99 -38 78 7 74 55 -109 -45 -115 96 102 -22 23 118 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused. Feb 21, 2024 11:57:20 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=2615, securePort=-1, UDP Port=-1, id={11 -22 29 -91 114 9 70 -98 -92 74 -106 -67 -81 -98 0 71 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:20 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=2542, securePort=-1, UDP Port=-1, id={77 -122 99 -38 78 7 74 55 -109 -45 -115 96 102 -22 23 118 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=2526, securePort=-1, UDP Port=-1, id={85 -4 -39 -1 45 42 77 -71 -90 -77 -128 -71 -16 -67 -29 51 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=2541, securePort=-1, UDP Port=-1, id={-64 -41 -39 -1 -69 7 69 26 -109 68 -7 50 -127 27 100 -116 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=2542, securePort=-1, UDP Port=-1, id={84 55 118 3 98 -53 70 40 -81 -117 80 85 50 -24 -80 8 }, payload={}, command={}, domain={69 82 -20 -93 -101 -58 67 90 -123 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4011,{127, 0, 0, 1},4011, alive=2515, securePort=-1, UDP Port=-1, id={-96 -120 18 83 -17 -97 65 59 -75 -80 24 -94 14 -27 -47 82 }, payload={}, command={}, domain={-65 -26 -94 -3 -55 18 75 8 -77 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=2542, securePort=-1, UDP Port=-1, id={100 -117 -53 -43 -101 100 65 -117 -123 -6 -12 10 24 -11 85 -75 }, payload={}, command={}, domain={-65 -26 -94 -3 -55 18 75 8 -77 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:21 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Feb 21, 2024 11:57:21 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=1005, securePort=-1, UDP Port=-1, id={-83 43 -88 69 114 -68 79 -12 -69 -104 -38 39 10 95 -14 81 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=1007, securePort=-1, UDP Port=-1, id={-83 43 -88 69 114 -68 79 -12 -69 -104 -38 39 10 95 -14 81 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=1001, securePort=-1, UDP Port=-1, id={35 54 -67 23 -22 59 74 -19 -125 107 12 -22 -77 -38 53 100 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1015, securePort=-1, UDP Port=-1, id={17 31 -53 81 -40 -7 76 63 -98 -96 -113 68 -76 -123 -2 76 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=1001, securePort=-1, UDP Port=-1, id={35 54 -67 23 -22 59 74 -19 -125 107 12 -22 -77 -38 53 100 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1042, securePort=-1, UDP Port=-1, id={17 31 -53 81 -40 -7 76 63 -98 -96 -113 68 -76 -123 -2 76 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1023, securePort=-1, UDP Port=-1, id={-81 30 -22 -114 111 32 74 -66 -84 18 86 -1 59 -37 21 9 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1023, securePort=-1, UDP Port=-1, id={-81 30 -22 -114 111 32 74 -66 -84 18 86 -1 59 -37 21 9 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4003,{127, 0, 0, 1},4003, alive=1013, securePort=-1, UDP Port=-1, id={32 -120 65 -123 30 -12 71 -60 -108 17 99 -4 -89 -60 -82 -39 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1010, securePort=-1, UDP Port=-1, id={-5 113 87 69 10 -3 66 126 -85 -54 -30 98 -93 -124 -67 97 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=1023, securePort=-1, UDP Port=-1, id={-34 63 -53 64 -40 -27 72 -119 -70 -46 -30 -59 -70 53 -25 -122 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=1002, securePort=-1, UDP Port=-1, id={-15 48 -82 108 -114 -100 72 41 -106 42 -30 -35 36 -73 -25 -12 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4013,{127, 0, 0, 1},4013, alive=1001, securePort=-1, UDP Port=-1, id={20 103 -44 26 -60 -88 69 -112 -86 9 -67 81 -26 -40 74 78 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=1004, securePort=-1, UDP Port=-1, id={-103 -77 -124 17 -28 -6 66 118 -78 93 20 -19 94 -80 -103 24 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4003,{127, 0, 0, 1},4003, alive=1013, securePort=-1, UDP Port=-1, id={32 -120 65 -123 30 -12 71 -60 -108 17 99 -4 -89 -60 -82 -39 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1010, securePort=-1, UDP Port=-1, id={-5 113 87 69 10 -3 66 126 -85 -54 -30 98 -93 -124 -67 97 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=1023, securePort=-1, UDP Port=-1, id={-34 63 -53 64 -40 -27 72 -119 -70 -46 -30 -59 -70 53 -25 -122 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=1002, securePort=-1, UDP Port=-1, id={-15 48 -82 108 -114 -100 72 41 -106 42 -30 -35 36 -73 -25 -12 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4013,{127, 0, 0, 1},4013, alive=1001, securePort=-1, UDP Port=-1, id={20 103 -44 26 -60 -88 69 -112 -86 9 -67 81 -26 -40 74 78 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=1004, securePort=-1, UDP Port=-1, id={-103 -77 -124 17 -28 -6 66 118 -78 93 20 -19 94 -80 -103 24 }, payload={}, command={}, domain={102 -37 -78 -45 -53 -119 77 113 -92 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:22 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Feb 21, 2024 11:57:23 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@62b1b384[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@3b5c8b01[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda/0x00007f957302fa60@287ea7ab]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@610694f1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2081) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:433) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:381) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:578) Feb 21, 2024 11:57:24 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@170752bf[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@6dd032c1[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda/0x00007f957302fa60@58c11b0]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@610694f1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2081) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:433) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:381) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:578) Feb 21, 2024 11:57:25 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@76575839[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@3e8cfca1[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda/0x00007f957302fa60@5be86b51]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@610694f1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2081) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:433) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:381) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:578) Feb 21, 2024 11:57:25 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Received memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=1514, securePort=-1, UDP Port=-1, id={3 -72 77 -119 -1 20 64 121 -106 24 52 98 -115 -57 -114 47 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-57 -95 36 127 -1 30 68 40 -101 ...(16)}]] message. Will verify. Feb 21, 2024 11:57:25 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Verification complete. Member still alive[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=1514, securePort=-1, UDP Port=-1, id={3 -72 77 -119 -1 20 64 121 -106 24 52 98 -115 -57 -114 47 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-57 -95 36 127 -1 30 68 40 -101 ...(16)}]] Feb 21, 2024 11:57:25 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1002, securePort=-1, UDP Port=-1, id={68 27 -40 -12 -119 7 64 -28 -128 -6 101 102 -10 -72 60 50 }, payload={}, command={}, domain={-82 -112 -100 -70 -7 -20 73 -50 -117 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:25 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=1002, securePort=-1, UDP Port=-1, id={126 -126 21 28 23 13 77 -121 -106 -41 0 29 -83 82 6 65 }, payload={}, command={}, domain={-82 -112 -100 -70 -7 -20 73 -50 -117 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:26 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@799ebb40[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@333af311[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda/0x00007f957302fa60@40102939]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@610694f1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2081) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:433) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:381) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:578) Feb 21, 2024 11:57:26 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@752491ea[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@3874deea[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda/0x00007f957302fa60@796c713c]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@610694f1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2081) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:433) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:381) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:578) Feb 21, 2024 11:57:27 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Feb 21, 2024 11:57:27 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4001] Feb 21, 2024 11:57:27 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Feb 21, 2024 11:57:27 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Feb 21, 2024 11:57:28 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=3508, securePort=-1, UDP Port=-1, id={68 27 -40 -12 -119 7 64 -28 -128 -6 101 102 -10 -72 60 50 }, payload={}, command={}, domain={-82 -112 -100 -70 -7 -20 73 -50 -117 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:28 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=3508, securePort=-1, UDP Port=-1, id={126 -126 21 28 23 13 77 -121 -106 -41 0 29 -83 82 6 65 }, payload={}, command={}, domain={-82 -112 -100 -70 -7 -20 73 -50 -117 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:28 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Feb 21, 2024 11:57:28 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Feb 21, 2024 11:57:29 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Feb 21, 2024 11:57:29 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Feb 21, 2024 11:57:29 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4002] Feb 21, 2024 11:57:29 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Feb 21, 2024 11:57:29 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Feb 21, 2024 11:57:30 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=5509, securePort=-1, UDP Port=-1, id={126 -126 21 28 23 13 77 -121 -106 -41 0 29 -83 82 6 65 }, payload={}, command={}, domain={-82 -112 -100 -70 -7 -20 73 -50 -117 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:30 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=5518, securePort=-1, UDP Port=-1, id={68 27 -40 -12 -119 7 64 -28 -128 -6 101 102 -10 -72 60 50 }, payload={}, command={}, domain={-82 -112 -100 -70 -7 -20 73 -50 -117 ...(16)}]] was refused to join cluster Feb 21, 2024 11:57:30 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Feb 21, 2024 11:57:31 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Feb 21, 2024 11:57:31 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Feb 21, 2024 11:57:31 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@52f3e127[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@580b8a57[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda/0x00007f957302fa60@56cc7f26]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@2a17b7b6[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2081) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:433) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:381) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:578) Feb 21, 2024 11:57:32 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Feb 21, 2024 11:57:32 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Feb 21, 2024 11:57:32 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4000] Feb 21, 2024 11:57:32 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Feb 21, 2024 11:57:32 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Feb 21, 2024 11:57:33 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Feb 21, 2024 11:57:33 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Feb 21, 2024 11:57:34 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Feb 21, 2024 11:57:34 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Feb 21, 2024 11:57:34 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4001] Feb 21, 2024 11:57:34 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Feb 21, 2024 11:57:34 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Feb 21, 2024 11:57:35 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Feb 21, 2024 11:57:35 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Feb 21, 2024 11:57:36 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Feb 21, 2024 11:57:36 AM org.apache.catalina.tribes.transport.nio.ParallelNioSender doLoop WARNING: Member send is failing for:[tcp://{127, 0, 0, 1}:4001] ; Setting to suspect and retrying. Feb 21, 2024 11:57:36 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Received memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=2013, securePort=-1, UDP Port=-1, id={89 111 -89 43 -100 -16 64 10 -105 -20 1 58 -59 -91 -45 -19 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={74 107 41 15 -109 88 76 10 -103 ...(16)}]] message. Will verify. Feb 21, 2024 11:57:36 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Verification complete. Member disappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=2013, securePort=-1, UDP Port=-1, id={89 111 -89 43 -100 -16 64 10 -105 -20 1 58 -59 -91 -45 -19 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={74 107 41 15 -109 88 76 10 -103 ...(16)}]] Feb 21, 2024 11:57:36 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@1e77f20d[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@5f4af836[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda/0x00007f957302fa60@5b756707]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@394e1a0f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2081) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:433) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:381) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:578) ------------- ---------------- --------------- Testcase: testTcpMcastFail took 9.525 sec Testcase: testTcpFailureMemberAdd took 4.259 sec Testcase: testTcpSendFailureMemberDrop took 4.196 sec