Testsuite: org.apache.catalina.tribes.group.interceptors.TestTcpFailureDetector Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.887 sec ------------- Standard Output --------------- testTcpMcastFail() Channel-2:member added[Channel-1] Channel-1:member added[Channel-2] testTcpFailureMemberAdd() Channel-2:member added[Channel-1] testTcpSendFailureMemberDrop() Channel-2:member added[Channel-1] Channel-1:member added[Channel-2] Channel-1:member disappeared[Channel-2] ------------- ---------------- --------------- ------------- Standard Error ----------------- Sep 28, 2023 11:01:17 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Sep 28, 2023 11:01:17 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4012] Sep 28, 2023 11:01:17 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Sep 28, 2023 11:01:17 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4003,{127, 0, 0, 1},4003, alive=1008, securePort=-1, UDP Port=-1, id={-3 -4 122 86 -4 -96 73 124 -75 -64 17 -59 -53 -78 -82 -4 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=1030, securePort=-1, UDP Port=-1, id={44 62 -66 101 -20 -107 70 112 -120 105 78 -1 81 -43 55 -77 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=1010, securePort=-1, UDP Port=-1, id={-3 -80 -98 72 116 -56 68 45 -96 62 -114 -99 127 -100 -116 20 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1129, securePort=-1, UDP Port=-1, id={109 15 77 51 -95 -52 70 111 -127 -77 9 -45 -43 -102 -30 27 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1134, securePort=-1, UDP Port=-1, id={14 -7 -119 61 -74 -89 64 104 -78 124 -42 -39 42 -78 -50 -81 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1013, securePort=-1, UDP Port=-1, id={63 9 -68 -89 -127 16 77 8 -122 96 103 102 34 -86 -122 -51 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=1011, securePort=-1, UDP Port=-1, id={-90 124 24 97 -44 -74 68 -114 -91 62 -120 121 -127 -38 21 -102 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=1003, securePort=-1, UDP Port=-1, id={35 14 -97 -73 92 -119 76 -16 -83 -36 87 113 50 55 93 54 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=1006, securePort=-1, UDP Port=-1, id={64 -9 35 119 -53 -115 76 112 -100 -19 56 21 43 -28 66 32 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=1050, securePort=-1, UDP Port=-1, id={-48 -123 -37 -69 -119 125 73 -26 -95 -2 -2 66 100 109 97 -43 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4011,{127, 0, 0, 1},4011, alive=1009, securePort=-1, UDP Port=-1, id={-108 -22 -94 121 -110 -43 69 -18 -82 119 -49 -1 -6 38 120 -6 }, payload={}, command={}, domain={-72 80 99 100 -24 -4 79 -72 -86 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=1021, securePort=-1, UDP Port=-1, id={125 -16 -116 25 122 15 64 37 -107 -126 43 39 101 56 94 58 }, payload={}, command={}, domain={-72 80 99 100 -24 -4 79 -72 -86 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Sep 28, 2023 11:01:18 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Sep 28, 2023 11:01:19 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Sep 28, 2023 11:01:19 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Sep 28, 2023 11:01:19 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4000] Sep 28, 2023 11:01:19 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Sep 28, 2023 11:01:19 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Sep 28, 2023 11:01:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=2518, securePort=-1, UDP Port=-1, id={-3 -80 -98 72 116 -56 68 45 -96 62 -114 -99 127 -100 -116 20 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:19 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=2567, securePort=-1, UDP Port=-1, id={44 62 -66 101 -20 -107 70 112 -120 105 78 -1 81 -43 55 -77 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:20 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=2536, securePort=-1, UDP Port=-1, id={63 9 -68 -89 -127 16 77 8 -122 96 103 102 34 -86 -122 -51 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:20 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=2538, securePort=-1, UDP Port=-1, id={35 14 -97 -73 92 -119 76 -16 -83 -36 87 113 50 55 93 54 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:20 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=2542, securePort=-1, UDP Port=-1, id={64 -9 35 119 -53 -115 76 112 -100 -19 56 21 43 -28 66 32 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:20 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=2549, securePort=-1, UDP Port=-1, id={-90 124 24 97 -44 -74 68 -114 -91 62 -120 121 -127 -38 21 -102 }, payload={}, command={}, domain={22 -28 -105 -30 -54 125 74 23 -72 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:20 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4011,{127, 0, 0, 1},4011, alive=2519, securePort=-1, UDP Port=-1, id={-108 -22 -94 121 -110 -43 69 -18 -82 119 -49 -1 -6 38 120 -6 }, payload={}, command={}, domain={-72 80 99 100 -24 -4 79 -72 -86 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:20 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=2528, securePort=-1, UDP Port=-1, id={125 -16 -116 25 122 15 64 37 -107 -126 43 39 101 56 94 58 }, payload={}, command={}, domain={-72 80 99 100 -24 -4 79 -72 -86 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:20 AM org.apache.catalina.tribes.io.BufferPool getBufferPool INFO: Created a buffer pool with max size:[104857600] bytes of type: [org.apache.catalina.tribes.io.BufferPool15Impl] Sep 28, 2023 11:01:20 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Sep 28, 2023 11:01:20 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1002, securePort=-1, UDP Port=-1, id={90 94 24 69 102 -117 68 86 -90 71 -44 2 89 55 -78 -75 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1002, securePort=-1, UDP Port=-1, id={90 94 24 69 102 -117 68 86 -90 71 -44 2 89 55 -78 -75 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4003,{127, 0, 0, 1},4003, alive=1001, securePort=-1, UDP Port=-1, id={-86 18 0 -89 -97 -72 65 115 -73 78 -66 22 120 39 107 -5 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4003,{127, 0, 0, 1},4003, alive=1001, securePort=-1, UDP Port=-1, id={-86 18 0 -89 -97 -72 65 115 -73 78 -66 22 120 39 107 -5 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=1001, securePort=-1, UDP Port=-1, id={-51 -15 87 -113 -112 -16 69 2 -126 124 -61 -97 2 79 88 -47 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1029, securePort=-1, UDP Port=-1, id={-41 108 102 -4 56 65 66 -120 -97 -24 -17 -55 115 62 109 -113 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=1039, securePort=-1, UDP Port=-1, id={-51 -15 87 -113 -112 -16 69 2 -126 124 -61 -97 2 79 88 -47 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1029, securePort=-1, UDP Port=-1, id={-41 108 102 -4 56 65 66 -120 -97 -24 -17 -55 115 62 109 -113 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=1010, securePort=-1, UDP Port=-1, id={-49 -62 99 -83 -64 75 75 -60 -118 122 -71 5 -94 83 -94 5 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=1040, securePort=-1, UDP Port=-1, id={-39 -52 76 107 -100 -113 79 -38 -97 -11 26 71 -82 79 -65 30 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=1022, securePort=-1, UDP Port=-1, id={-34 121 31 -60 -4 113 66 127 -72 -57 112 22 85 18 71 26 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=1016, securePort=-1, UDP Port=-1, id={-125 -48 15 49 125 101 65 -32 -69 -101 10 33 67 -79 118 28 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=1010, securePort=-1, UDP Port=-1, id={-49 -62 99 -83 -64 75 75 -60 -118 122 -71 5 -94 83 -94 5 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4013,{127, 0, 0, 1},4013, alive=1018, securePort=-1, UDP Port=-1, id={119 -62 95 64 -21 -90 64 113 -95 -11 -100 117 -102 4 -120 75 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=1015, securePort=-1, UDP Port=-1, id={77 -35 73 72 -65 -94 71 1 -128 125 -84 -3 -112 -106 7 -112 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=1040, securePort=-1, UDP Port=-1, id={-39 -52 76 107 -100 -113 79 -38 -97 -11 26 71 -82 79 -65 30 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=1022, securePort=-1, UDP Port=-1, id={-34 121 31 -60 -4 113 66 127 -72 -57 112 22 85 18 71 26 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=1016, securePort=-1, UDP Port=-1, id={-125 -48 15 49 125 101 65 -32 -69 -101 10 33 67 -79 118 28 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4013,{127, 0, 0, 1},4013, alive=1018, securePort=-1, UDP Port=-1, id={119 -62 95 64 -21 -90 64 113 -95 -11 -100 117 -102 4 -120 75 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=1015, securePort=-1, UDP Port=-1, id={77 -35 73 72 -65 -94 71 1 -128 125 -84 -3 -112 -106 7 -112 }, payload={}, command={}, domain={-117 83 22 -78 -114 -84 73 28 -77 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:21 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Sep 28, 2023 11:01:22 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@4853ac2f[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@4a362529[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@2d8e2a99]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@45cd7cc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Sep 28, 2023 11:01:23 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@44eaef67[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@2e730496[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@790e41ad]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@45cd7cc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Sep 28, 2023 11:01:24 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@37d85d2b[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@2fd2307e[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@24da0f00]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@45cd7cc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Sep 28, 2023 11:01:24 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Received memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1516, securePort=-1, UDP Port=-1, id={7 -2 -77 124 80 19 77 -82 -108 -56 -88 -61 -37 35 -52 -76 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-92 -8 -40 -74 -68 -62 72 -85 -69 ...(16)}]] message. Will verify. Sep 28, 2023 11:01:24 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Verification complete. Member still alive[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1516, securePort=-1, UDP Port=-1, id={7 -2 -77 124 80 19 77 -82 -108 -56 -88 -61 -37 35 -52 -76 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-92 -8 -40 -74 -68 -62 72 -85 -69 ...(16)}]] Sep 28, 2023 11:01:25 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1002, securePort=-1, UDP Port=-1, id={-26 -92 99 -115 -72 98 73 61 -104 19 -105 124 109 72 -18 -31 }, payload={}, command={}, domain={-49 87 -83 -78 -48 -99 67 -35 -121 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:25 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1005, securePort=-1, UDP Port=-1, id={-116 76 106 -77 -40 -91 69 -40 -73 -125 6 -77 3 21 -28 106 }, payload={}, command={}, domain={-49 87 -83 -78 -48 -99 67 -35 -121 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:25 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@7aa67508[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@19c9b29d[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@2603af8e]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@45cd7cc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Sep 28, 2023 11:01:26 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@4b3947b2[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@553a15c3[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@35d012ce]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@45cd7cc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 27] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Sep 28, 2023 11:01:27 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Sep 28, 2023 11:01:27 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4000] Sep 28, 2023 11:01:27 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Sep 28, 2023 11:01:27 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Sep 28, 2023 11:01:27 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=3506, securePort=-1, UDP Port=-1, id={-26 -92 99 -115 -72 98 73 61 -104 19 -105 124 109 72 -18 -31 }, payload={}, command={}, domain={-49 87 -83 -78 -48 -99 67 -35 -121 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:27 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=3510, securePort=-1, UDP Port=-1, id={-116 76 106 -77 -40 -91 69 -40 -73 -125 6 -77 3 21 -28 106 }, payload={}, command={}, domain={-49 87 -83 -78 -48 -99 67 -35 -121 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:28 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Sep 28, 2023 11:01:28 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Sep 28, 2023 11:01:29 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Sep 28, 2023 11:01:29 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Sep 28, 2023 11:01:29 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4002] Sep 28, 2023 11:01:29 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Sep 28, 2023 11:01:29 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Sep 28, 2023 11:01:29 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=5507, securePort=-1, UDP Port=-1, id={-26 -92 99 -115 -72 98 73 61 -104 19 -105 124 109 72 -18 -31 }, payload={}, command={}, domain={-49 87 -83 -78 -48 -99 67 -35 -121 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:29 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=5511, securePort=-1, UDP Port=-1, id={-116 76 106 -77 -40 -91 69 -40 -73 -125 6 -77 3 21 -28 106 }, payload={}, command={}, domain={-49 87 -83 -78 -48 -99 67 -35 -121 ...(16)}]] was refused to join cluster Sep 28, 2023 11:01:30 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Sep 28, 2023 11:01:30 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Sep 28, 2023 11:01:30 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Sep 28, 2023 11:01:31 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@d6f0c8c[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@79767fc[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@950897e]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@486d8345[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Sep 28, 2023 11:01:31 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Sep 28, 2023 11:01:31 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Sep 28, 2023 11:01:31 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4000] Sep 28, 2023 11:01:31 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Sep 28, 2023 11:01:31 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Sep 28, 2023 11:01:32 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Sep 28, 2023 11:01:32 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Sep 28, 2023 11:01:33 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Sep 28, 2023 11:01:33 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Sep 28, 2023 11:01:33 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4001] Sep 28, 2023 11:01:33 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Sep 28, 2023 11:01:33 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Sep 28, 2023 11:01:34 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Sep 28, 2023 11:01:34 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Sep 28, 2023 11:01:35 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Sep 28, 2023 11:01:35 AM org.apache.catalina.tribes.transport.nio.ParallelNioSender doLoop WARNING: Member send is failing for:[tcp://{127, 0, 0, 1}:4001] ; Setting to suspect and retrying. Sep 28, 2023 11:01:35 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Received memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=2032, securePort=-1, UDP Port=-1, id={-66 -103 -126 104 32 -107 79 -125 -111 80 -109 -88 67 -14 -102 48 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-127 42 -113 -73 49 86 67 -119 -113 ...(16)}]] message. Will verify. Sep 28, 2023 11:01:35 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Verification complete. Member disappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=2032, securePort=-1, UDP Port=-1, id={-66 -103 -126 104 32 -107 79 -125 -111 80 -109 -88 67 -14 -102 48 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-127 42 -113 -73 49 86 67 -119 -113 ...(16)}]] Sep 28, 2023 11:01:35 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@67c30377[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@26fe094c[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@22c1894b]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@8c1d8e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) ------------- ---------------- --------------- Testcase: testTcpMcastFail took 9.448 sec Testcase: testTcpFailureMemberAdd took 4.227 sec Testcase: testTcpSendFailureMemberDrop took 4.179 sec