Testsuite: org.apache.catalina.tribes.group.interceptors.TestTcpFailureDetector Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.866 sec ------------- Standard Output --------------- testTcpMcastFail() Channel-2:member added[Channel-1] Channel-1:member added[Channel-2] testTcpFailureMemberAdd() Channel-2:member added[Channel-1] testTcpSendFailureMemberDrop() Channel-2:member added[Channel-1] Channel-1:member added[Channel-2] Channel-1:member disappeared[Channel-2] ------------- ---------------- --------------- ------------- Standard Error ----------------- Oct 05, 2023 10:56:50 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Oct 05, 2023 10:56:50 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4012] Oct 05, 2023 10:56:50 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Oct 05, 2023 10:56:50 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1066, securePort=-1, UDP Port=-1, id={64 61 -3 -24 -19 22 70 79 -66 14 -8 92 -22 -103 79 -82 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=1043, securePort=-1, UDP Port=-1, id={101 -63 -37 76 92 -72 79 -43 -69 -76 -104 37 93 -55 -66 -15 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=1021, securePort=-1, UDP Port=-1, id={100 -75 119 18 -5 -108 69 0 -72 -94 71 -78 94 117 -36 16 }, payload={}, command={}, domain={88 31 47 -115 80 -45 64 -65 -123 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=1022, securePort=-1, UDP Port=-1, id={-4 -36 12 126 -29 -97 72 8 -83 -108 0 49 -35 -41 122 -89 }, payload={}, command={}, domain={88 31 47 -115 80 -45 64 -65 -123 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1076, securePort=-1, UDP Port=-1, id={-93 41 87 -96 -101 -127 76 31 -109 57 73 -108 -108 22 -37 -35 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4011,{127, 0, 0, 1},4011, alive=1003, securePort=-1, UDP Port=-1, id={57 -68 53 -76 124 -100 79 107 -119 -84 -87 -10 76 73 -62 66 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=1006, securePort=-1, UDP Port=-1, id={-10 -45 -45 -54 -48 -65 65 76 -69 80 -104 -18 39 -39 71 98 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=1012, securePort=-1, UDP Port=-1, id={-123 -7 -27 37 65 108 72 -68 -95 -51 -101 -21 92 -15 12 -78 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=1010, securePort=-1, UDP Port=-1, id={-47 26 -91 -21 -7 -127 75 88 -92 -118 66 80 88 106 -63 -43 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1019, securePort=-1, UDP Port=-1, id={-15 108 67 -123 -121 43 64 106 -113 -128 4 -56 111 -60 -122 -60 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=1039, securePort=-1, UDP Port=-1, id={-104 16 37 -106 8 -4 78 -118 -100 -51 10 67 -25 6 -67 -30 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4003,{127, 0, 0, 1},4003, alive=1064, securePort=-1, UDP Port=-1, id={84 7 22 -51 -77 61 65 54 -112 58 60 -27 -38 20 -61 95 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Oct 05, 2023 10:56:51 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Oct 05, 2023 10:56:52 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Oct 05, 2023 10:56:52 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Oct 05, 2023 10:56:52 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4003] Oct 05, 2023 10:56:52 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Oct 05, 2023 10:56:52 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=2568, securePort=-1, UDP Port=-1, id={64 61 -3 -24 -19 22 70 79 -66 14 -8 92 -22 -103 79 -82 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=2523, securePort=-1, UDP Port=-1, id={-4 -36 12 126 -29 -97 72 8 -83 -108 0 49 -35 -41 122 -89 }, payload={}, command={}, domain={88 31 47 -115 80 -45 64 -65 -123 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4006,{127, 0, 0, 1},4006, alive=2523, securePort=-1, UDP Port=-1, id={100 -75 119 18 -5 -108 69 0 -72 -94 71 -78 94 117 -36 16 }, payload={}, command={}, domain={88 31 47 -115 80 -45 64 -65 -123 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=2606, securePort=-1, UDP Port=-1, id={-123 -7 -27 37 65 108 72 -68 -95 -51 -101 -21 92 -15 12 -78 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=2666, securePort=-1, UDP Port=-1, id={101 -63 -37 76 92 -72 79 -43 -69 -76 -104 37 93 -55 -66 -15 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=2611, securePort=-1, UDP Port=-1, id={-10 -45 -45 -54 -48 -65 65 76 -69 80 -104 -18 39 -39 71 98 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=2613, securePort=-1, UDP Port=-1, id={-47 26 -91 -21 -7 -127 75 88 -92 -118 66 80 88 106 -63 -43 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=2648, securePort=-1, UDP Port=-1, id={-104 16 37 -106 8 -4 78 -118 -100 -51 10 67 -25 6 -67 -30 }, payload={}, command={}, domain={-93 121 -41 -116 -93 76 69 59 -92 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.io.BufferPool getBufferPool INFO: Created a buffer pool with max size:[104857600] bytes of type: [org.apache.catalina.tribes.io.BufferPool15Impl] Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Oct 05, 2023 10:56:53 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1002, securePort=-1, UDP Port=-1, id={67 -122 -116 45 100 -125 72 104 -98 -6 25 59 86 -37 -82 -8 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1002, securePort=-1, UDP Port=-1, id={67 -122 -116 45 100 -125 72 104 -98 -6 25 59 86 -37 -82 -8 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1010, securePort=-1, UDP Port=-1, id={-104 -10 -116 25 -91 73 69 -34 -112 40 -118 50 86 17 37 93 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=1010, securePort=-1, UDP Port=-1, id={-104 -10 -116 25 -91 73 69 -34 -112 40 -118 50 86 17 37 93 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=1005, securePort=-1, UDP Port=-1, id={88 -34 -13 101 114 1 77 -104 -126 122 -8 36 114 104 112 -104 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4002,{127, 0, 0, 1},4002, alive=1005, securePort=-1, UDP Port=-1, id={88 -34 -13 101 114 1 77 -104 -126 122 -8 36 114 104 112 -104 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1005, securePort=-1, UDP Port=-1, id={-46 -108 -71 103 -11 73 67 66 -100 40 4 -37 79 66 31 -103 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=1005, securePort=-1, UDP Port=-1, id={17 -122 -100 50 -16 -69 67 97 -100 44 4 -106 44 81 -116 -43 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=1004, securePort=-1, UDP Port=-1, id={86 125 32 44 76 -66 72 -6 -125 77 -127 -69 -54 -119 -57 41 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4005,{127, 0, 0, 1},4005, alive=1005, securePort=-1, UDP Port=-1, id={-46 -108 -71 103 -11 73 67 66 -100 40 4 -37 79 66 31 -103 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4007,{127, 0, 0, 1},4007, alive=1005, securePort=-1, UDP Port=-1, id={17 -122 -100 50 -16 -69 67 97 -100 44 4 -106 44 81 -116 -43 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4008,{127, 0, 0, 1},4008, alive=1004, securePort=-1, UDP Port=-1, id={86 125 32 44 76 -66 72 -6 -125 77 -127 -69 -54 -119 -57 41 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=1029, securePort=-1, UDP Port=-1, id={7 -45 -81 -38 127 -43 74 -16 -119 -114 64 -49 -80 122 -122 97 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=1032, securePort=-1, UDP Port=-1, id={-113 -4 104 117 85 49 69 99 -77 32 -74 -53 -106 107 42 -58 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4013,{127, 0, 0, 1},4013, alive=1033, securePort=-1, UDP Port=-1, id={-89 37 -70 111 104 -2 77 44 -95 -57 -7 -44 35 0 -43 56 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4011,{127, 0, 0, 1},4011, alive=1038, securePort=-1, UDP Port=-1, id={-70 -11 47 67 4 -83 70 -25 -122 -86 -103 8 -52 67 -115 84 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4010,{127, 0, 0, 1},4010, alive=1029, securePort=-1, UDP Port=-1, id={7 -45 -81 -38 127 -43 74 -16 -119 -114 64 -49 -80 122 -122 97 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4009,{127, 0, 0, 1},4009, alive=1032, securePort=-1, UDP Port=-1, id={-113 -4 104 117 85 49 69 99 -77 32 -74 -53 -106 107 42 -58 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4013,{127, 0, 0, 1},4013, alive=1033, securePort=-1, UDP Port=-1, id={-89 37 -70 111 104 -2 77 44 -95 -57 -7 -44 35 0 -43 56 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4011,{127, 0, 0, 1},4011, alive=1038, securePort=-1, UDP Port=-1, id={-70 -11 47 67 4 -83 70 -25 -122 -86 -103 8 -52 67 -115 84 }, payload={}, command={}, domain={-62 -106 103 -36 -78 114 72 -14 -70 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:54 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Oct 05, 2023 10:56:56 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@11431549[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@3180af17[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@3b68e31f]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@3ac827b7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Oct 05, 2023 10:56:56 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@240b08d2[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@28352e67[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@1fa3ff3e]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@3ac827b7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Oct 05, 2023 10:56:57 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Received memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4003,{127, 0, 0, 1},4003, alive=1507, securePort=-1, UDP Port=-1, id={124 79 71 18 9 117 68 83 -75 -18 -88 -16 -48 33 -65 25 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-92 9 -54 -24 87 10 74 75 -84 ...(16)}]] message. Will verify. Oct 05, 2023 10:56:57 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Verification complete. Member still alive[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4003,{127, 0, 0, 1},4003, alive=1507, securePort=-1, UDP Port=-1, id={124 79 71 18 9 117 68 83 -75 -18 -88 -16 -48 33 -65 25 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-92 9 -54 -24 87 10 74 75 -84 ...(16)}]] Oct 05, 2023 10:56:57 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@53feaec2[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@149d6bd[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@514bcf1c]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@3ac827b7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Oct 05, 2023 10:56:57 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=1004, securePort=-1, UDP Port=-1, id={114 -47 72 -48 -120 34 66 106 -94 -19 93 30 -93 -26 -118 -16 }, payload={}, command={}, domain={-51 37 119 -95 -7 49 68 -113 -119 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:57 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=1004, securePort=-1, UDP Port=-1, id={114 -94 -104 62 77 -120 78 -41 -110 49 58 73 -87 -27 -77 59 }, payload={}, command={}, domain={-51 37 119 -95 -7 49 68 -113 -119 ...(16)}]] was refused to join cluster Oct 05, 2023 10:56:58 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@2fd47531[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@400ac712[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@14c0d220]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@3ac827b7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Oct 05, 2023 10:56:59 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@4b7fa5f5[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@554da483[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@470e2dd8]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@3ac827b7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 28] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Oct 05, 2023 10:57:00 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Oct 05, 2023 10:57:00 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4001] Oct 05, 2023 10:57:00 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Oct 05, 2023 10:57:00 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Oct 05, 2023 10:57:00 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=3509, securePort=-1, UDP Port=-1, id={114 -47 72 -48 -120 34 66 106 -94 -19 93 30 -93 -26 -118 -16 }, payload={}, command={}, domain={-51 37 119 -95 -7 49 68 -113 -119 ...(16)}]] was refused to join cluster Oct 05, 2023 10:57:00 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=3508, securePort=-1, UDP Port=-1, id={114 -94 -104 62 77 -120 78 -41 -110 49 58 73 -87 -27 -77 59 }, payload={}, command={}, domain={-51 37 119 -95 -7 49 68 -113 -119 ...(16)}]] was refused to join cluster Oct 05, 2023 10:57:01 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Oct 05, 2023 10:57:01 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Oct 05, 2023 10:57:02 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Oct 05, 2023 10:57:02 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Oct 05, 2023 10:57:02 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4002] Oct 05, 2023 10:57:02 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Oct 05, 2023 10:57:02 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Oct 05, 2023 10:57:02 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4000,{127, 0, 0, 1},4000, alive=5510, securePort=-1, UDP Port=-1, id={114 -47 72 -48 -120 34 66 106 -94 -19 93 30 -93 -26 -118 -16 }, payload={}, command={}, domain={-51 37 119 -95 -7 49 68 -113 -119 ...(16)}]] was refused to join cluster Oct 05, 2023 10:57:02 AM org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor memberAdded INFO: Member [org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4004,{127, 0, 0, 1},4004, alive=5523, securePort=-1, UDP Port=-1, id={114 -94 -104 62 77 -120 78 -41 -110 49 58 73 -87 -27 -77 59 }, payload={}, command={}, domain={-51 37 119 -95 -7 49 68 -113 -119 ...(16)}]] was refused to join cluster Oct 05, 2023 10:57:03 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Oct 05, 2023 10:57:03 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Oct 05, 2023 10:57:03 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Oct 05, 2023 10:57:03 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@d6f0c8c[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@79767fc[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@950897e]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@486d8345[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 6] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) Oct 05, 2023 10:57:04 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Oct 05, 2023 10:57:04 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Oct 05, 2023 10:57:04 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4000] Oct 05, 2023 10:57:04 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Oct 05, 2023 10:57:04 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Oct 05, 2023 10:57:05 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Oct 05, 2023 10:57:05 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Oct 05, 2023 10:57:06 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Oct 05, 2023 10:57:06 AM org.apache.catalina.tribes.group.GroupChannel start WARNING: No utility executor was set, creating one Oct 05, 2023 10:57:06 AM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Receiver Server Socket bound to:[localhost/127.0.0.1:4001] Oct 05, 2023 10:57:06 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket INFO: Setting cluster mcast soTimeout to [500] Oct 05, 2023 10:57:06 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[4] Oct 05, 2023 10:57:07 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[4] Oct 05, 2023 10:57:07 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Sleeping for [1000] milliseconds to establish cluster membership, start level:[8] Oct 05, 2023 10:57:08 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers INFO: Done sleeping, membership established, start level:[8] Oct 05, 2023 10:57:08 AM org.apache.catalina.tribes.transport.nio.ParallelNioSender doLoop WARNING: Member send is failing for:[tcp://{127, 0, 0, 1}:4001] ; Setting to suspect and retrying. Oct 05, 2023 10:57:08 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Received memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=2016, securePort=-1, UDP Port=-1, id={82 35 22 46 -64 -31 75 -62 -126 50 19 -38 118 -31 30 56 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-81 99 116 -102 88 48 68 -84 -71 ...(16)}]] message. Will verify. Oct 05, 2023 10:57:08 AM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared INFO: Verification complete. Member disappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 0, 1}:4001,{127, 0, 0, 1},4001, alive=2016, securePort=-1, UDP Port=-1, id={82 35 22 46 -64 -31 75 -62 -126 50 19 -38 118 -31 30 56 }, payload={67 104 97 110 110 101 108 45 50 ...(9)}, command={}, domain={-81 99 116 -102 88 48 68 -84 -71 ...(16)}]] Oct 05, 2023 10:57:08 AM org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread run WARNING: Error receiving mcast package. Sleeping 500ms java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@67c30377[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@26fe094c[Wrapped task = org.apache.catalina.tribes.membership.McastServiceImpl$$Lambda$69/0x00000008010c1230@22c1894b]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@8c1d8e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2] at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705) at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:432) at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:380) at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:580) ------------- ---------------- --------------- Testcase: testTcpMcastFail took 9.411 sec Testcase: testTcpFailureMemberAdd took 4.223 sec Testcase: testTcpSendFailureMemberDrop took 4.215 sec