2024-04-09 16:54:00,026 DEBUG [main] util.ClassSize(228): Using Unsafe to estimate memory layout 2024-04-09 16:54:00,372 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.ipc.TestBlockingIPC timeout: 13 mins 2024-04-09 16:54:00,456 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testRTEDuringConnectionSetup Thread=10, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=1564, ProcessCount=195, AvailableMemoryMB=564 2024-04-09 16:54:00,787 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:01,108 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2024-04-09 16:54:01,297 INFO [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2024-04-09 16:54:01,375 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2024-04-09 16:54:01,647 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=37471 2024-04-09 16:54:01,837 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f62932c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:01,861 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:01,864 INFO [Listener,port=37471] ipc.SimpleRpcServer$Listener(214): Listener,port=37471: starting 2024-04-09 16:54:01,861 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:02,332 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:02,481 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:37471 2024-04-09 16:54:02,485 DEBUG [Time-limited test] ipc.FailedServers(52): Added failed server with address localhost:37471 to list caused by java.lang.RuntimeException: Injected fault 2024-04-09 16:54:02,486 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:02,486 INFO [Time-limited test] ipc.AbstractTestIPC(171): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: java.io.IOException: Call to address=localhost:37471 failed on local exception: java.io.IOException: Could not set up IO Streams to localhost:37471 2024-04-09 16:54:02,488 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 37471 2024-04-09 16:54:02,488 INFO [Listener,port=37471] ipc.SimpleRpcServer$Listener(258): Listener,port=37471: stopping 2024-04-09 16:54:02,489 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:02,489 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:02,516 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testRTEDuringConnectionSetup Thread=13 (was 10) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: Idle-Rpc-Conn-Sweeper-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=291 (was 290) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1564 (was 1564), ProcessCount=196 (was 195) - ProcessCount LEAK? -, AvailableMemoryMB=275 (was 564) 2024-04-09 16:54:02,537 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testTimeout Thread=13, OpenFileDescriptor=291, MaxFileDescriptor=60000, SystemLoadAverage=1564, ProcessCount=196, AvailableMemoryMB=264 2024-04-09 16:54:02,537 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:02,553 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=34765 2024-04-09 16:54:02,556 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@44498cec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:02,561 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:02,561 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:02,561 INFO [Listener,port=34765] ipc.SimpleRpcServer$Listener(214): Listener,port=34765: starting 2024-04-09 16:54:02,568 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:02,579 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:34765 2024-04-09 16:54:02,647 INFO [Reader=1,bindAddress=localhost,port=34765] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:46726, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:02,705 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=0,methodName=pause], waitTime=136ms, rpcTimeout=100ms 2024-04-09 16:54:02,813 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=1,methodName=pause], waitTime=104ms, rpcTimeout=100ms 2024-04-09 16:54:02,918 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=2,methodName=pause], waitTime=104ms, rpcTimeout=100ms 2024-04-09 16:54:03,029 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=3,methodName=pause], waitTime=111ms, rpcTimeout=100ms 2024-04-09 16:54:03,139 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=4,methodName=pause], waitTime=110ms, rpcTimeout=100ms 2024-04-09 16:54:03,249 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=5,methodName=pause], waitTime=108ms, rpcTimeout=100ms 2024-04-09 16:54:03,358 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=6,methodName=pause], waitTime=108ms, rpcTimeout=100ms 2024-04-09 16:54:03,474 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7,methodName=pause], waitTime=115ms, rpcTimeout=100ms 2024-04-09 16:54:03,581 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8,methodName=pause], waitTime=107ms, rpcTimeout=100ms 2024-04-09 16:54:03,689 INFO [Time-limited test] ipc.AbstractTestIPC(293): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:34765 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=9,methodName=pause], waitTime=107ms, rpcTimeout=100ms 2024-04-09 16:54:03,689 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:03,690 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 34765 2024-04-09 16:54:03,693 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:03,697 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:03,693 INFO [Listener,port=34765] ipc.SimpleRpcServer$Listener(258): Listener,port=34765: stopping 2024-04-09 16:54:03,712 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.SimpleRpcServerResponder(238): 127.0.0.1:46726: output error -- closing 2024-04-09 16:54:03,713 WARN [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(181): FifoRpcScheduler.handler-pool-0: caught a ClosedChannelException, this means that the server localhost/127.0.0.1:34765 was processing a request but the client went away. The error message was: null 2024-04-09 16:54:03,734 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testTimeout Thread=16 (was 13) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcClient-timer-pool-0 java.lang.Thread.sleep(Native Method) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: FifoRpcScheduler.handler-pool-0 java.lang.String.regionMatches(String.java:1376) java.net.URL.(URL.java:558) java.net.URL.(URL.java:508) sun.misc.URLClassPath$FileLoader.getResource(URLClassPath.java:1369) sun.misc.URLClassPath.getResource(URLClassPath.java:250) java.net.URLClassLoader$1.run(URLClassLoader.java:366) java.net.URLClassLoader$1.run(URLClassLoader.java:363) java.security.AccessController.doPrivileged(Native Method) java.net.URLClassLoader.findClass(URLClassLoader.java:362) java.lang.ClassLoader.loadClass(ClassLoader.java:418) sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) java.lang.ClassLoader.loadClass(ClassLoader.java:351) org.apache.hbase.thirdparty.com.google.protobuf.CodedOutputStream$HeapNioEncoder.flush(CodedOutputStream.java:1525) org.apache.hadoop.hbase.ipc.ServerCall.writeToCOS(ServerCall.java:374) org.apache.hadoop.hbase.ipc.ServerCall.createHeaderAndMessageBytes(ServerCall.java:381) org.apache.hadoop.hbase.ipc.ServerCall.createHeaderAndMessageBytes(ServerCall.java:359) org.apache.hadoop.hbase.ipc.ServerCall.setResponse(ServerCall.java:267) org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:167) org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:113) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=294 (was 291) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1564), ProcessCount=200 (was 196) - ProcessCount LEAK? -, AvailableMemoryMB=282 (was 264) - AvailableMemoryMB LEAK? - 2024-04-09 16:54:03,744 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(99): FifoRpcScheduler.handler-pool-0: skipped callId: 1 service: TestProtobufRpcProto methodName: pause size: 20 connection: 127.0.0.1:46726 deadline: 1712681642806 param: TODO: class org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestProtos$PauseRequestProto connection: 127.0.0.1:46726 2024-04-09 16:54:03,744 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(99): FifoRpcScheduler.handler-pool-0: skipped callId: 2 service: TestProtobufRpcProto methodName: pause size: 20 connection: 127.0.0.1:46726 deadline: 1712681642914 param: TODO: class org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestProtos$PauseRequestProto connection: 127.0.0.1:46726 2024-04-09 16:54:03,766 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(99): FifoRpcScheduler.handler-pool-0: skipped callId: 3 service: TestProtobufRpcProto methodName: pause size: 20 connection: 127.0.0.1:46726 deadline: 1712681643019 param: TODO: class org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestProtos$PauseRequestProto connection: 127.0.0.1:46726 2024-04-09 16:54:03,766 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(99): FifoRpcScheduler.handler-pool-0: skipped callId: 4 service: TestProtobufRpcProto methodName: pause size: 20 connection: 127.0.0.1:46726 deadline: 1712681643134 param: TODO: class org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestProtos$PauseRequestProto connection: 127.0.0.1:46726 2024-04-09 16:54:03,767 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(99): FifoRpcScheduler.handler-pool-0: skipped callId: 5 service: TestProtobufRpcProto methodName: pause size: 20 connection: 127.0.0.1:46726 deadline: 1712681643242 param: TODO: class org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestProtos$PauseRequestProto connection: 127.0.0.1:46726 2024-04-09 16:54:03,767 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(99): FifoRpcScheduler.handler-pool-0: skipped callId: 6 service: TestProtobufRpcProto methodName: pause size: 20 connection: 127.0.0.1:46726 deadline: 1712681643351 param: TODO: class org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestProtos$PauseRequestProto connection: 127.0.0.1:46726 2024-04-09 16:54:03,767 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(99): FifoRpcScheduler.handler-pool-0: skipped callId: 7 service: TestProtobufRpcProto methodName: pause size: 20 connection: 127.0.0.1:46726 deadline: 1712681643459 param: TODO: class org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestProtos$PauseRequestProto connection: 127.0.0.1:46726 2024-04-09 16:54:03,767 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(99): FifoRpcScheduler.handler-pool-0: skipped callId: 8 service: TestProtobufRpcProto methodName: pause size: 20 connection: 127.0.0.1:46726 deadline: 1712681643575 param: TODO: class org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestProtos$PauseRequestProto connection: 127.0.0.1:46726 2024-04-09 16:54:03,768 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(99): FifoRpcScheduler.handler-pool-0: skipped callId: 9 service: TestProtobufRpcProto methodName: pause size: 20 connection: 127.0.0.1:46726 deadline: 1712681643683 param: TODO: class org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestProtos$PauseRequestProto connection: 127.0.0.1:46726 2024-04-09 16:54:03,792 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testRpcScheduler Thread=16, OpenFileDescriptor=294, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=200, AvailableMemoryMB=275 2024-04-09 16:54:04,930 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:04,948 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=40715 2024-04-09 16:54:05,061 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47508ceb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:05,062 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:05,076 INFO [Listener,port=40715] ipc.SimpleRpcServer$Listener(214): Listener,port=40715: starting 2024-04-09 16:54:05,076 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler$MockitoMock$292915890 as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:05,098 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:05,099 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:40715 2024-04-09 16:54:05,130 INFO [Reader=1,bindAddress=localhost,port=40715] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:41188, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:05,262 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:05,263 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 40715 2024-04-09 16:54:05,264 DEBUG [BRPC Connection (86992864) to localhost:40715 from jenkins (attempt: 1)] ipc.BlockingRpcConnection(369): Interrupted while waiting for work 2024-04-09 16:54:05,269 INFO [Listener,port=40715] ipc.SimpleRpcServer$Listener(258): Listener,port=40715: stopping 2024-04-09 16:54:05,273 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:05,273 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:05,327 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testRpcScheduler Thread=21 (was 16) Potentially hanging thread: RpcServer.responder sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.doRunLoop(SimpleRpcServerResponder.java:112) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.run(SimpleRpcServerResponder.java:60) Potentially hanging thread: RpcServer idle connection scanner for port 40715 java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: Reader=1,bindAddress=localhost,port=40715 sun.nio.ch.Util.free(Util.java:305) sun.nio.ch.Util.access$000(Util.java:44) sun.nio.ch.Util$1.threadTerminated(Util.java:64) sun.nio.ch.Util$1.threadTerminated(Util.java:55) jdk.internal.misc.TerminatingThreadLocal._threadTerminated(TerminatingThreadLocal.java:74) jdk.internal.misc.TerminatingThreadLocal.threadTerminated(TerminatingThreadLocal.java:70) java.lang.Thread.exit(Thread.java:760) Potentially hanging thread: Reader=2,bindAddress=localhost,port=40715 java.lang.System.arraycopy(Native Method) java.lang.ThreadGroup.remove(ThreadGroup.java:969) java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942) java.lang.Thread.exit(Thread.java:763) Potentially hanging thread: Listener,port=40715 sun.misc.Unsafe.freeMemory(Native Method) sun.nio.ch.AllocatedNativeObject.free(AllocatedNativeObject.java:61) sun.nio.ch.EPollArrayWrapper.closeEPollFD(EPollArrayWrapper.java:264) sun.nio.ch.EPollSelectorImpl.implClose(EPollSelectorImpl.java:153) sun.nio.ch.SelectorImpl.implCloseSelector(SelectorImpl.java:113) java.nio.channels.spi.AbstractSelector.close(AbstractSelector.java:111) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.run(SimpleRpcServer.java:262) - Thread LEAK? -, OpenFileDescriptor=297 (was 294) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=197 (was 200), AvailableMemoryMB=250 (was 275) 2024-04-09 16:54:05,355 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testAsyncRemoteError Thread=15, OpenFileDescriptor=296, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=197, AvailableMemoryMB=237 2024-04-09 16:54:05,355 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ff01a0e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:05,356 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:05,358 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=42901 2024-04-09 16:54:05,369 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:05,370 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:05,383 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:05,384 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:42901 2024-04-09 16:54:05,376 INFO [Listener,port=42901] ipc.SimpleRpcServer$Listener(214): Listener,port=42901: starting 2024-04-09 16:54:05,434 INFO [Reader=1,bindAddress=localhost,port=42901] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:41074, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:05,447 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.DoNotRetryIOException: server error! at org.apache.hadoop.hbase.ipc.TestProtobufRpcServiceImpl.error(TestProtobufRpcServiceImpl.java:104) at org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestRpcServiceProtos$TestProtobufRpcProto$2.callBlockingMethod(TestRpcServiceProtos.java:142) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:113) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-09 16:54:05,454 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(144): callId: 0 service: TestProtobufRpcProto methodName: error size: 17 connection: 127.0.0.1:41074 deadline: 9223372036854775807, exception=org.apache.hadoop.hbase.DoNotRetryIOException: server error! 2024-04-09 16:54:05,462 INFO [Time-limited test] ipc.AbstractTestIPC(374): Caught expected exception: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): org.apache.hadoop.hbase.DoNotRetryIOException: server error! at org.apache.hadoop.hbase.ipc.TestProtobufRpcServiceImpl.error(TestProtobufRpcServiceImpl.java:104) at org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestRpcServiceProtos$TestProtobufRpcProto$2.callBlockingMethod(TestRpcServiceProtos.java:142) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:113) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-09 16:54:06,161 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:06,162 DEBUG [BRPC Connection (86992864) to localhost:42901 from jenkins (attempt: 1)] ipc.BlockingRpcConnection(369): Interrupted while waiting for work 2024-04-09 16:54:06,162 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 42901 2024-04-09 16:54:06,163 INFO [Listener,port=42901] ipc.SimpleRpcServer$Listener(258): Listener,port=42901: stopping 2024-04-09 16:54:06,165 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:06,165 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:06,195 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testAsyncRemoteError Thread=17 (was 15) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=294 (was 296), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=197 (was 197), AvailableMemoryMB=237 (was 237) 2024-04-09 16:54:06,218 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testAsyncEcho Thread=17, OpenFileDescriptor=294, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=197, AvailableMemoryMB=235 2024-04-09 16:54:06,284 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:06,301 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=39189 2024-04-09 16:54:06,306 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@359a2938, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:06,306 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:06,329 INFO [Listener,port=39189] ipc.SimpleRpcServer$Listener(214): Listener,port=39189: starting 2024-04-09 16:54:06,317 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:06,335 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:06,336 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:39189 2024-04-09 16:54:06,342 INFO [Reader=1,bindAddress=localhost,port=39189] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:59794, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:06,370 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:06,370 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 39189 2024-04-09 16:54:06,371 INFO [Listener,port=39189] ipc.SimpleRpcServer$Listener(258): Listener,port=39189: stopping 2024-04-09 16:54:06,371 DEBUG [BRPC Connection (86992864) to localhost:39189 from jenkins (attempt: 1)] ipc.BlockingRpcConnection(369): Interrupted while waiting for work 2024-04-09 16:54:06,417 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:06,417 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:06,450 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testAsyncEcho Thread=19 (was 17) Potentially hanging thread: RpcServer.responder sun.misc.Unsafe.getInt(Native Method) sun.nio.ch.NativeObject.getInt(NativeObject.java:257) sun.nio.ch.EPollArrayWrapper.getDescriptor(EPollArrayWrapper.java:165) sun.nio.ch.EPollSelectorImpl.updateSelectedKeys(EPollSelectorImpl.java:119) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:98) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.doRunLoop(SimpleRpcServerResponder.java:112) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.run(SimpleRpcServerResponder.java:60) - Thread LEAK? -, OpenFileDescriptor=294 (was 294), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=200 (was 197) - ProcessCount LEAK? -, AvailableMemoryMB=275 (was 235) - AvailableMemoryMB LEAK? - 2024-04-09 16:54:06,477 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testConnectionCloseWithOutstandingRPCs Thread=17, OpenFileDescriptor=294, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=200, AvailableMemoryMB=274 2024-04-09 16:54:06,493 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=40285 2024-04-09 16:54:06,502 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1af77779, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:06,505 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:06,511 INFO [Listener,port=40285] ipc.SimpleRpcServer$Listener(214): Listener,port=40285: starting 2024-04-09 16:54:06,509 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:06,526 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:06,526 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:40285 2024-04-09 16:54:06,531 INFO [Reader=1,bindAddress=localhost,port=40285] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:35016, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:06,531 DEBUG [Reader=1,bindAddress=localhost,port=40285] ipc.SimpleRpcServer$Listener(326): Caught exception while reading: org.apache.hadoop.hbase.DoNotRetryIOException: Failing for test at org.apache.hadoop.hbase.ipc.TestBlockingIPC$TestFailingRpcServer$FailingConnection.processRequest(TestBlockingIPC.java:94) at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:444) at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.process(SimpleServerRpcConnection.java:286) at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.readAndProcess(SimpleServerRpcConnection.java:252) at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.doRead(SimpleRpcServer.java:319) at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:181) at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-09 16:54:06,534 DEBUG [BRPC Connection (86992864) to localhost:40285 from jenkins (attempt: 1)] ipc.BlockingRpcConnection(787): Closing connection after error in call null java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.hadoop.hbase.ipc.BlockingRpcConnection.readResponse(BlockingRpcConnection.java:708) at org.apache.hadoop.hbase.ipc.BlockingRpcConnection.run(BlockingRpcConnection.java:393) at java.lang.Thread.run(Thread.java:750) 2024-04-09 16:54:06,547 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:06,547 INFO [Time-limited test] ipc.AbstractTestIPC(324): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: java.io.EOFException: Call to address=localhost:40285 failed on local exception: java.io.EOFException 2024-04-09 16:54:06,547 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 40285 2024-04-09 16:54:06,561 INFO [Listener,port=40285] ipc.SimpleRpcServer$Listener(258): Listener,port=40285: stopping 2024-04-09 16:54:06,570 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:06,571 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:06,622 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testConnectionCloseWithOutstandingRPCs Thread=20 (was 17) Potentially hanging thread: Reader=0,bindAddress=localhost,port=40285 sun.misc.Unsafe.getInt(Native Method) sun.nio.ch.NativeObject.getInt(NativeObject.java:257) sun.nio.ch.EPollArrayWrapper.getDescriptor(EPollArrayWrapper.java:165) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:271) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:174) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Reader=1,bindAddress=localhost,port=40285 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:174) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer idle connection scanner for port 40285 java.util.TimerThread.mainLoop(Timer.java:528) java.util.TimerThread.run(Timer.java:505) - Thread LEAK? -, OpenFileDescriptor=297 (was 294) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=200 (was 200), AvailableMemoryMB=237 (was 274) 2024-04-09 16:54:06,655 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testBadPreambleHeader Thread=17, OpenFileDescriptor=294, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=200, AvailableMemoryMB=236 2024-04-09 16:54:06,657 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting 2024-04-09 16:54:06,693 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=35105 2024-04-09 16:54:06,699 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@215004f0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:06,711 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:06,744 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:06,746 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:06,747 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:35105 2024-04-09 16:54:06,748 INFO [Listener,port=35105] ipc.SimpleRpcServer$Listener(214): Listener,port=35105: starting 2024-04-09 16:54:06,751 WARN [Reader=1,bindAddress=localhost,port=35105] ipc.ServerRpcConnection(706): serverVersion=0, clientVersion=0, authMethod=-10, authName=unknown from 127.0.0.1:51700 2024-04-09 16:54:06,780 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:06,780 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 35105 2024-04-09 16:54:06,780 INFO [Listener,port=35105] ipc.SimpleRpcServer$Listener(258): Listener,port=35105: stopping 2024-04-09 16:54:06,792 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:06,792 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:06,820 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testBadPreambleHeader Thread=21 (was 17) Potentially hanging thread: Reader=0,bindAddress=localhost,port=35105 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:174) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Reader=2,bindAddress=localhost,port=35105 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:174) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Reader=1,bindAddress=localhost,port=35105 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:174) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer idle connection scanner for port 35105 java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) - Thread LEAK? -, OpenFileDescriptor=294 (was 294), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=200 (was 200), AvailableMemoryMB=269 (was 236) - AvailableMemoryMB LEAK? - 2024-04-09 16:54:06,863 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testCompressCellBlock Thread=17, OpenFileDescriptor=296, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=200, AvailableMemoryMB=272 2024-04-09 16:54:06,886 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:06,893 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=39647 2024-04-09 16:54:06,910 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1822afc0, compressor=org.apache.hadoop.io.compress.GzipCodec@caaa566, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:06,913 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:06,915 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:06,916 INFO [Listener,port=39647] ipc.SimpleRpcServer$Listener(214): Listener,port=39647: starting 2024-04-09 16:54:06,913 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:06,994 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:39647 2024-04-09 16:54:07,049 INFO [Reader=1,bindAddress=localhost,port=39647] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:51226, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:07,160 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:07,160 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 39647 2024-04-09 16:54:07,161 INFO [Listener,port=39647] ipc.SimpleRpcServer$Listener(258): Listener,port=39647: stopping 2024-04-09 16:54:07,161 DEBUG [BRPC Connection (86992864) to localhost:39647 from jenkins (attempt: 1)] ipc.BlockingRpcConnection(369): Interrupted while waiting for work 2024-04-09 16:54:07,168 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:07,169 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:07,214 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testCompressCellBlock Thread=19 (was 17) Potentially hanging thread: Listener,port=39647 java.nio.channels.spi.AbstractSelector.cancel(AbstractSelector.java:92) java.nio.channels.spi.AbstractSelectionKey.cancel(AbstractSelectionKey.java:73) java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:247) java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:115) org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.close(SimpleServerRpcConnection.java:315) org.apache.hadoop.hbase.ipc.SimpleRpcServer$ConnectionManager.close(SimpleRpcServer.java:615) org.apache.hadoop.hbase.ipc.SimpleRpcServer$ConnectionManager.closeAll(SimpleRpcServer.java:647) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.run(SimpleRpcServer.java:272) Potentially hanging thread: Reader=1,bindAddress=localhost,port=39647 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:174) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=294 (was 296), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=197 (was 200), AvailableMemoryMB=245 (was 272) 2024-04-09 16:54:07,239 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testRpcServerForNotNullRemoteAddressInCallObject Thread=17, OpenFileDescriptor=294, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=197, AvailableMemoryMB=244 2024-04-09 16:54:07,239 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:07,258 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=37337 2024-04-09 16:54:07,266 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2cc0e168, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:07,285 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:07,297 INFO [Listener,port=37337] ipc.SimpleRpcServer$Listener(214): Listener,port=37337: starting 2024-04-09 16:54:07,293 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:07,306 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:07,306 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:37337 2024-04-09 16:54:07,310 INFO [Reader=1,bindAddress=localhost,port=37337] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:53028, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:07,322 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:07,323 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 37337 2024-04-09 16:54:07,329 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:07,333 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:07,329 INFO [Listener,port=37337] ipc.SimpleRpcServer$Listener(258): Listener,port=37337: stopping 2024-04-09 16:54:07,329 DEBUG [BRPC Connection (86992864) to localhost:37337 from jenkins (attempt: 1)] ipc.BlockingRpcConnection(369): Interrupted while waiting for work 2024-04-09 16:54:07,371 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testRpcServerForNotNullRemoteAddressInCallObject Thread=24 (was 17) Potentially hanging thread: Reader=1,bindAddress=localhost,port=37337 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209) java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285) java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1006) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.responder sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.doRunLoop(SimpleRpcServerResponder.java:112) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.run(SimpleRpcServerResponder.java:60) Potentially hanging thread: RpcServer idle connection scanner for port 37337 java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: Listener,port=37337 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.run(SimpleRpcServer.java:219) Potentially hanging thread: BRPC Connection (86992864) to localhost:37337 from jenkins (attempt: 1) java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.ipc.BlockingRpcConnection.waitForWork(BlockingRpcConnection.java:352) org.apache.hadoop.hbase.ipc.BlockingRpcConnection.run(BlockingRpcConnection.java:392) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Reader=0,bindAddress=localhost,port=37337 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209) java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285) java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1006) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=297 (was 294) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=197 (was 197), AvailableMemoryMB=243 (was 244) 2024-04-09 16:54:07,395 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testRemoteError Thread=17, OpenFileDescriptor=296, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=197, AvailableMemoryMB=242 2024-04-09 16:54:07,396 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:07,399 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=43737 2024-04-09 16:54:07,400 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d49d622, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:07,404 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:07,405 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:07,405 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:43737 2024-04-09 16:54:07,405 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:07,416 INFO [Listener,port=43737] ipc.SimpleRpcServer$Listener(214): Listener,port=43737: starting 2024-04-09 16:54:07,422 INFO [Reader=1,bindAddress=localhost,port=43737] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:36718, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:07,428 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.DoNotRetryIOException: server error! at org.apache.hadoop.hbase.ipc.TestProtobufRpcServiceImpl.error(TestProtobufRpcServiceImpl.java:104) at org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestRpcServiceProtos$TestProtobufRpcProto$2.callBlockingMethod(TestRpcServiceProtos.java:142) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:113) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-09 16:54:07,428 DEBUG [FifoRpcScheduler.handler-pool-0] ipc.CallRunner(144): callId: 0 service: TestProtobufRpcProto methodName: error size: 17 connection: 127.0.0.1:36718 deadline: 9223372036854775807, exception=org.apache.hadoop.hbase.DoNotRetryIOException: server error! 2024-04-09 16:54:07,433 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:07,434 INFO [Time-limited test] ipc.AbstractTestIPC(264): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): org.apache.hadoop.hbase.DoNotRetryIOException: server error! at org.apache.hadoop.hbase.ipc.TestProtobufRpcServiceImpl.error(TestProtobufRpcServiceImpl.java:104) at org.apache.hadoop.hbase.shaded.ipc.protobuf.generated.TestRpcServiceProtos$TestProtobufRpcProto$2.callBlockingMethod(TestRpcServiceProtos.java:142) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:113) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-09 16:54:07,434 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 43737 2024-04-09 16:54:07,434 INFO [Listener,port=43737] ipc.SimpleRpcServer$Listener(258): Listener,port=43737: stopping 2024-04-09 16:54:07,434 DEBUG [BRPC Connection (86992864) to localhost:43737 from jenkins (attempt: 1)] ipc.BlockingRpcConnection(369): Interrupted while waiting for work 2024-04-09 16:54:07,435 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:07,442 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:07,487 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testRemoteError Thread=20 (was 17) Potentially hanging thread: Reader=2,bindAddress=localhost,port=43737 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:174) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.responder org.apache.log4j.Category.callAppenders(Category.java:205) org.apache.log4j.Category.forcedLog(Category.java:384) org.apache.log4j.Category.log(Category.java:810) org.slf4j.impl.Reload4jLoggerAdapter.info(Reload4jLoggerAdapter.java:284) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.doRunLoop(SimpleRpcServerResponder.java:158) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.run(SimpleRpcServerResponder.java:60) Potentially hanging thread: BRPC Connection (86992864) to localhost:43737 from jenkins (attempt: 1) java.io.BufferedOutputStream.write(BufferedOutputStream.java:117) org.apache.maven.surefire.api.util.internal.Channels$4.writeImpl(Channels.java:199) org.apache.maven.surefire.api.util.internal.AbstractNoninterruptibleWritableChannel.write(AbstractNoninterruptibleWritableChannel.java:66) org.apache.maven.surefire.api.util.internal.AbstractNoninterruptibleWritableChannel.writeBuffered(AbstractNoninterruptibleWritableChannel.java:49) org.apache.maven.surefire.api.stream.AbstractStreamEncoder.write(AbstractStreamEncoder.java:69) org.apache.maven.surefire.booter.spi.EventChannelEncoder.write(EventChannelEncoder.java:288) org.apache.maven.surefire.booter.spi.EventChannelEncoder.setOutErr(EventChannelEncoder.java:173) org.apache.maven.surefire.booter.spi.EventChannelEncoder.testOutput(EventChannelEncoder.java:168) org.apache.maven.surefire.api.booter.ForkingRunListener.writeTestOutput(ForkingRunListener.java:99) org.apache.maven.surefire.api.booter.ForkingRunListener.writeTestOutput(ForkingRunListener.java:43) org.apache.maven.surefire.common.junit4.JUnit4RunListener.writeTestOutput(JUnit4RunListener.java:202) org.apache.maven.surefire.api.report.ConsoleOutputCapture$ForwardingPrintStream.write(ConsoleOutputCapture.java:57) sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295) sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141) java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229) org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:54) org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:317) org.apache.log4j.WriterAppender.append(WriterAppender.java:156) org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:232) org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:57) org.apache.log4j.Category.callAppenders(Category.java:206) org.apache.log4j.Category.forcedLog(Category.java:384) org.apache.log4j.Category.log(Category.java:810) org.slf4j.impl.Reload4jLoggerAdapter.debug(Reload4jLoggerAdapter.java:189) org.apache.hadoop.hbase.ipc.BlockingRpcConnection.waitForWork(BlockingRpcConnection.java:369) org.apache.hadoop.hbase.ipc.BlockingRpcConnection.run(BlockingRpcConnection.java:392) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=294 (was 296), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=197 (was 197), AvailableMemoryMB=239 (was 242) 2024-04-09 16:54:07,512 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testNoCodec Thread=17, OpenFileDescriptor=296, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=197, AvailableMemoryMB=237 2024-04-09 16:54:07,557 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:07,567 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=39363 2024-04-09 16:54:07,574 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=null, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:07,582 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:07,597 INFO [Listener,port=39363] ipc.SimpleRpcServer$Listener(214): Listener,port=39363: starting 2024-04-09 16:54:07,593 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:07,605 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:07,605 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:39363 2024-04-09 16:54:07,627 INFO [Reader=1,bindAddress=localhost,port=39363] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:40620, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:07,635 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:07,636 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 39363 2024-04-09 16:54:07,637 DEBUG [BRPC Connection (86992864) to localhost:39363 from jenkins (attempt: 1)] ipc.BlockingRpcConnection(369): Interrupted while waiting for work 2024-04-09 16:54:07,637 INFO [Listener,port=39363] ipc.SimpleRpcServer$Listener(258): Listener,port=39363: stopping 2024-04-09 16:54:07,640 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:07,640 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:07,663 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testNoCodec Thread=22 (was 17) Potentially hanging thread: RpcServer idle connection scanner for port 39363 java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RpcServer.responder sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.doRunLoop(SimpleRpcServerResponder.java:112) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.run(SimpleRpcServerResponder.java:60) Potentially hanging thread: Listener,port=39363 org.apache.log4j.Category.callAppenders(Category.java:205) org.apache.log4j.Category.forcedLog(Category.java:384) org.apache.log4j.Category.log(Category.java:810) org.slf4j.impl.Reload4jLoggerAdapter.info(Reload4jLoggerAdapter.java:284) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.run(SimpleRpcServer.java:258) Potentially hanging thread: Reader=2,bindAddress=localhost,port=39363 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:174) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=304 (was 296) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=197 (was 197), AvailableMemoryMB=1064 (was 237) - AvailableMemoryMB LEAK? - 2024-04-09 16:54:07,685 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testRpcMaxRequestSize Thread=17, OpenFileDescriptor=296, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=197, AvailableMemoryMB=1288 2024-04-09 16:54:07,687 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:07,692 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=44291 2024-04-09 16:54:07,699 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ecdcc16, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:07,703 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:07,703 INFO [Listener,port=44291] ipc.SimpleRpcServer$Listener(214): Listener,port=44291: starting 2024-04-09 16:54:07,709 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:07,722 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:07,723 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:44291 2024-04-09 16:54:07,724 INFO [Reader=1,bindAddress=localhost,port=44291] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:49006, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:07,741 WARN [Reader=1,bindAddress=localhost,port=44291] ipc.SimpleServerRpcConnection(187): RPC data length of 1260 received from 127.0.0.1 is greater than max allowed 1000. Set "hbase.ipc.max.request.size" on server to override this limit (not recommended) 2024-04-09 16:54:07,759 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:07,759 INFO [Time-limited test] ipc.AbstractTestIPC(225): Caught expected exception: org.apache.hbase.thirdparty.com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.RequestTooBigException): RPC data length of 1260 received from 127.0.0.1 is greater than max allowed 1000. Set "hbase.ipc.max.request.size" on server to override this limit (not recommended) 2024-04-09 16:54:07,759 DEBUG [BRPC Connection (86992864) to localhost:44291 from jenkins (attempt: 1)] ipc.BlockingRpcConnection(369): Interrupted while waiting for work 2024-04-09 16:54:07,761 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 44291 2024-04-09 16:54:07,765 INFO [Listener,port=44291] ipc.SimpleRpcServer$Listener(258): Listener,port=44291: stopping 2024-04-09 16:54:07,779 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:07,779 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:07,799 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testRpcMaxRequestSize Thread=21 (was 17) Potentially hanging thread: RpcServer.responder sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.doRunLoop(SimpleRpcServerResponder.java:112) org.apache.hadoop.hbase.ipc.SimpleRpcServerResponder.run(SimpleRpcServerResponder.java:60) Potentially hanging thread: RpcServer idle connection scanner for port 44291 java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: Reader=0,bindAddress=localhost,port=44291 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:174) org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Listener,port=44291 org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.run(SimpleRpcServer.java:261) - Thread LEAK? -, OpenFileDescriptor=304 (was 296) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1535 (was 1535), ProcessCount=197 (was 197), AvailableMemoryMB=1370 (was 1288) - AvailableMemoryMB LEAK? - 2024-04-09 16:54:07,817 INFO [Time-limited test] hbase.ResourceChecker(147): before: ipc.TestBlockingIPC#testAsyncTimeout Thread=17, OpenFileDescriptor=294, MaxFileDescriptor=60000, SystemLoadAverage=1535, ProcessCount=197, AvailableMemoryMB=1367 2024-04-09 16:54:07,817 INFO [Time-limited test] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.SimpleRpcServer hosting hbase.test.pb.TestProtobufRpcProto 2024-04-09 16:54:07,835 INFO [Time-limited test] ipc.SimpleRpcServer$Listener(134): testRpcServer: started 3 reader(s) listening on port=43379 2024-04-09 16:54:07,838 DEBUG [Time-limited test] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b4fd899, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2024-04-09 16:54:07,849 DEBUG [RpcServer.responder] ipc.SimpleRpcServerResponder(58): RpcServer.responder: starting 2024-04-09 16:54:07,858 INFO [Listener,port=43379] ipc.SimpleRpcServer$Listener(214): Listener,port=43379: starting 2024-04-09 16:54:07,858 INFO [Time-limited test] ipc.FifoRpcScheduler(61): Using FifoRpcScheduler as user call queue; handlerCount=1; maxQueueLength=10 2024-04-09 16:54:07,860 DEBUG [Time-limited test] ipc.RpcConnection(122): Using SIMPLE authentication for service=TestProtobufRpcProto, sasl=false 2024-04-09 16:54:07,860 DEBUG [Time-limited test] ipc.BlockingRpcConnection(501): Connecting to localhost:43379 2024-04-09 16:54:07,866 INFO [Reader=1,bindAddress=localhost,port=43379] ipc.ServerRpcConnection(540): Connection from 127.0.0.1:48210, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=TestProtobufRpcProto 2024-04-09 16:54:07,971 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=0,methodName=pause], waitTime=110ms, rpcTimeout=100ms 2024-04-09 16:54:07,971 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=1,methodName=pause], waitTime=106ms, rpcTimeout=100ms 2024-04-09 16:54:07,971 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=2,methodName=pause], waitTime=107ms, rpcTimeout=100ms 2024-04-09 16:54:07,977 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=3,methodName=pause], waitTime=107ms, rpcTimeout=100ms 2024-04-09 16:54:07,978 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=4,methodName=pause], waitTime=106ms, rpcTimeout=100ms 2024-04-09 16:54:07,979 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=5,methodName=pause], waitTime=106ms, rpcTimeout=100ms 2024-04-09 16:54:07,979 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=6,methodName=pause], waitTime=106ms, rpcTimeout=100ms 2024-04-09 16:54:07,979 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7,methodName=pause], waitTime=106ms, rpcTimeout=100ms 2024-04-09 16:54:07,979 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8,methodName=pause], waitTime=107ms, rpcTimeout=100ms 2024-04-09 16:54:07,985 INFO [Time-limited test] ipc.AbstractTestIPC(411): Caught expected exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=localhost:43379 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=9,methodName=pause], waitTime=107ms, rpcTimeout=100ms 2024-04-09 16:54:07,986 DEBUG [Time-limited test] ipc.AbstractRpcClient(489): Stopping rpc client 2024-04-09 16:54:07,986 INFO [Time-limited test] ipc.SimpleRpcServer(438): Stopping server on 43379 2024-04-09 16:54:07,986 INFO [Listener,port=43379] ipc.SimpleRpcServer$Listener(258): Listener,port=43379: stopping 2024-04-09 16:54:07,996 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(158): RpcServer.responder: stopped 2024-04-09 16:54:07,996 INFO [RpcServer.responder] ipc.SimpleRpcServerResponder(62): RpcServer.responder: stopping 2024-04-09 16:54:07,996 DEBUG [Reader=1,bindAddress=localhost,port=43379] ipc.SimpleRpcServer$Listener(326): Caught exception while reading: java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:406) at org.apache.hadoop.hbase.ipc.RpcServer.channelRead(RpcServer.java:611) at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.read4Bytes(SimpleServerRpcConnection.java:139) at org.apache.hadoop.hbase.ipc.SimpleServerRpcConnection.readAndProcess(SimpleServerRpcConnection.java:161) at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener.doRead(SimpleRpcServer.java:319) at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.doRunLoop(SimpleRpcServer.java:181) at org.apache.hadoop.hbase.ipc.SimpleRpcServer$Listener$Reader.run(SimpleRpcServer.java:154) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-09 16:54:08,016 INFO [Time-limited test] hbase.ResourceChecker(175): after: ipc.TestBlockingIPC#testAsyncTimeout Thread=18 (was 17) - Thread LEAK? -, OpenFileDescriptor=294 (was 294), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1540 (was 1535) - SystemLoadAverage LEAK? -, ProcessCount=197 (was 197), AvailableMemoryMB=1340 (was 1367)