2024-03-21 12:54:12,234 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.region.TestMasterRegionCompaction timeout: 13 mins 2024-03-21 12:54:12,278 INFO [Time-limited test] hbase.ResourceChecker(147): before: master.region.TestMasterRegionCompaction#test Thread=9, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=925, ProcessCount=193, AvailableMemoryMB=1700 2024-03-21 12:54:12,320 INFO [Time-limited test] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2024-03-21 12:54:12,321 INFO [Time-limited test] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2024-03-21 12:54:12,989 INFO [Time-limited test] region.MasterRegion(309): Create or load local region for table 'test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2024-03-21 12:54:13,126 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2024-03-21 12:54:13,642 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2024-03-21 12:54:13,870 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2024-03-21 12:54:13,870 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2024-03-21 12:54:13,874 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-03-21 12:54:13,885 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-03-21 12:54:13,892 INFO [Time-limited test] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2024-03-21 12:54:14,092 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2024-03-21 12:54:14,203 INFO [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2024-03-21 12:54:14,272 DEBUG [Time-limited test] util.ClassSize(228): Using Unsafe to estimate memory layout 2024-03-21 12:54:14,285 INFO [Time-limited test] regionserver.HRegion(7693): Creating {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local-tmp 2024-03-21 12:54:14,421 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-03-21 12:54:14,545 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-03-21 12:54:14,545 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:14,545 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:14,547 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 0 ms 2024-03-21 12:54:14,547 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:14,547 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:14,548 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-21 12:54:14,556 DEBUG [Time-limited test] region.MasterRegion(191): WALDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/WALs/localhost,12345,1711025652945 2024-03-21 12:54:14,613 INFO [Time-limited test] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=localhost%2C12345%2C1711025652945, suffix=, logDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/WALs/localhost,12345,1711025652945, archiveDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/oldWALs, maxLogs=32 2024-03-21 12:54:14,747 INFO [Time-limited test] wal.AbstractFSWAL(806): New WAL /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/WALs/localhost,12345,1711025652945/localhost%2C12345%2C1711025652945.1711025654659 2024-03-21 12:54:14,753 DEBUG [Time-limited test] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [] 2024-03-21 12:54:14,754 DEBUG [Time-limited test] regionserver.HRegion(7854): Opening region: {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''} 2024-03-21 12:54:14,755 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-03-21 12:54:14,759 DEBUG [Time-limited test] regionserver.HRegion(7894): checking encryption for fb3251d5e68787855bc0679a918ea02e 2024-03-21 12:54:14,761 DEBUG [Time-limited test] regionserver.HRegion(7897): checking classloading for fb3251d5e68787855bc0679a918ea02e 2024-03-21 12:54:14,885 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region fb3251d5e68787855bc0679a918ea02e 2024-03-21 12:54:14,888 WARN [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(529): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1. DEBUG log level might have more details. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-21 12:54:14,889 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-21 12:54:14,938 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f1 2024-03-21 12:54:14,961 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-03-21 12:54:14,961 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f2 of region fb3251d5e68787855bc0679a918ea02e 2024-03-21 12:54:14,962 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(532): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2 java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-21 12:54:14,963 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-21 12:54:14,964 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f2 2024-03-21 12:54:14,965 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-03-21 12:54:14,974 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-03-21 12:54:14,976 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-03-21 12:54:14,996 DEBUG [Time-limited test] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table test:local descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2024-03-21 12:54:15,000 DEBUG [Time-limited test] regionserver.HRegion(1055): writing seq id for fb3251d5e68787855bc0679a918ea02e 2024-03-21 12:54:15,013 DEBUG [Time-limited test] wal.WALSplitUtil(408): Wrote file=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2024-03-21 12:54:15,014 INFO [Time-limited test] regionserver.HRegion(1072): Opened fb3251d5e68787855bc0679a918ea02e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67147995, jitterRate=5.830973386764526E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2024-03-21 12:54:15,014 DEBUG [Time-limited test] regionserver.HRegion(965): Region open journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-21 12:54:15,017 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2024-03-21 12:54:15,040 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2024-03-21 12:54:15,041 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2024-03-21 12:54:15,043 DEBUG [Time-limited test] cleaner.HFileCleaner(242): Starting for large file=Thread[Time-limited test-HFileCleaner.large.0-1711025655043,5,FailOnTimeoutGroup] 2024-03-21 12:54:15,045 DEBUG [Time-limited test] cleaner.HFileCleaner(257): Starting for small files=Thread[Time-limited test-HFileCleaner.small.0-1711025655044,5,FailOnTimeoutGroup] 2024-03-21 12:54:15,045 INFO [Time-limited test] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS is enabled. 2024-03-21 12:54:15,054 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-21 12:54:15,147 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-03-21 12:54:15,222 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on cab965faec424cee9c8d684560e8946c 2024-03-21 12:54:15,282 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/cab965faec424cee9c8d684560e8946c 2024-03-21 12:54:15,366 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 8df13ca59681475fa68554cb13d388af 2024-03-21 12:54:15,368 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/8df13ca59681475fa68554cb13d388af 2024-03-21 12:54:15,371 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/cab965faec424cee9c8d684560e8946c as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/cab965faec424cee9c8d684560e8946c 2024-03-21 12:54:15,374 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/cab965faec424cee9c8d684560e8946c, entries=1, sequenceid=4, filesize=4.8 K 2024-03-21 12:54:15,376 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/8df13ca59681475fa68554cb13d388af as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/8df13ca59681475fa68554cb13d388af 2024-03-21 12:54:15,379 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/8df13ca59681475fa68554cb13d388af, entries=1, sequenceid=4, filesize=4.8 K 2024-03-21 12:54:15,380 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 233ms, sequenceid=4, compaction requested=false 2024-03-21 12:54:15,381 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-21 12:54:15,382 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-03-21 12:54:15,390 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on aa8c5de0641c46428a4f624649bdf364 2024-03-21 12:54:15,392 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/aa8c5de0641c46428a4f624649bdf364 2024-03-21 12:54:15,402 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on aadf51586d454ff58d5836be4aaafcb4 2024-03-21 12:54:15,405 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/aadf51586d454ff58d5836be4aaafcb4 2024-03-21 12:54:15,407 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/aa8c5de0641c46428a4f624649bdf364 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/aa8c5de0641c46428a4f624649bdf364 2024-03-21 12:54:15,411 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/aa8c5de0641c46428a4f624649bdf364, entries=1, sequenceid=8, filesize=4.8 K 2024-03-21 12:54:15,412 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/aadf51586d454ff58d5836be4aaafcb4 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/aadf51586d454ff58d5836be4aaafcb4 2024-03-21 12:54:15,415 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/aadf51586d454ff58d5836be4aaafcb4, entries=1, sequenceid=8, filesize=4.8 K 2024-03-21 12:54:15,416 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 34ms, sequenceid=8, compaction requested=false 2024-03-21 12:54:15,416 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-21 12:54:15,417 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-03-21 12:54:15,428 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 3ee540bb5bdc4af8a92d97787fd36a5e 2024-03-21 12:54:15,430 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/3ee540bb5bdc4af8a92d97787fd36a5e 2024-03-21 12:54:15,436 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 1d373c4f023f4cf5ae2333ea7fff7e41 2024-03-21 12:54:15,438 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/1d373c4f023f4cf5ae2333ea7fff7e41 2024-03-21 12:54:15,441 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/3ee540bb5bdc4af8a92d97787fd36a5e as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3ee540bb5bdc4af8a92d97787fd36a5e 2024-03-21 12:54:15,443 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3ee540bb5bdc4af8a92d97787fd36a5e, entries=1, sequenceid=12, filesize=4.8 K 2024-03-21 12:54:15,444 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/1d373c4f023f4cf5ae2333ea7fff7e41 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/1d373c4f023f4cf5ae2333ea7fff7e41 2024-03-21 12:54:15,447 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/1d373c4f023f4cf5ae2333ea7fff7e41, entries=1, sequenceid=12, filesize=4.8 K 2024-03-21 12:54:15,448 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 31ms, sequenceid=12, compaction requested=true 2024-03-21 12:54:15,448 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-21 12:54:15,451 INFO [test:local-Flusher] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=31 B heapSize=632 B 2024-03-21 12:54:15,454 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-03-21 12:54:15,457 DEBUG [test:local-Flusher] hfile.HFile$WriterFactory(307): Unable to set drop behind on f3f01448f244409ea51e4c228a3d536a 2024-03-21 12:54:15,459 INFO [test:local-Flusher] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=16 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/f3f01448f244409ea51e4c228a3d536a 2024-03-21 12:54:15,461 DEBUG [test:local-Flusher] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/f3f01448f244409ea51e4c228a3d536a as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/f3f01448f244409ea51e4c228a3d536a 2024-03-21 12:54:15,464 INFO [test:local-Flusher] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/f3f01448f244409ea51e4c228a3d536a, entries=1, sequenceid=16, filesize=4.8 K 2024-03-21 12:54:15,465 INFO [test:local-Flusher] regionserver.HRegion(2948): Finished flush of dataSize ~31 B/31, heapSize ~360 B/360, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 14ms, sequenceid=16, compaction requested=true 2024-03-21 12:54:15,465 DEBUG [test:local-Flusher] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-21 12:54:15,468 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2024-03-21 12:54:15,470 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f1 is initiating major compaction (all files) 2024-03-21 12:54:15,471 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f1 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:15,471 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/cab965faec424cee9c8d684560e8946c, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/aa8c5de0641c46428a4f624649bdf364, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3ee540bb5bdc4af8a92d97787fd36a5e, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/f3f01448f244409ea51e4c228a3d536a] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=19 K 2024-03-21 12:54:15,472 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting cab965faec424cee9c8d684560e8946c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1711025655087 2024-03-21 12:54:15,472 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting aa8c5de0641c46428a4f624649bdf364, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1711025655382 2024-03-21 12:54:15,473 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 3ee540bb5bdc4af8a92d97787fd36a5e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1711025655416 2024-03-21 12:54:15,473 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting f3f01448f244409ea51e4c228a3d536a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=16, earliestPutTs=1711025655451 2024-03-21 12:54:15,492 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on e89ed82d46ad43bebb14ef7d1acb1e8e 2024-03-21 12:54:15,499 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/e89ed82d46ad43bebb14ef7d1acb1e8e as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/e89ed82d46ad43bebb14ef7d1acb1e8e 2024-03-21 12:54:15,512 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 4 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f1 of fb3251d5e68787855bc0679a918ea02e into e89ed82d46ad43bebb14ef7d1acb1e8e(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-03-21 12:54:15,512 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-21 12:54:15,512 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2024-03-21 12:54:15,512 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f2 is initiating major compaction (all files) 2024-03-21 12:54:15,513 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f2 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:15,513 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/8df13ca59681475fa68554cb13d388af, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/aadf51586d454ff58d5836be4aaafcb4, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/1d373c4f023f4cf5ae2333ea7fff7e41] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=14.3 K 2024-03-21 12:54:15,513 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 8df13ca59681475fa68554cb13d388af, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1711025655087 2024-03-21 12:54:15,514 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting aadf51586d454ff58d5836be4aaafcb4, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1711025655382 2024-03-21 12:54:15,514 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 1d373c4f023f4cf5ae2333ea7fff7e41, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1711025655416 2024-03-21 12:54:15,522 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on 4a9222dc18cd468986a33fbffeab3df7 2024-03-21 12:54:15,527 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/4a9222dc18cd468986a33fbffeab3df7 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/4a9222dc18cd468986a33fbffeab3df7 2024-03-21 12:54:15,530 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 3 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f2 of fb3251d5e68787855bc0679a918ea02e into 4a9222dc18cd468986a33fbffeab3df7(size=4.9 K), total size for store is 4.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-03-21 12:54:15,530 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-21 12:54:15,531 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/cab965faec424cee9c8d684560e8946c, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/aa8c5de0641c46428a4f624649bdf364, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3ee540bb5bdc4af8a92d97787fd36a5e, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/f3f01448f244409ea51e4c228a3d536a] to archive 2024-03-21 12:54:15,533 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-03-21 12:54:15,535 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/cab965faec424cee9c8d684560e8946c to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/cab965faec424cee9c8d684560e8946c 2024-03-21 12:54:15,535 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/aa8c5de0641c46428a4f624649bdf364 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/aa8c5de0641c46428a4f624649bdf364 2024-03-21 12:54:15,536 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3ee540bb5bdc4af8a92d97787fd36a5e to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3ee540bb5bdc4af8a92d97787fd36a5e 2024-03-21 12:54:15,537 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/f3f01448f244409ea51e4c228a3d536a to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/f3f01448f244409ea51e4c228a3d536a 2024-03-21 12:54:15,547 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(463): Failed to traverse and delete the path: file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive does not exist at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:451) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1804) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1849) at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:675) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:413) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$runCleaner$0(CleanerChore.java:236) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-21 12:54:15,547 INFO [TestMasterRegionCompaction.Chore.1] cleaner.CleanerChore(239): Failed to traverse and delete the dir: file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive does not exist at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.runCleaner(CleanerChore.java:237) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:210) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive does not exist at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:451) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1804) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1849) at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:675) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:413) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$runCleaner$0(CleanerChore.java:236) ... 3 more 2024-03-21 12:54:15,547 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-03-21 12:54:15,555 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-03-21 12:54:15,562 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3ee540bb5bdc4af8a92d97787fd36a5e to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3ee540bb5bdc4af8a92d97787fd36a5e$-masterlocalhfile-$ 2024-03-21 12:54:15,563 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/cab965faec424cee9c8d684560e8946c to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/cab965faec424cee9c8d684560e8946c$-masterlocalhfile-$ 2024-03-21 12:54:15,564 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/aa8c5de0641c46428a4f624649bdf364 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/aa8c5de0641c46428a4f624649bdf364$-masterlocalhfile-$ 2024-03-21 12:54:15,565 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/f3f01448f244409ea51e4c228a3d536a to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/f3f01448f244409ea51e4c228a3d536a$-masterlocalhfile-$ 2024-03-21 12:54:15,565 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/8df13ca59681475fa68554cb13d388af, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/aadf51586d454ff58d5836be4aaafcb4, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/1d373c4f023f4cf5ae2333ea7fff7e41] to archive 2024-03-21 12:54:15,566 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-03-21 12:54:15,567 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/8df13ca59681475fa68554cb13d388af to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/8df13ca59681475fa68554cb13d388af 2024-03-21 12:54:15,568 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/aadf51586d454ff58d5836be4aaafcb4 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/aadf51586d454ff58d5836be4aaafcb4 2024-03-21 12:54:15,569 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/1d373c4f023f4cf5ae2333ea7fff7e41 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/1d373c4f023f4cf5ae2333ea7fff7e41 2024-03-21 12:54:15,571 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/1d373c4f023f4cf5ae2333ea7fff7e41 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/1d373c4f023f4cf5ae2333ea7fff7e41$-masterlocalhfile-$ 2024-03-21 12:54:15,572 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/8df13ca59681475fa68554cb13d388af to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/8df13ca59681475fa68554cb13d388af$-masterlocalhfile-$ 2024-03-21 12:54:15,573 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/aadf51586d454ff58d5836be4aaafcb4 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/ccbce623-c8fc-69bd-cd0a-9953888a4bd6/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/aadf51586d454ff58d5836be4aaafcb4$-masterlocalhfile-$ 2024-03-21 12:54:16,059 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 14 ms. 2024-03-21 12:54:16,549 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-03-21 12:54:17,049 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-03-21 12:54:17,549 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-03-21 12:54:18,051 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 5 ms. 2024-03-21 12:54:18,549 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-03-21 12:54:19,111 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 66 ms. 2024-03-21 12:54:19,549 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-03-21 12:54:20,049 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-03-21 12:54:20,550 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 5 ms. 2024-03-21 12:54:21,051 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 5 ms. 2024-03-21 12:54:21,547 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-03-21 12:54:22,047 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-03-21 12:54:22,548 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-03-21 12:54:23,047 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-03-21 12:54:23,548 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-03-21 12:54:24,051 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 5 ms. 2024-03-21 12:54:24,546 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-21 12:54:25,046 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-21 12:54:25,546 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-21 12:54:26,046 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-21 12:54:26,546 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-21 12:54:27,045 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-21 12:54:27,546 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-21 12:54:27,662 INFO [Time-limited test] region.MasterRegion(167): Closing local region {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, isAbort=true 2024-03-21 12:54:27,666 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-03-21 12:54:27,666 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:27,667 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:27,667 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 0 ms 2024-03-21 12:54:27,667 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:27,670 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-21 12:54:27,670 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-21 12:54:27,670 INFO [test:local-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2024-03-21 12:54:27,670 INFO [Time-limited test] hbase.ChoreService(369): Chore service for: TestMasterRegionCompaction had [ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS] on shutdown 2024-03-21 12:54:27,713 INFO [Time-limited test] hbase.ResourceChecker(175): after: master.region.TestMasterRegionCompaction#test Thread=16 (was 9) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Time-limited test-HFileCleaner.large.0-1711025655043 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:236) Potentially hanging thread: Time-limited test-HFileCleaner.small.0-1711025655044 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:251) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=294 (was 290) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=894 (was 925), ProcessCount=196 (was 193) - ProcessCount LEAK? -, AvailableMemoryMB=684 (was 1700)