2024-04-12 03:53:16,922 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.region.TestMasterRegionCompaction timeout: 13 mins 2024-04-12 03:53:16,961 INFO [Time-limited test] hbase.ResourceChecker(147): before: master.region.TestMasterRegionCompaction#test Thread=9, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=170, AvailableMemoryMB=9214 2024-04-12 03:53:16,993 INFO [Time-limited test] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2024-04-12 03:53:16,993 INFO [Time-limited test] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2024-04-12 03:53:17,684 INFO [Time-limited test] region.MasterRegion(309): Create or load local region for table 'test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2024-04-12 03:53:17,800 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2024-04-12 03:53:18,293 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2024-04-12 03:53:18,574 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2024-04-12 03:53:18,574 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2024-04-12 03:53:18,577 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-04-12 03:53:18,585 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-04-12 03:53:18,591 INFO [Time-limited test] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2024-04-12 03:53:18,754 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2024-04-12 03:53:18,851 INFO [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2024-04-12 03:53:18,921 DEBUG [Time-limited test] util.ClassSize(228): Using Unsafe to estimate memory layout 2024-04-12 03:53:18,934 INFO [Time-limited test] regionserver.HRegion(7693): Creating {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local-tmp 2024-04-12 03:53:19,062 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-04-12 03:53:19,193 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-04-12 03:53:19,193 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:19,194 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:19,195 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 0 ms 2024-04-12 03:53:19,195 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:19,195 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:19,196 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-12 03:53:19,198 DEBUG [Time-limited test] region.MasterRegion(191): WALDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/WALs/localhost,12345,1712893997650 2024-04-12 03:53:19,236 INFO [Time-limited test] wal.AbstractFSWAL(492): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=localhost%2C12345%2C1712893997650, suffix=, logDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/WALs/localhost,12345,1712893997650, archiveDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/oldWALs, maxLogs=32 2024-04-12 03:53:19,336 INFO [Time-limited test] wal.AbstractFSWAL(809): New WAL /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/WALs/localhost,12345,1712893997650/localhost%2C12345%2C1712893997650.1712893999255 2024-04-12 03:53:19,336 DEBUG [Time-limited test] wal.AbstractFSWAL(890): Create new AsyncFSWAL writer with pipeline: [] 2024-04-12 03:53:19,337 DEBUG [Time-limited test] regionserver.HRegion(7854): Opening region: {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''} 2024-04-12 03:53:19,337 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-04-12 03:53:19,341 DEBUG [Time-limited test] regionserver.HRegion(7894): checking encryption for fb3251d5e68787855bc0679a918ea02e 2024-04-12 03:53:19,342 DEBUG [Time-limited test] regionserver.HRegion(7897): checking classloading for fb3251d5e68787855bc0679a918ea02e 2024-04-12 03:53:19,404 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region fb3251d5e68787855bc0679a918ea02e 2024-04-12 03:53:19,406 WARN [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(529): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1. DEBUG log level might have more details. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-12 03:53:19,407 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-12 03:53:19,437 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f1 2024-04-12 03:53:19,459 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-04-12 03:53:19,460 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f2 of region fb3251d5e68787855bc0679a918ea02e 2024-04-12 03:53:19,461 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(532): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2 java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-12 03:53:19,461 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-12 03:53:19,463 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f2 2024-04-12 03:53:19,463 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-04-12 03:53:19,468 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-04-12 03:53:19,469 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-04-12 03:53:19,489 DEBUG [Time-limited test] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table test:local descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2024-04-12 03:53:19,497 DEBUG [Time-limited test] regionserver.HRegion(1055): writing seq id for fb3251d5e68787855bc0679a918ea02e 2024-04-12 03:53:19,508 DEBUG [Time-limited test] wal.WALSplitUtil(408): Wrote file=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2024-04-12 03:53:19,509 INFO [Time-limited test] regionserver.HRegion(1072): Opened fb3251d5e68787855bc0679a918ea02e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72785479, jitterRate=0.08458815515041351}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2024-04-12 03:53:19,510 DEBUG [Time-limited test] regionserver.HRegion(965): Region open journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-12 03:53:19,512 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2024-04-12 03:53:19,535 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2024-04-12 03:53:19,536 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2024-04-12 03:53:19,542 DEBUG [Time-limited test] cleaner.HFileCleaner(242): Starting for large file=Thread[Time-limited test-HFileCleaner.large.0-1712893999538,5,FailOnTimeoutGroup] 2024-04-12 03:53:19,547 DEBUG [Time-limited test] cleaner.HFileCleaner(257): Starting for small files=Thread[Time-limited test-HFileCleaner.small.0-1712893999543,5,FailOnTimeoutGroup] 2024-04-12 03:53:19,547 INFO [Time-limited test] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS is enabled. 2024-04-12 03:53:19,550 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-12 03:53:19,618 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-04-12 03:53:19,698 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on fa5deebf91594b009c35fbd503ee0e4d 2024-04-12 03:53:19,767 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/fa5deebf91594b009c35fbd503ee0e4d 2024-04-12 03:53:19,843 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 006b3e3d974c4ec387973aa110feee5f 2024-04-12 03:53:19,847 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/006b3e3d974c4ec387973aa110feee5f 2024-04-12 03:53:19,850 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/fa5deebf91594b009c35fbd503ee0e4d as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/fa5deebf91594b009c35fbd503ee0e4d 2024-04-12 03:53:19,854 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/fa5deebf91594b009c35fbd503ee0e4d, entries=1, sequenceid=4, filesize=4.8 K 2024-04-12 03:53:19,856 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/006b3e3d974c4ec387973aa110feee5f as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/006b3e3d974c4ec387973aa110feee5f 2024-04-12 03:53:19,860 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/006b3e3d974c4ec387973aa110feee5f, entries=1, sequenceid=4, filesize=4.8 K 2024-04-12 03:53:19,861 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 243ms, sequenceid=4, compaction requested=false 2024-04-12 03:53:19,862 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-12 03:53:19,863 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-04-12 03:53:19,881 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 2c8bafa30d2f47fdaa1457ab303af044 2024-04-12 03:53:19,884 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/2c8bafa30d2f47fdaa1457ab303af044 2024-04-12 03:53:19,894 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 38ef99077d5147cc9eb7b595a4cd791b 2024-04-12 03:53:19,896 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/38ef99077d5147cc9eb7b595a4cd791b 2024-04-12 03:53:19,899 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/2c8bafa30d2f47fdaa1457ab303af044 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2c8bafa30d2f47fdaa1457ab303af044 2024-04-12 03:53:19,902 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2c8bafa30d2f47fdaa1457ab303af044, entries=1, sequenceid=8, filesize=4.8 K 2024-04-12 03:53:19,903 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/38ef99077d5147cc9eb7b595a4cd791b as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/38ef99077d5147cc9eb7b595a4cd791b 2024-04-12 03:53:19,906 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/38ef99077d5147cc9eb7b595a4cd791b, entries=1, sequenceid=8, filesize=4.8 K 2024-04-12 03:53:19,907 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 44ms, sequenceid=8, compaction requested=false 2024-04-12 03:53:19,908 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-12 03:53:19,909 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-04-12 03:53:19,921 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 9303af1d003746fc9ecfc48fd70fdfb1 2024-04-12 03:53:19,923 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/9303af1d003746fc9ecfc48fd70fdfb1 2024-04-12 03:53:19,933 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 075221aeab48459ea81391f5c022d5d9 2024-04-12 03:53:19,934 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/075221aeab48459ea81391f5c022d5d9 2024-04-12 03:53:19,937 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/9303af1d003746fc9ecfc48fd70fdfb1 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9303af1d003746fc9ecfc48fd70fdfb1 2024-04-12 03:53:19,939 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9303af1d003746fc9ecfc48fd70fdfb1, entries=1, sequenceid=12, filesize=4.8 K 2024-04-12 03:53:19,940 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/075221aeab48459ea81391f5c022d5d9 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/075221aeab48459ea81391f5c022d5d9 2024-04-12 03:53:19,942 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/075221aeab48459ea81391f5c022d5d9, entries=1, sequenceid=12, filesize=4.8 K 2024-04-12 03:53:19,943 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 34ms, sequenceid=12, compaction requested=true 2024-04-12 03:53:19,944 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-12 03:53:19,948 INFO [test:local-Flusher] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=31 B heapSize=632 B 2024-04-12 03:53:19,951 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-04-12 03:53:19,959 DEBUG [test:local-Flusher] hfile.HFile$WriterFactory(307): Unable to set drop behind on c7ac0f6196ee4a57bcc7b0b78322d635 2024-04-12 03:53:19,961 INFO [test:local-Flusher] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=16 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/c7ac0f6196ee4a57bcc7b0b78322d635 2024-04-12 03:53:19,964 DEBUG [test:local-Flusher] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/c7ac0f6196ee4a57bcc7b0b78322d635 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/c7ac0f6196ee4a57bcc7b0b78322d635 2024-04-12 03:53:19,967 INFO [test:local-Flusher] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/c7ac0f6196ee4a57bcc7b0b78322d635, entries=1, sequenceid=16, filesize=4.8 K 2024-04-12 03:53:19,968 INFO [test:local-Flusher] regionserver.HRegion(2948): Finished flush of dataSize ~31 B/31, heapSize ~360 B/360, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 19ms, sequenceid=16, compaction requested=true 2024-04-12 03:53:19,968 DEBUG [test:local-Flusher] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-12 03:53:19,970 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2024-04-12 03:53:19,972 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f1 is initiating major compaction (all files) 2024-04-12 03:53:19,973 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f1 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:19,973 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/fa5deebf91594b009c35fbd503ee0e4d, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2c8bafa30d2f47fdaa1457ab303af044, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9303af1d003746fc9ecfc48fd70fdfb1, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/c7ac0f6196ee4a57bcc7b0b78322d635] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=19 K 2024-04-12 03:53:19,973 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting fa5deebf91594b009c35fbd503ee0e4d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1712893999579 2024-04-12 03:53:19,974 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 2c8bafa30d2f47fdaa1457ab303af044, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1712893999863 2024-04-12 03:53:19,974 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 9303af1d003746fc9ecfc48fd70fdfb1, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1712893999908 2024-04-12 03:53:19,974 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting c7ac0f6196ee4a57bcc7b0b78322d635, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=16, earliestPutTs=1712893999947 2024-04-12 03:53:19,997 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on 9f1040999d93471e95311503f107a94d 2024-04-12 03:53:20,003 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/9f1040999d93471e95311503f107a94d as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9f1040999d93471e95311503f107a94d 2024-04-12 03:53:20,013 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 4 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f1 of fb3251d5e68787855bc0679a918ea02e into 9f1040999d93471e95311503f107a94d(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-04-12 03:53:20,013 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-12 03:53:20,013 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2024-04-12 03:53:20,013 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f2 is initiating major compaction (all files) 2024-04-12 03:53:20,014 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f2 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:20,014 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/006b3e3d974c4ec387973aa110feee5f, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/38ef99077d5147cc9eb7b595a4cd791b, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/075221aeab48459ea81391f5c022d5d9] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=14.3 K 2024-04-12 03:53:20,014 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 006b3e3d974c4ec387973aa110feee5f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1712893999579 2024-04-12 03:53:20,015 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 38ef99077d5147cc9eb7b595a4cd791b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1712893999863 2024-04-12 03:53:20,015 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 075221aeab48459ea81391f5c022d5d9, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1712893999908 2024-04-12 03:53:20,027 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on 35cf96393881466f86cd21023fd8b380 2024-04-12 03:53:20,032 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/35cf96393881466f86cd21023fd8b380 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/35cf96393881466f86cd21023fd8b380 2024-04-12 03:53:20,035 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 3 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f2 of fb3251d5e68787855bc0679a918ea02e into 35cf96393881466f86cd21023fd8b380(size=4.9 K), total size for store is 4.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-04-12 03:53:20,035 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-12 03:53:20,036 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/fa5deebf91594b009c35fbd503ee0e4d, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2c8bafa30d2f47fdaa1457ab303af044, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9303af1d003746fc9ecfc48fd70fdfb1, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/c7ac0f6196ee4a57bcc7b0b78322d635] to archive 2024-04-12 03:53:20,040 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-04-12 03:53:20,042 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/fa5deebf91594b009c35fbd503ee0e4d to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/fa5deebf91594b009c35fbd503ee0e4d 2024-04-12 03:53:20,043 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2c8bafa30d2f47fdaa1457ab303af044 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2c8bafa30d2f47fdaa1457ab303af044 2024-04-12 03:53:20,044 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9303af1d003746fc9ecfc48fd70fdfb1 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9303af1d003746fc9ecfc48fd70fdfb1 2024-04-12 03:53:20,044 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/c7ac0f6196ee4a57bcc7b0b78322d635 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/c7ac0f6196ee4a57bcc7b0b78322d635 2024-04-12 03:53:20,049 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(463): Failed to traverse and delete the path: file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive does not exist at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:451) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1804) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1849) at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:675) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:413) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$runCleaner$0(CleanerChore.java:236) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-12 03:53:20,050 INFO [TestMasterRegionCompaction.Chore.1] cleaner.CleanerChore(239): Failed to traverse and delete the dir: file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive does not exist at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.runCleaner(CleanerChore.java:237) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:210) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive does not exist at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:451) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1804) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1849) at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:675) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:413) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$runCleaner$0(CleanerChore.java:236) ... 3 more 2024-04-12 03:53:20,050 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-04-12 03:53:20,052 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-04-12 03:53:20,063 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/fa5deebf91594b009c35fbd503ee0e4d to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/fa5deebf91594b009c35fbd503ee0e4d$-masterlocalhfile-$ 2024-04-12 03:53:20,063 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9303af1d003746fc9ecfc48fd70fdfb1 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9303af1d003746fc9ecfc48fd70fdfb1$-masterlocalhfile-$ 2024-04-12 03:53:20,064 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/c7ac0f6196ee4a57bcc7b0b78322d635 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/c7ac0f6196ee4a57bcc7b0b78322d635$-masterlocalhfile-$ 2024-04-12 03:53:20,064 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2c8bafa30d2f47fdaa1457ab303af044 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2c8bafa30d2f47fdaa1457ab303af044$-masterlocalhfile-$ 2024-04-12 03:53:20,065 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/006b3e3d974c4ec387973aa110feee5f, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/38ef99077d5147cc9eb7b595a4cd791b, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/075221aeab48459ea81391f5c022d5d9] to archive 2024-04-12 03:53:20,065 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-04-12 03:53:20,066 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/006b3e3d974c4ec387973aa110feee5f to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/006b3e3d974c4ec387973aa110feee5f 2024-04-12 03:53:20,067 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/38ef99077d5147cc9eb7b595a4cd791b to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/38ef99077d5147cc9eb7b595a4cd791b 2024-04-12 03:53:20,068 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/075221aeab48459ea81391f5c022d5d9 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/075221aeab48459ea81391f5c022d5d9 2024-04-12 03:53:20,071 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/075221aeab48459ea81391f5c022d5d9 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/075221aeab48459ea81391f5c022d5d9$-masterlocalhfile-$ 2024-04-12 03:53:20,071 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/006b3e3d974c4ec387973aa110feee5f to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/006b3e3d974c4ec387973aa110feee5f$-masterlocalhfile-$ 2024-04-12 03:53:20,072 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/38ef99077d5147cc9eb7b595a4cd791b to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b450e48e-1a02-0276-a829-68ecce5c93e5/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/38ef99077d5147cc9eb7b595a4cd791b$-masterlocalhfile-$ 2024-04-12 03:53:20,558 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 11 ms. 2024-04-12 03:53:21,053 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 5 ms. 2024-04-12 03:53:21,553 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 5 ms. 2024-04-12 03:53:22,052 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-12 03:53:22,552 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-12 03:53:23,052 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-12 03:53:23,551 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-12 03:53:24,052 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-12 03:53:24,551 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-12 03:53:25,052 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-12 03:53:25,551 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-12 03:53:26,049 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-12 03:53:26,549 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-12 03:53:27,049 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-12 03:53:27,549 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-12 03:53:28,049 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-12 03:53:28,550 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-12 03:53:29,048 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-12 03:53:29,548 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-12 03:53:30,048 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-12 03:53:30,548 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-12 03:53:31,048 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-12 03:53:31,548 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-12 03:53:32,048 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-12 03:53:32,158 INFO [Time-limited test] region.MasterRegion(167): Closing local region {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, isAbort=true 2024-04-12 03:53:32,162 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-04-12 03:53:32,162 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:32,162 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:32,162 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 0 ms 2024-04-12 03:53:32,163 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:32,164 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-12 03:53:32,165 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-12 03:53:32,165 INFO [test:local-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2024-04-12 03:53:32,165 INFO [Time-limited test] hbase.ChoreService(369): Chore service for: TestMasterRegionCompaction had [ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS] on shutdown 2024-04-12 03:53:32,201 INFO [Time-limited test] hbase.ResourceChecker(175): after: master.region.TestMasterRegionCompaction#test Thread=16 (was 9) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Time-limited test-HFileCleaner.large.0-1712893999538 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:236) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Time-limited test-HFileCleaner.small.0-1712893999543 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:251) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=294 (was 290) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 286), ProcessCount=170 (was 170), AvailableMemoryMB=8312 (was 9214)