[2023-02-08T07:33:34,325][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-master-0] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sdc)]], net usable_space [8.7gb], net total_space [9.7gb], types [ext4]
[2023-02-08T07:33:34,327][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-master-0] heap size [2gb], compressed ordinary object pointers [true]
[2023-02-08T07:33:34,491][INFO ][o.e.n.Node               ] [elasticsearch-master-0] node name [elasticsearch-master-0], node ID [gGzMFXxjQVObiSjShXBCNA]
[2023-02-08T07:33:34,492][INFO ][o.e.n.Node               ] [elasticsearch-master-0] version[6.8.23], pid[1], build[default/docker/4f67856/2022-01-06T21:30:50.087716Z], OS[Linux/5.10.147+/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2023-02-08T07:33:34,492][INFO ][o.e.n.Node               ] [elasticsearch-master-0] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.io.tmpdir=/tmp/elasticsearch-5613165729211612296, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Dlog4j2.formatMsgNoLookups=true, -Des.cgroups.hierarchy.override=/, -Xms2g, -Xmx2g, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [aggs-matrix-stats]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [analysis-common]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [ingest-common]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [ingest-geoip]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [ingest-user-agent]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [lang-expression]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [lang-mustache]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [lang-painless]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [mapper-extras]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [parent-join]
[2023-02-08T07:33:36,167][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [percolator]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [rank-eval]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [reindex]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [repository-url]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [transport-netty4]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [tribe]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-ccr]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-core]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-deprecation]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-graph]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-ilm]
[2023-02-08T07:33:36,168][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-logstash]
[2023-02-08T07:33:36,169][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-ml]
[2023-02-08T07:33:36,169][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-monitoring]
[2023-02-08T07:33:36,169][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-rollup]
[2023-02-08T07:33:36,169][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-security]
[2023-02-08T07:33:36,169][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-sql]
[2023-02-08T07:33:36,169][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-upgrade]
[2023-02-08T07:33:36,169][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-watcher]
[2023-02-08T07:33:36,170][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded plugin [experimental-highlighter]
[2023-02-08T07:33:36,170][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded plugin [extra]
2023-02-08 07:34:29
Full thread dump OpenJDK 64-Bit Server VM (15.0.1+9 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007f203e46bbc0, length=14, elements={
0x00007f203c02a7c0, 0x00007f203c61bcb0, 0x00007f203c61d4d0, 0x00007f203c6222d0,
0x00007f203c623ac0, 0x00007f203c625840, 0x00007f203c627190, 0x00007f203c6289b0,
0x00007f203c665e20, 0x00007f203c6699d0, 0x00007f203ce7acb0, 0x00007f203cf959e0,
0x00007f203db1f510, 0x00007f203e46aa20
}

"main" #1 prio=5 os_prio=0 cpu=54730.17ms elapsed=57.13s tid=0x00007f203c02a7c0 nid=0x4d runnable  [0x00007f204634f000]
   java.lang.Thread.State: RUNNABLE
	at java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$UnmodifiableEntry.getKey(java.base@15.0.1/Collections.java:1767)
	at org.elasticsearch.common.settings.Settings$FilteredMap$1$1.hasNext(Settings.java:1276)
	at java.util.AbstractMap.putAll(java.base@15.0.1/AbstractMap.java:280)
	at java.util.TreeMap.putAll(java.base@15.0.1/TreeMap.java:326)
	at java.util.TreeMap.<init>(java.base@15.0.1/TreeMap.java:187)
	at org.elasticsearch.common.settings.Settings.<init>(Settings.java:103)
	at org.elasticsearch.common.settings.Settings.getByPrefix(Settings.java:209)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:890)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:859)
	at org.elasticsearch.common.settings.AbstractScopedSettings.archiveUnknownOrInvalidSettings(AbstractScopedSettings.java:885)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.archiveBrokenIndexSettings(MetaDataIndexUpgradeService.java:220)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:95)
	at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:246)
	at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:89)
	at org.elasticsearch.node.Node.<init>(Node.java:499)
	at org.elasticsearch.node.Node.<init>(Node.java:266)
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
	at org.elasticsearch.cli.Command.main(Command.java:90)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=2.12ms elapsed=56.51s tid=0x00007f203c61bcb0 nid=0x55 waiting on condition  [0x00007f200b577000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@15.0.1/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@15.0.1/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@15.0.1/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=5.49ms elapsed=56.51s tid=0x00007f203c61d4d0 nid=0x56 in Object.wait()  [0x00007f200b476000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@15.0.1/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.21ms elapsed=56.50s tid=0x00007f203c6222d0 nid=0x57 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=10.05ms elapsed=56.50s tid=0x00007f203c623ac0 nid=0x58 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=6903.17ms elapsed=56.50s tid=0x00007f203c625840 nid=0x59 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=1933.42ms elapsed=56.50s tid=0x00007f203c627190 nid=0x5a waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #8 daemon prio=9 os_prio=0 cpu=5.99ms elapsed=56.50s tid=0x00007f203c6289b0 nid=0x5b runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Notification Thread" #9 daemon prio=9 os_prio=0 cpu=0.09ms elapsed=56.49s tid=0x00007f203c665e20 nid=0x5c runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #10 daemon prio=8 os_prio=0 cpu=2.73ms elapsed=56.49s tid=0x00007f203c6699d0 nid=0x5e in Object.wait()  [0x00007f200ac6d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@15.0.1/CleanerImpl.java:148)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)
	at jdk.internal.misc.InnocuousThread.run(java.base@15.0.1/InnocuousThread.java:134)

"process reaper" #15 daemon prio=10 os_prio=0 cpu=0.22ms elapsed=55.64s tid=0x00007f203ce7acb0 nid=0x61 runnable  [0x00007f2020029000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ProcessHandleImpl.waitForProcessExit0(java.base@15.0.1/Native Method)
	at java.lang.ProcessHandleImpl$1.run(java.base@15.0.1/ProcessHandleImpl.java:142)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"process reaper" #17 daemon prio=10 os_prio=0 cpu=0.96ms elapsed=55.61s tid=0x00007f203cf959e0 nid=0x64 waiting on condition  [0x00007f2009d1a000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@15.0.1/Native Method)
	- parking to wait for  <0x00000000801013b8> (a java.util.concurrent.SynchronousQueue$TransferStack)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@15.0.1/LockSupport.java:252)
	at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@15.0.1/SynchronousQueue.java:462)
	at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@15.0.1/SynchronousQueue.java:361)
	at java.util.concurrent.SynchronousQueue.poll(java.base@15.0.1/SynchronousQueue.java:937)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@15.0.1/ThreadPoolExecutor.java:1055)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1116)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"elasticsearch[elasticsearch-master-0][[timer]]" #19 daemon prio=5 os_prio=0 cpu=12.46ms elapsed=53.30s tid=0x00007f203db1f510 nid=0x65 waiting on condition  [0x00007f2009b5c000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(java.base@15.0.1/Native Method)
	at org.elasticsearch.threadpool.ThreadPool$CachedTimeThread.run(ThreadPool.java:589)

"elasticsearch[elasticsearch-master-0][scheduler][T#1]" #20 daemon prio=5 os_prio=0 cpu=5.08ms elapsed=50.73s tid=0x00007f203e46aa20 nid=0x66 waiting on condition  [0x00007f2009cf8000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@15.0.1/Native Method)
	- parking to wait for  <0x0000000080712928> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@15.0.1/LockSupport.java:252)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@15.0.1/AbstractQueuedSynchronizer.java:1661)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:1182)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@15.0.1/ThreadPoolExecutor.java:1056)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1116)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"VM Thread" os_prio=0 cpu=48.01ms elapsed=56.51s tid=0x00007f203c618f70 nid=0x54 runnable  

"GC Thread#0" os_prio=0 cpu=1015.64ms elapsed=57.12s tid=0x00007f203c068610 nid=0x4e runnable  

"GC Thread#1" os_prio=0 cpu=1019.13ms elapsed=57.12s tid=0x00007f203c085030 nid=0x51 runnable  

"G1 Main Marker" os_prio=0 cpu=2.08ms elapsed=57.12s tid=0x00007f203c07a080 nid=0x4f runnable  

"G1 Conc#0" os_prio=0 cpu=176.48ms elapsed=57.12s tid=0x00007f203c07b430 nid=0x50 runnable  

"G1 Refine#0" os_prio=0 cpu=12.06ms elapsed=56.52s tid=0x00007f203c5e6330 nid=0x52 runnable  

"G1 Refine#1" os_prio=0 cpu=0.06ms elapsed=50.11s tid=0x00007f201000b0e0 nid=0x67 runnable  

"G1 Young RemSet Sampling" os_prio=0 cpu=33.08ms elapsed=56.52s tid=0x00007f203c5e7660 nid=0x53 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=33.54ms elapsed=56.49s tid=0x00007f203c667b00 nid=0x5d waiting on condition  

JNI global refs: 30, weak refs: 45

Heap
 garbage-first heap   total 2097152K, used 505864K [0x0000000080000000, 0x0000000100000000)
  region size 1024K, 364 young (372736K), 1 survivors (1024K)
 Metaspace       used 48857K, capacity 50760K, committed 51024K, reserved 1093632K
  class space    used 6474K, capacity 7107K, committed 7168K, reserved 1048576K

2023-02-08 07:34:55
Full thread dump OpenJDK 64-Bit Server VM (15.0.1+9 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007f1fcc000d40, length=13, elements={
0x00007f203c02a7c0, 0x00007f203c61bcb0, 0x00007f203c61d4d0, 0x00007f203c6222d0,
0x00007f203c623ac0, 0x00007f203c625840, 0x00007f203c627190, 0x00007f203c6289b0,
0x00007f203c665e20, 0x00007f203c6699d0, 0x00007f203ce7acb0, 0x00007f203db1f510,
0x00007f203e46aa20
}

"main" #1 prio=5 os_prio=0 cpu=80409.10ms elapsed=82.85s tid=0x00007f203c02a7c0 nid=0x4d runnable  [0x00007f204634f000]
   java.lang.Thread.State: RUNNABLE
	at java.util.TreeMap.fixAfterInsertion(java.base@15.0.1/TreeMap.java:2574)
	at java.util.TreeMap.addEntry(java.base@15.0.1/TreeMap.java:762)
	at java.util.TreeMap.put(java.base@15.0.1/TreeMap.java:820)
	at java.util.TreeMap.put(java.base@15.0.1/TreeMap.java:534)
	at java.util.AbstractMap.putAll(java.base@15.0.1/AbstractMap.java:281)
	at java.util.TreeMap.putAll(java.base@15.0.1/TreeMap.java:326)
	at java.util.TreeMap.<init>(java.base@15.0.1/TreeMap.java:187)
	at org.elasticsearch.common.settings.Settings.<init>(Settings.java:103)
	at org.elasticsearch.common.settings.Settings.getByPrefix(Settings.java:209)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:890)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:859)
	at org.elasticsearch.common.settings.AbstractScopedSettings.archiveUnknownOrInvalidSettings(AbstractScopedSettings.java:885)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.archiveBrokenIndexSettings(MetaDataIndexUpgradeService.java:220)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:95)
	at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:246)
	at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:89)
	at org.elasticsearch.node.Node.<init>(Node.java:499)
	at org.elasticsearch.node.Node.<init>(Node.java:266)
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
	at org.elasticsearch.cli.Command.main(Command.java:90)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=2.16ms elapsed=82.23s tid=0x00007f203c61bcb0 nid=0x55 waiting on condition  [0x00007f200b577000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@15.0.1/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@15.0.1/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@15.0.1/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=5.49ms elapsed=82.23s tid=0x00007f203c61d4d0 nid=0x56 in Object.wait()  [0x00007f200b476000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@15.0.1/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.35ms elapsed=82.22s tid=0x00007f203c6222d0 nid=0x57 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=11.38ms elapsed=82.22s tid=0x00007f203c623ac0 nid=0x58 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=6908.67ms elapsed=82.22s tid=0x00007f203c625840 nid=0x59 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=1938.47ms elapsed=82.22s tid=0x00007f203c627190 nid=0x5a waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #8 daemon prio=9 os_prio=0 cpu=5.99ms elapsed=82.22s tid=0x00007f203c6289b0 nid=0x5b runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Notification Thread" #9 daemon prio=9 os_prio=0 cpu=0.09ms elapsed=82.21s tid=0x00007f203c665e20 nid=0x5c runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #10 daemon prio=8 os_prio=0 cpu=2.79ms elapsed=82.21s tid=0x00007f203c6699d0 nid=0x5e in Object.wait()  [0x00007f200ac6d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@15.0.1/CleanerImpl.java:148)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)
	at jdk.internal.misc.InnocuousThread.run(java.base@15.0.1/InnocuousThread.java:134)

"process reaper" #15 daemon prio=10 os_prio=0 cpu=0.22ms elapsed=81.36s tid=0x00007f203ce7acb0 nid=0x61 runnable  [0x00007f2020029000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ProcessHandleImpl.waitForProcessExit0(java.base@15.0.1/Native Method)
	at java.lang.ProcessHandleImpl$1.run(java.base@15.0.1/ProcessHandleImpl.java:142)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"elasticsearch[elasticsearch-master-0][[timer]]" #19 daemon prio=5 os_prio=0 cpu=17.65ms elapsed=79.02s tid=0x00007f203db1f510 nid=0x65 waiting on condition  [0x00007f2009b5c000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(java.base@15.0.1/Native Method)
	at org.elasticsearch.threadpool.ThreadPool$CachedTimeThread.run(ThreadPool.java:589)

"elasticsearch[elasticsearch-master-0][scheduler][T#1]" #20 daemon prio=5 os_prio=0 cpu=6.28ms elapsed=76.45s tid=0x00007f203e46aa20 nid=0x66 waiting on condition  [0x00007f2009cf8000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@15.0.1/Native Method)
	- parking to wait for  <0x0000000080712928> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@15.0.1/LockSupport.java:252)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@15.0.1/AbstractQueuedSynchronizer.java:1661)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:1182)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@15.0.1/ThreadPoolExecutor.java:1056)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1116)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"VM Thread" os_prio=0 cpu=59.34ms elapsed=82.23s tid=0x00007f203c618f70 nid=0x54 runnable  

"GC Thread#0" os_prio=0 cpu=1038.51ms elapsed=82.84s tid=0x00007f203c068610 nid=0x4e runnable  

"GC Thread#1" os_prio=0 cpu=1042.27ms elapsed=82.84s tid=0x00007f203c085030 nid=0x51 runnable  

"G1 Main Marker" os_prio=0 cpu=2.08ms elapsed=82.84s tid=0x00007f203c07a080 nid=0x4f runnable  

"G1 Conc#0" os_prio=0 cpu=176.48ms elapsed=82.84s tid=0x00007f203c07b430 nid=0x50 runnable  

"G1 Refine#0" os_prio=0 cpu=12.06ms elapsed=82.24s tid=0x00007f203c5e6330 nid=0x52 runnable  

"G1 Refine#1" os_prio=0 cpu=0.06ms elapsed=75.83s tid=0x00007f201000b0e0 nid=0x67 runnable  

"G1 Young RemSet Sampling" os_prio=0 cpu=49.19ms elapsed=82.24s tid=0x00007f203c5e7660 nid=0x53 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=49.73ms elapsed=82.21s tid=0x00007f203c667b00 nid=0x5d waiting on condition  

JNI global refs: 30, weak refs: 45

Heap
 garbage-first heap   total 2097152K, used 691365K [0x0000000080000000, 0x0000000100000000)
  region size 1024K, 545 young (558080K), 1 survivors (1024K)
 Metaspace       used 48866K, capacity 50760K, committed 51024K, reserved 1093632K
  class space    used 6474K, capacity 7107K, committed 7168K, reserved 1048576K

2023-02-08 07:35:40
Full thread dump OpenJDK 64-Bit Server VM (15.0.1+9 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007f1fcc000d40, length=13, elements={
0x00007f203c02a7c0, 0x00007f203c61bcb0, 0x00007f203c61d4d0, 0x00007f203c6222d0,
0x00007f203c623ac0, 0x00007f203c625840, 0x00007f203c627190, 0x00007f203c6289b0,
0x00007f203c665e20, 0x00007f203c6699d0, 0x00007f203ce7acb0, 0x00007f203db1f510,
0x00007f203e46aa20
}

"main" #1 prio=5 os_prio=0 cpu=125659.83ms elapsed=128.17s tid=0x00007f203c02a7c0 nid=0x4d runnable  [0x00007f204634f000]
   java.lang.Thread.State: RUNNABLE
	at java.util.TreeMap.fixAfterInsertion(java.base@15.0.1/TreeMap.java:2574)
	at java.util.TreeMap.addEntry(java.base@15.0.1/TreeMap.java:762)
	at java.util.TreeMap.put(java.base@15.0.1/TreeMap.java:820)
	at java.util.TreeMap.put(java.base@15.0.1/TreeMap.java:534)
	at java.util.AbstractMap.putAll(java.base@15.0.1/AbstractMap.java:281)
	at java.util.TreeMap.putAll(java.base@15.0.1/TreeMap.java:326)
	at java.util.TreeMap.<init>(java.base@15.0.1/TreeMap.java:187)
	at org.elasticsearch.common.settings.Settings.<init>(Settings.java:103)
	at org.elasticsearch.common.settings.Settings.getByPrefix(Settings.java:209)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:890)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:859)
	at org.elasticsearch.common.settings.AbstractScopedSettings.archiveUnknownOrInvalidSettings(AbstractScopedSettings.java:885)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.archiveBrokenIndexSettings(MetaDataIndexUpgradeService.java:220)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:95)
	at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:246)
	at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:89)
	at org.elasticsearch.node.Node.<init>(Node.java:499)
	at org.elasticsearch.node.Node.<init>(Node.java:266)
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
	at org.elasticsearch.cli.Command.main(Command.java:90)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=2.16ms elapsed=127.56s tid=0x00007f203c61bcb0 nid=0x55 waiting on condition  [0x00007f200b577000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@15.0.1/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@15.0.1/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@15.0.1/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=5.49ms elapsed=127.56s tid=0x00007f203c61d4d0 nid=0x56 in Object.wait()  [0x00007f200b476000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@15.0.1/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.54ms elapsed=127.55s tid=0x00007f203c6222d0 nid=0x57 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=13.83ms elapsed=127.55s tid=0x00007f203c623ac0 nid=0x58 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=6918.40ms elapsed=127.55s tid=0x00007f203c625840 nid=0x59 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=1941.52ms elapsed=127.55s tid=0x00007f203c627190 nid=0x5a waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #8 daemon prio=9 os_prio=0 cpu=5.99ms elapsed=127.55s tid=0x00007f203c6289b0 nid=0x5b runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Notification Thread" #9 daemon prio=9 os_prio=0 cpu=0.09ms elapsed=127.54s tid=0x00007f203c665e20 nid=0x5c runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #10 daemon prio=8 os_prio=0 cpu=2.79ms elapsed=127.54s tid=0x00007f203c6699d0 nid=0x5e in Object.wait()  [0x00007f200ac6d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@15.0.1/CleanerImpl.java:148)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)
	at jdk.internal.misc.InnocuousThread.run(java.base@15.0.1/InnocuousThread.java:134)

"process reaper" #15 daemon prio=10 os_prio=0 cpu=0.22ms elapsed=126.69s tid=0x00007f203ce7acb0 nid=0x61 runnable  [0x00007f2020029000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ProcessHandleImpl.waitForProcessExit0(java.base@15.0.1/Native Method)
	at java.lang.ProcessHandleImpl$1.run(java.base@15.0.1/ProcessHandleImpl.java:142)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"elasticsearch[elasticsearch-master-0][[timer]]" #19 daemon prio=5 os_prio=0 cpu=27.86ms elapsed=124.35s tid=0x00007f203db1f510 nid=0x65 waiting on condition  [0x00007f2009b5c000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(java.base@15.0.1/Native Method)
	at org.elasticsearch.threadpool.ThreadPool$CachedTimeThread.run(ThreadPool.java:589)

"elasticsearch[elasticsearch-master-0][scheduler][T#1]" #20 daemon prio=5 os_prio=0 cpu=8.60ms elapsed=121.78s tid=0x00007f203e46aa20 nid=0x66 waiting on condition  [0x00007f2009cf8000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@15.0.1/Native Method)
	- parking to wait for  <0x0000000080712928> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@15.0.1/LockSupport.java:252)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@15.0.1/AbstractQueuedSynchronizer.java:1661)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:1182)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@15.0.1/ThreadPoolExecutor.java:1056)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1116)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"VM Thread" os_prio=0 cpu=77.94ms elapsed=127.56s tid=0x00007f203c618f70 nid=0x54 runnable  

"GC Thread#0" os_prio=0 cpu=1077.57ms elapsed=128.17s tid=0x00007f203c068610 nid=0x4e runnable  

"GC Thread#1" os_prio=0 cpu=1079.40ms elapsed=128.17s tid=0x00007f203c085030 nid=0x51 runnable  

"G1 Main Marker" os_prio=0 cpu=2.08ms elapsed=128.17s tid=0x00007f203c07a080 nid=0x4f runnable  

"G1 Conc#0" os_prio=0 cpu=176.48ms elapsed=128.17s tid=0x00007f203c07b430 nid=0x50 runnable  

"G1 Refine#0" os_prio=0 cpu=12.06ms elapsed=127.57s tid=0x00007f203c5e6330 nid=0x52 runnable  

"G1 Refine#1" os_prio=0 cpu=0.06ms elapsed=121.16s tid=0x00007f201000b0e0 nid=0x67 runnable  

"G1 Young RemSet Sampling" os_prio=0 cpu=77.02ms elapsed=127.57s tid=0x00007f203c5e7660 nid=0x53 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=79.56ms elapsed=127.54s tid=0x00007f203c667b00 nid=0x5d waiting on condition  

JNI global refs: 30, weak refs: 45

Heap
 garbage-first heap   total 2097152K, used 970697K [0x0000000080000000, 0x0000000100000000)
  region size 1024K, 818 young (837632K), 1 survivors (1024K)
 Metaspace       used 48866K, capacity 50760K, committed 51024K, reserved 1093632K
  class space    used 6474K, capacity 7107K, committed 7168K, reserved 1048576K

2023-02-08 07:36:22
Full thread dump OpenJDK 64-Bit Server VM (15.0.1+9 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007f1fcc000d40, length=13, elements={
0x00007f203c02a7c0, 0x00007f203c61bcb0, 0x00007f203c61d4d0, 0x00007f203c6222d0,
0x00007f203c623ac0, 0x00007f203c625840, 0x00007f203c627190, 0x00007f203c6289b0,
0x00007f203c665e20, 0x00007f203c6699d0, 0x00007f203ce7acb0, 0x00007f203db1f510,
0x00007f203e46aa20
}

"main" #1 prio=5 os_prio=0 cpu=167342.49ms elapsed=169.92s tid=0x00007f203c02a7c0 nid=0x4d runnable  [0x00007f204634f000]
   java.lang.Thread.State: RUNNABLE
	at java.util.TreeMap.fixAfterInsertion(java.base@15.0.1/TreeMap.java:2574)
	at java.util.TreeMap.addEntry(java.base@15.0.1/TreeMap.java:762)
	at java.util.TreeMap.put(java.base@15.0.1/TreeMap.java:820)
	at java.util.TreeMap.put(java.base@15.0.1/TreeMap.java:534)
	at java.util.AbstractMap.putAll(java.base@15.0.1/AbstractMap.java:281)
	at java.util.TreeMap.putAll(java.base@15.0.1/TreeMap.java:326)
	at java.util.TreeMap.<init>(java.base@15.0.1/TreeMap.java:187)
	at org.elasticsearch.common.settings.Settings.<init>(Settings.java:103)
	at org.elasticsearch.common.settings.Settings.getByPrefix(Settings.java:209)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:890)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:859)
	at org.elasticsearch.common.settings.AbstractScopedSettings.archiveUnknownOrInvalidSettings(AbstractScopedSettings.java:885)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.archiveBrokenIndexSettings(MetaDataIndexUpgradeService.java:220)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:95)
	at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:246)
	at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:89)
	at org.elasticsearch.node.Node.<init>(Node.java:499)
	at org.elasticsearch.node.Node.<init>(Node.java:266)
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
	at org.elasticsearch.cli.Command.main(Command.java:90)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=2.16ms elapsed=169.30s tid=0x00007f203c61bcb0 nid=0x55 waiting on condition  [0x00007f200b577000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@15.0.1/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@15.0.1/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@15.0.1/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=5.49ms elapsed=169.30s tid=0x00007f203c61d4d0 nid=0x56 in Object.wait()  [0x00007f200b476000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@15.0.1/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.67ms elapsed=169.30s tid=0x00007f203c6222d0 nid=0x57 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=16.37ms elapsed=169.30s tid=0x00007f203c623ac0 nid=0x58 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=7017.50ms elapsed=169.30s tid=0x00007f203c625840 nid=0x59 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=1947.68ms elapsed=169.30s tid=0x00007f203c627190 nid=0x5a waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #8 daemon prio=9 os_prio=0 cpu=5.99ms elapsed=169.30s tid=0x00007f203c6289b0 nid=0x5b runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Notification Thread" #9 daemon prio=9 os_prio=0 cpu=0.09ms elapsed=169.29s tid=0x00007f203c665e20 nid=0x5c runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #10 daemon prio=8 os_prio=0 cpu=2.85ms elapsed=169.29s tid=0x00007f203c6699d0 nid=0x5e in Object.wait()  [0x00007f200ac6d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@15.0.1/CleanerImpl.java:148)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)
	at jdk.internal.misc.InnocuousThread.run(java.base@15.0.1/InnocuousThread.java:134)

"process reaper" #15 daemon prio=10 os_prio=0 cpu=0.22ms elapsed=168.43s tid=0x00007f203ce7acb0 nid=0x61 runnable  [0x00007f2020029000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ProcessHandleImpl.waitForProcessExit0(java.base@15.0.1/Native Method)
	at java.lang.ProcessHandleImpl$1.run(java.base@15.0.1/ProcessHandleImpl.java:142)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"elasticsearch[elasticsearch-master-0][[timer]]" #19 daemon prio=5 os_prio=0 cpu=37.67ms elapsed=166.10s tid=0x00007f203db1f510 nid=0x65 waiting on condition  [0x00007f2009b5c000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(java.base@15.0.1/Native Method)
	at org.elasticsearch.threadpool.ThreadPool$CachedTimeThread.run(ThreadPool.java:589)

"elasticsearch[elasticsearch-master-0][scheduler][T#1]" #20 daemon prio=5 os_prio=0 cpu=10.45ms elapsed=163.53s tid=0x00007f203e46aa20 nid=0x66 waiting on condition  [0x00007f2009cf8000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@15.0.1/Native Method)
	- parking to wait for  <0x0000000080712928> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@15.0.1/LockSupport.java:252)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@15.0.1/AbstractQueuedSynchronizer.java:1661)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:1182)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@15.0.1/ThreadPoolExecutor.java:1056)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1116)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"VM Thread" os_prio=0 cpu=95.54ms elapsed=169.31s tid=0x00007f203c618f70 nid=0x54 runnable  

"GC Thread#0" os_prio=0 cpu=1118.92ms elapsed=169.92s tid=0x00007f203c068610 nid=0x4e runnable  

"GC Thread#1" os_prio=0 cpu=1120.36ms elapsed=169.92s tid=0x00007f203c085030 nid=0x51 runnable  

"G1 Main Marker" os_prio=0 cpu=2.08ms elapsed=169.92s tid=0x00007f203c07a080 nid=0x4f runnable  

"G1 Conc#0" os_prio=0 cpu=176.48ms elapsed=169.92s tid=0x00007f203c07b430 nid=0x50 runnable  

"G1 Refine#0" os_prio=0 cpu=12.06ms elapsed=169.31s tid=0x00007f203c5e6330 nid=0x52 runnable  

"G1 Refine#1" os_prio=0 cpu=0.06ms elapsed=162.91s tid=0x00007f201000b0e0 nid=0x67 runnable  

"G1 Young RemSet Sampling" os_prio=0 cpu=101.83ms elapsed=169.31s tid=0x00007f203c5e7660 nid=0x53 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=108.87ms elapsed=169.29s tid=0x00007f203c667b00 nid=0x5d waiting on condition  

JNI global refs: 30, weak refs: 45

Heap
 garbage-first heap   total 2097152K, used 877530K [0x0000000080000000, 0x0000000100000000)
  region size 1024K, 727 young (744448K), 1 survivors (1024K)
 Metaspace       used 48877K, capacity 50824K, committed 51024K, reserved 1093632K
  class space    used 6474K, capacity 7107K, committed 7168K, reserved 1048576K

2023-02-08 07:37:39
Full thread dump OpenJDK 64-Bit Server VM (15.0.1+9 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007f1fcc000d40, length=13, elements={
0x00007f203c02a7c0, 0x00007f203c61bcb0, 0x00007f203c61d4d0, 0x00007f203c6222d0,
0x00007f203c623ac0, 0x00007f203c625840, 0x00007f203c627190, 0x00007f203c6289b0,
0x00007f203c665e20, 0x00007f203c6699d0, 0x00007f203ce7acb0, 0x00007f203db1f510,
0x00007f203e46aa20
}

"main" #1 prio=5 os_prio=0 cpu=244521.54ms elapsed=247.22s tid=0x00007f203c02a7c0 nid=0x4d runnable  [0x00007f204634f000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.String.substring(java.base@15.0.1/String.java:1876)
	at org.elasticsearch.common.settings.Settings$FilteredMap$1$1$1.getKey(Settings.java:1301)
	at org.elasticsearch.common.settings.Settings$FilteredMap$1$1$1.getKey(Settings.java:1298)
	at java.util.AbstractMap.putAll(java.base@15.0.1/AbstractMap.java:281)
	at java.util.TreeMap.putAll(java.base@15.0.1/TreeMap.java:326)
	at java.util.TreeMap.<init>(java.base@15.0.1/TreeMap.java:187)
	at org.elasticsearch.common.settings.Settings.<init>(Settings.java:103)
	at org.elasticsearch.common.settings.Settings.getByPrefix(Settings.java:209)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:890)
	at org.elasticsearch.common.settings.Setting$GroupSetting.get(Setting.java:859)
	at org.elasticsearch.common.settings.AbstractScopedSettings.archiveUnknownOrInvalidSettings(AbstractScopedSettings.java:885)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.archiveBrokenIndexSettings(MetaDataIndexUpgradeService.java:220)
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:95)
	at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:246)
	at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:89)
	at org.elasticsearch.node.Node.<init>(Node.java:499)
	at org.elasticsearch.node.Node.<init>(Node.java:266)
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
	at org.elasticsearch.cli.Command.main(Command.java:90)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=2.16ms elapsed=246.60s tid=0x00007f203c61bcb0 nid=0x55 waiting on condition  [0x00007f200b577000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@15.0.1/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@15.0.1/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@15.0.1/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=5.49ms elapsed=246.60s tid=0x00007f203c61d4d0 nid=0x56 in Object.wait()  [0x00007f200b476000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x00000000801002e8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@15.0.1/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.79ms elapsed=246.60s tid=0x00007f203c6222d0 nid=0x57 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=20.60ms elapsed=246.60s tid=0x00007f203c623ac0 nid=0x58 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=7018.34ms elapsed=246.60s tid=0x00007f203c625840 nid=0x59 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=1953.87ms elapsed=246.60s tid=0x00007f203c627190 nid=0x5a waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #8 daemon prio=9 os_prio=0 cpu=5.99ms elapsed=246.60s tid=0x00007f203c6289b0 nid=0x5b runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Notification Thread" #9 daemon prio=9 os_prio=0 cpu=0.09ms elapsed=246.59s tid=0x00007f203c665e20 nid=0x5c runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #10 daemon prio=8 os_prio=0 cpu=2.93ms elapsed=246.58s tid=0x00007f203c6699d0 nid=0x5e in Object.wait()  [0x00007f200ac6d000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@15.0.1/Native Method)
	- waiting on <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@15.0.1/ReferenceQueue.java:155)
	- locked <0x0000000080100ac8> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@15.0.1/CleanerImpl.java:148)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)
	at jdk.internal.misc.InnocuousThread.run(java.base@15.0.1/InnocuousThread.java:134)

"process reaper" #15 daemon prio=10 os_prio=0 cpu=0.22ms elapsed=245.73s tid=0x00007f203ce7acb0 nid=0x61 runnable  [0x00007f2020029000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ProcessHandleImpl.waitForProcessExit0(java.base@15.0.1/Native Method)
	at java.lang.ProcessHandleImpl$1.run(java.base@15.0.1/ProcessHandleImpl.java:142)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1130)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"elasticsearch[elasticsearch-master-0][[timer]]" #19 daemon prio=5 os_prio=0 cpu=55.76ms elapsed=243.40s tid=0x00007f203db1f510 nid=0x65 sleeping [0x00007f2009b5c000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(java.base@15.0.1/Native Method)
	at org.elasticsearch.threadpool.ThreadPool$CachedTimeThread.run(ThreadPool.java:589)

"elasticsearch[elasticsearch-master-0][scheduler][T#1]" #20 daemon prio=5 os_prio=0 cpu=14.71ms elapsed=240.83s tid=0x00007f203e46aa20 nid=0x66 waiting on condition  [0x00007f2009cf8000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@15.0.1/Native Method)
	- parking to wait for  <0x0000000080712928> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@15.0.1/LockSupport.java:252)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@15.0.1/AbstractQueuedSynchronizer.java:1661)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:1182)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@15.0.1/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@15.0.1/ThreadPoolExecutor.java:1056)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@15.0.1/ThreadPoolExecutor.java:1116)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@15.0.1/ThreadPoolExecutor.java:630)
	at java.lang.Thread.run(java.base@15.0.1/Thread.java:832)

"VM Thread" os_prio=0 cpu=125.12ms elapsed=246.61s tid=0x00007f203c618f70 nid=0x54 runnable  

"GC Thread#0" os_prio=0 cpu=1183.34ms elapsed=247.22s tid=0x00007f203c068610 nid=0x4e runnable  

"GC Thread#1" os_prio=0 cpu=1184.92ms elapsed=247.22s tid=0x00007f203c085030 nid=0x51 runnable  

"G1 Main Marker" os_prio=0 cpu=2.08ms elapsed=247.22s tid=0x00007f203c07a080 nid=0x4f runnable  

"G1 Conc#0" os_prio=0 cpu=176.48ms elapsed=247.22s tid=0x00007f203c07b430 nid=0x50 runnable  

"G1 Refine#0" os_prio=0 cpu=12.06ms elapsed=246.61s tid=0x00007f203c5e6330 nid=0x52 runnable  

"G1 Refine#1" os_prio=0 cpu=0.06ms elapsed=240.21s tid=0x00007f201000b0e0 nid=0x67 runnable  

"G1 Young RemSet Sampling" os_prio=0 cpu=151.15ms elapsed=246.61s tid=0x00007f203c5e7660 nid=0x53 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=162.36ms elapsed=246.59s tid=0x00007f203c667b00 nid=0x5d waiting on condition  

JNI global refs: 30, weak refs: 45

Heap
 garbage-first heap   total 2097152K, used 319599K [0x0000000080000000, 0x0000000100000000)
  region size 1024K, 182 young (186368K), 1 survivors (1024K)
 Metaspace       used 48882K, capacity 50824K, committed 51024K, reserved 1093632K
  class space    used 6474K, capacity 7107K, committed 7168K, reserved 1048576K