ERROR REPORT 679F8267-00000-000001 ============================================================ Application: LINBIT® LINSTOR Module: Controller Version: 1.29.0 Build ID: b2be7208a777f0743d4c7187062678cd5416fccf Build time: 2024-07-31T10:00:44+00:00 Error time: 2025-02-06 23:02:49 Node: xencc-hp03 Thread: MainWorkerPool-14 Access context information Identity: PUBLIC Role: PUBLIC Domain: PUBLIC Peer: Node: 'xencc-hp03' ============================================================ Reported error: =============== Category: RuntimeException Class name: TransactionException Class canonical name: com.linbit.linstor.transaction.TransactionException Generated at: Method 'startTransaction', Source file 'ControllerSQLTransactionMgrGenerator.java', Line #32 Error message: Failed to start transaction Error context: Failed to start transaction ErrorContext: Asynchronous stage backtrace: Error has been observed at the following site(s): *__checkpoint ⇢ Handle event *__checkpoint ⇢ Fallback error handling wrapper Original Stack Trace: Call backtrace: Method Native Class:Line number startTransaction N com.linbit.linstor.transaction.manager.ControllerSQLTransactionMgrGenerator:32 Suppressed exception 1 of 1: =============== Category: RuntimeException Class name: OnAssemblyException Class canonical name: reactor.core.publisher.FluxOnAssembly.OnAssemblyException Generated at: Method 'startTransaction', Source file 'ControllerSQLTransactionMgrGenerator.java', Line #32 Error message: Error has been observed at the following site(s): *__checkpoint ⇢ Handle event *__checkpoint ⇢ Fallback error handling wrapper Original Stack Trace: Error context: Failed to start transaction Call backtrace: Method Native Class:Line number startTransaction N com.linbit.linstor.transaction.manager.ControllerSQLTransactionMgrGenerator:32 doInScope N com.linbit.linstor.core.apicallhandler.ScopeRunner:152 lambda$fluxInScope$0 N com.linbit.linstor.core.apicallhandler.ScopeRunner:100 call N reactor.core.publisher.MonoCallable:72 trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:127 subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49 subscribe N reactor.core.publisher.Flux:8759 onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195 request N reactor.core.publisher.Operators$ScalarSubscription:2545 onSubscribe N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141 subscribe N reactor.core.publisher.MonoJust:55 subscribe N reactor.core.publisher.MonoDeferContextual:55 subscribe N reactor.core.publisher.InternalFluxOperator:62 subscribe N reactor.core.publisher.FluxDefer:54 subscribe N reactor.core.publisher.Mono:4495 onComplete N reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber:258 subscribe N reactor.core.publisher.FluxConcatArray:78 subscribe N reactor.core.publisher.Flux:8773 trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:200 subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49 subscribe N reactor.core.publisher.InternalFluxOperator:55 subscribe N reactor.core.publisher.FluxDefer:54 subscribe N reactor.core.publisher.Flux:8773 onNext N reactor.core.publisher.FluxFlatMap$FlatMapMain:427 drainAsync N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:453 drain N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:724 onNext N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:256 drainFused N reactor.core.publisher.SinkManyUnicast:319 drain N reactor.core.publisher.SinkManyUnicast:362 tryEmitNext N reactor.core.publisher.SinkManyUnicast:237 tryEmitNext N reactor.core.publisher.SinkManySerialized:100 processInOrder N com.linbit.linstor.netcom.TcpConnectorPeer:416 doProcessMessage N com.linbit.linstor.proto.CommonMessageProcessor:228 lambda$processMessage$2 N com.linbit.linstor.proto.CommonMessageProcessor:165 onNext N reactor.core.publisher.FluxPeek$PeekSubscriber:185 runAsync N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:440 run N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:527 call N reactor.core.scheduler.WorkerTask:84 call N reactor.core.scheduler.WorkerTask:37 run N java.util.concurrent.FutureTask:264 run N java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask:304 runWorker N java.util.concurrent.ThreadPoolExecutor:1128 run N java.util.concurrent.ThreadPoolExecutor$Worker:628 run N java.lang.Thread:829 Caused by: ========== Category: Exception Class name: JdbcSQLException Class canonical name: org.h2.jdbc.JdbcSQLException Generated at: Method 'getJdbcSQLException', Source file 'DbException.java', Line #357 Error message: IO Exception: null [90028-197] Call backtrace: Method Native Class:Line number getJdbcSQLException N org.h2.message.DbException:357 get N org.h2.message.DbException:168 convertIllegalStateException N org.h2.mvstore.db.MVTableEngine$Store:192 open N org.h2.mvstore.db.MVTableEngine$Store:168 init N org.h2.mvstore.db.MVTableEngine:100 getPageStore N org.h2.engine.Database:2538 open N org.h2.engine.Database:709 openDatabase N org.h2.engine.Database:286 N org.h2.engine.Database:280 openSession N org.h2.engine.Engine:66 openSession N org.h2.engine.Engine:179 createSessionAndValidate N org.h2.engine.Engine:157 createSession N org.h2.engine.Engine:140 createSession N org.h2.engine.Engine:28 connectEmbeddedOrServer N org.h2.engine.SessionRemote:351 N org.h2.jdbc.JdbcConnection:124 N org.h2.jdbc.JdbcConnection:103 connect N org.h2.Driver:69 getConnection N java.sql.DriverManager:677 getConnection N java.sql.DriverManager:189 createConnection N org.apache.commons.dbcp2.DriverManagerConnectionFactory:123 makeObject N org.apache.commons.dbcp2.PoolableConnectionFactory:355 create N org.apache.commons.pool2.impl.GenericObjectPool:889 borrowObject N org.apache.commons.pool2.impl.GenericObjectPool:424 borrowObject N org.apache.commons.pool2.impl.GenericObjectPool:349 getConnection N org.apache.commons.dbcp2.PoolingDataSource:134 getConnection N com.linbit.linstor.dbcp.DbConnectionPool:167 N com.linbit.linstor.transaction.manager.ControllerSQLTransactionMgr:20 startTransaction N com.linbit.linstor.transaction.manager.ControllerSQLTransactionMgrGenerator:28 doInScope N com.linbit.linstor.core.apicallhandler.ScopeRunner:152 lambda$fluxInScope$0 N com.linbit.linstor.core.apicallhandler.ScopeRunner:100 call N reactor.core.publisher.MonoCallable:72 trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:127 subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49 subscribe N reactor.core.publisher.Flux:8759 onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195 request N reactor.core.publisher.Operators$ScalarSubscription:2545 onSubscribe N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141 subscribe N reactor.core.publisher.MonoJust:55 subscribe N reactor.core.publisher.MonoDeferContextual:55 subscribe N reactor.core.publisher.InternalFluxOperator:62 subscribe N reactor.core.publisher.FluxDefer:54 subscribe N reactor.core.publisher.Mono:4495 onComplete N reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber:258 subscribe N reactor.core.publisher.FluxConcatArray:78 subscribe N reactor.core.publisher.Flux:8773 trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:200 subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49 subscribe N reactor.core.publisher.InternalFluxOperator:55 subscribe N reactor.core.publisher.FluxDefer:54 subscribe N reactor.core.publisher.Flux:8773 onNext N reactor.core.publisher.FluxFlatMap$FlatMapMain:427 drainAsync N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:453 drain N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:724 onNext N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:256 drainFused N reactor.core.publisher.SinkManyUnicast:319 drain N reactor.core.publisher.SinkManyUnicast:362 tryEmitNext N reactor.core.publisher.SinkManyUnicast:237 tryEmitNext N reactor.core.publisher.SinkManySerialized:100 processInOrder N com.linbit.linstor.netcom.TcpConnectorPeer:416 doProcessMessage N com.linbit.linstor.proto.CommonMessageProcessor:228 lambda$processMessage$2 N com.linbit.linstor.proto.CommonMessageProcessor:165 onNext N reactor.core.publisher.FluxPeek$PeekSubscriber:185 runAsync N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:440 run N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:527 call N reactor.core.scheduler.WorkerTask:84 call N reactor.core.scheduler.WorkerTask:37 run N java.util.concurrent.FutureTask:264 run N java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask:304 runWorker N java.util.concurrent.ThreadPoolExecutor:1128 run N java.util.concurrent.ThreadPoolExecutor$Worker:628 run N java.lang.Thread:829 Caused by: ========== Category: RuntimeException Class name: IllegalStateException Class canonical name: java.lang.IllegalStateException Generated at: Method 'newIllegalStateException', Source file 'DataUtils.java', Line #870 Error message: Reading from nio:/var/lib/linstor/linstordb.mv.db failed; file length 901120 read length 8192 at 0 [1.4.197/1] Call backtrace: Method Native Class:Line number newIllegalStateException N org.h2.mvstore.DataUtils:870 readFully N org.h2.mvstore.DataUtils:417 readFully N org.h2.mvstore.FileStore:98 readStoreHeader N org.h2.mvstore.MVStore:563 N org.h2.mvstore.MVStore:361 open N org.h2.mvstore.MVStore$Builder:2934 open N org.h2.mvstore.db.MVTableEngine$Store:155 init N org.h2.mvstore.db.MVTableEngine:100 getPageStore N org.h2.engine.Database:2538 open N org.h2.engine.Database:709 openDatabase N org.h2.engine.Database:286 N org.h2.engine.Database:280 openSession N org.h2.engine.Engine:66 openSession N org.h2.engine.Engine:179 createSessionAndValidate N org.h2.engine.Engine:157 createSession N org.h2.engine.Engine:140 createSession N org.h2.engine.Engine:28 connectEmbeddedOrServer N org.h2.engine.SessionRemote:351 N org.h2.jdbc.JdbcConnection:124 N org.h2.jdbc.JdbcConnection:103 connect N org.h2.Driver:69 getConnection N java.sql.DriverManager:677 getConnection N java.sql.DriverManager:189 createConnection N org.apache.commons.dbcp2.DriverManagerConnectionFactory:123 makeObject N org.apache.commons.dbcp2.PoolableConnectionFactory:355 create N org.apache.commons.pool2.impl.GenericObjectPool:889 borrowObject N org.apache.commons.pool2.impl.GenericObjectPool:424 borrowObject N org.apache.commons.pool2.impl.GenericObjectPool:349 getConnection N org.apache.commons.dbcp2.PoolingDataSource:134 getConnection N com.linbit.linstor.dbcp.DbConnectionPool:167 N com.linbit.linstor.transaction.manager.ControllerSQLTransactionMgr:20 startTransaction N com.linbit.linstor.transaction.manager.ControllerSQLTransactionMgrGenerator:28 doInScope N com.linbit.linstor.core.apicallhandler.ScopeRunner:152 lambda$fluxInScope$0 N com.linbit.linstor.core.apicallhandler.ScopeRunner:100 call N reactor.core.publisher.MonoCallable:72 trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:127 subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49 subscribe N reactor.core.publisher.Flux:8759 onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195 request N reactor.core.publisher.Operators$ScalarSubscription:2545 onSubscribe N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141 subscribe N reactor.core.publisher.MonoJust:55 subscribe N reactor.core.publisher.MonoDeferContextual:55 subscribe N reactor.core.publisher.InternalFluxOperator:62 subscribe N reactor.core.publisher.FluxDefer:54 subscribe N reactor.core.publisher.Mono:4495 onComplete N reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber:258 subscribe N reactor.core.publisher.FluxConcatArray:78 subscribe N reactor.core.publisher.Flux:8773 trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:200 subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49 subscribe N reactor.core.publisher.InternalFluxOperator:55 subscribe N reactor.core.publisher.FluxDefer:54 subscribe N reactor.core.publisher.Flux:8773 onNext N reactor.core.publisher.FluxFlatMap$FlatMapMain:427 drainAsync N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:453 drain N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:724 onNext N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:256 drainFused N reactor.core.publisher.SinkManyUnicast:319 drain N reactor.core.publisher.SinkManyUnicast:362 tryEmitNext N reactor.core.publisher.SinkManyUnicast:237 tryEmitNext N reactor.core.publisher.SinkManySerialized:100 processInOrder N com.linbit.linstor.netcom.TcpConnectorPeer:416 doProcessMessage N com.linbit.linstor.proto.CommonMessageProcessor:228 lambda$processMessage$2 N com.linbit.linstor.proto.CommonMessageProcessor:165 onNext N reactor.core.publisher.FluxPeek$PeekSubscriber:185 runAsync N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:440 run N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:527 call N reactor.core.scheduler.WorkerTask:84 call N reactor.core.scheduler.WorkerTask:37 run N java.util.concurrent.FutureTask:264 run N java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask:304 runWorker N java.util.concurrent.ThreadPoolExecutor:1128 run N java.util.concurrent.ThreadPoolExecutor$Worker:628 run N java.lang.Thread:829 Caused by: ========== Category: Exception Class name: IOException Class canonical name: java.io.IOException Generated at: Method 'pread0', Source file 'FileDispatcherImpl.java, Unknown line number Error message: Input/output error Call backtrace: Method Native Class:Line number pread0 Y sun.nio.ch.FileDispatcherImpl:unknown pread N sun.nio.ch.FileDispatcherImpl:54 readIntoNativeBuffer N sun.nio.ch.IOUtil:274 read N sun.nio.ch.IOUtil:245 readInternal N sun.nio.ch.FileChannelImpl:811 read N sun.nio.ch.FileChannelImpl:796 read N org.h2.store.fs.FileNio:74 readFully N org.h2.mvstore.DataUtils:403 readFully N org.h2.mvstore.FileStore:98 readStoreHeader N org.h2.mvstore.MVStore:563 N org.h2.mvstore.MVStore:361 open N org.h2.mvstore.MVStore$Builder:2934 open N org.h2.mvstore.db.MVTableEngine$Store:155 init N org.h2.mvstore.db.MVTableEngine:100 getPageStore N org.h2.engine.Database:2538 open N org.h2.engine.Database:709 openDatabase N org.h2.engine.Database:286 N org.h2.engine.Database:280 openSession N org.h2.engine.Engine:66 openSession N org.h2.engine.Engine:179 createSessionAndValidate N org.h2.engine.Engine:157 createSession N org.h2.engine.Engine:140 createSession N org.h2.engine.Engine:28 connectEmbeddedOrServer N org.h2.engine.SessionRemote:351 N org.h2.jdbc.JdbcConnection:124 N org.h2.jdbc.JdbcConnection:103 connect N org.h2.Driver:69 getConnection N java.sql.DriverManager:677 getConnection N java.sql.DriverManager:189 createConnection N org.apache.commons.dbcp2.DriverManagerConnectionFactory:123 makeObject N org.apache.commons.dbcp2.PoolableConnectionFactory:355 create N org.apache.commons.pool2.impl.GenericObjectPool:889 borrowObject N org.apache.commons.pool2.impl.GenericObjectPool:424 borrowObject N org.apache.commons.pool2.impl.GenericObjectPool:349 getConnection N org.apache.commons.dbcp2.PoolingDataSource:134 getConnection N com.linbit.linstor.dbcp.DbConnectionPool:167 N com.linbit.linstor.transaction.manager.ControllerSQLTransactionMgr:20 startTransaction N com.linbit.linstor.transaction.manager.ControllerSQLTransactionMgrGenerator:28 doInScope N com.linbit.linstor.core.apicallhandler.ScopeRunner:152 lambda$fluxInScope$0 N com.linbit.linstor.core.apicallhandler.ScopeRunner:100 call N reactor.core.publisher.MonoCallable:72 trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:127 subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49 subscribe N reactor.core.publisher.Flux:8759 onNext N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:195 request N reactor.core.publisher.Operators$ScalarSubscription:2545 onSubscribe N reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain:141 subscribe N reactor.core.publisher.MonoJust:55 subscribe N reactor.core.publisher.MonoDeferContextual:55 subscribe N reactor.core.publisher.InternalFluxOperator:62 subscribe N reactor.core.publisher.FluxDefer:54 subscribe N reactor.core.publisher.Mono:4495 onComplete N reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber:258 subscribe N reactor.core.publisher.FluxConcatArray:78 subscribe N reactor.core.publisher.Flux:8773 trySubscribeScalarMap N reactor.core.publisher.FluxFlatMap:200 subscribeOrReturn N reactor.core.publisher.MonoFlatMapMany:49 subscribe N reactor.core.publisher.InternalFluxOperator:55 subscribe N reactor.core.publisher.FluxDefer:54 subscribe N reactor.core.publisher.Flux:8773 onNext N reactor.core.publisher.FluxFlatMap$FlatMapMain:427 drainAsync N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:453 drain N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:724 onNext N reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber:256 drainFused N reactor.core.publisher.SinkManyUnicast:319 drain N reactor.core.publisher.SinkManyUnicast:362 tryEmitNext N reactor.core.publisher.SinkManyUnicast:237 tryEmitNext N reactor.core.publisher.SinkManySerialized:100 processInOrder N com.linbit.linstor.netcom.TcpConnectorPeer:416 doProcessMessage N com.linbit.linstor.proto.CommonMessageProcessor:228 lambda$processMessage$2 N com.linbit.linstor.proto.CommonMessageProcessor:165 onNext N reactor.core.publisher.FluxPeek$PeekSubscriber:185 runAsync N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:440 run N reactor.core.publisher.FluxPublishOn$PublishOnSubscriber:527 call N reactor.core.scheduler.WorkerTask:84 call N reactor.core.scheduler.WorkerTask:37 run N java.util.concurrent.FutureTask:264 run N java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask:304 runWorker N java.util.concurrent.ThreadPoolExecutor:1128 run N java.util.concurrent.ThreadPoolExecutor$Worker:628 run N java.lang.Thread:829 END OF ERROR REPORT.