Enlisted connection without active transaction in later Quarkus version

[iocanel]

Description

I tried to upgrade a working example I had from Quarkus 3.4.0 and Quarkus JBeret 2.0.0 to the latest and greatest (3.9.5 and 2.3.0) but I started getting the error bellow.

I tried to lower the version to see where the error is coming from and the highest I got is:
Quarkus: 3.8.4
Quarkus JBerer: 2.3.0 (2.3.1 seems to need Quarkus 3.9.0 or else it gives me some Hibernate config converter errors).

The exception:

2024-05-04 10:22:10,850 WARN  [com.arj.ats.jta] (executor-thread-2) ARJUNA016045: attempted rollback of < formatId=131077, gtrid_length=35, bqual_length=36, tx_uid=0:ffffc0a80106:a4bf:6635e220:1b,
 node_name=quarkus, branch_uid=0:ffffc0a80106:a4bf:6635e220:22, subordinatenodename=null, eis_name=0 > (io.agroal.narayana.LocalXAResource@34af3c75) failed with exception code XAException.XAER_RME
RR: javax.transaction.xa.XAException: Error trying to transactionRollback local transaction: Enlisted connection used without active transaction
        at io.agroal.narayana.XAExceptionUtils.xaException(XAExceptionUtils.java:20)
        at io.agroal.narayana.XAExceptionUtils.xaException(XAExceptionUtils.java:8)
        at io.agroal.narayana.LocalXAResource.rollback(LocalXAResource.java:89)
        at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:338)
        at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:644)
        at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:398)
        at io.agroal.narayana.NarayanaTransactionIntegration.associate(NarayanaTransactionIntegration.java:120)
        at io.agroal.pool.ConnectionPool.getConnection(ConnectionPool.java:257)
        at io.agroal.pool.DataSource.getConnection(DataSource.java:86)
        at io.quarkus.hibernate.orm.runtime.customized.QuarkusConnectionProvider.getConnection(QuarkusConnectionProvider.java:23)
        at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:46)
        at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:113)
        at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:143)
        at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:54)
        at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:153)
        at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:183)
        at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:155)
        at org.hibernate.sql.exec.spi.JdbcSelectExecutor.lambda$list$0(JdbcSelectExecutor.java:85)
        at org.hibernate.sql.results.jdbc.internal.DeferredResultSetAccess.executeQuery(DeferredResultSetAccess.java:231)
        at org.hibernate.sql.results.jdbc.internal.DeferredResultSetAccess.getResultSet(DeferredResultSetAccess.java:167)
        at org.hibernate.sql.results.jdbc.internal.JdbcValuesResultSetImpl.advanceNext(JdbcValuesResultSetImpl.java:218)
        at org.hibernate.sql.results.jdbc.internal.JdbcValuesResultSetImpl.processNext(JdbcValuesResultSetImpl.java:98)
        at org.hibernate.sql.results.jdbc.internal.AbstractJdbcValues.next(AbstractJdbcValues.java:19)
        at org.hibernate.sql.results.internal.RowProcessingStateStandardImpl.next(RowProcessingStateStandardImpl.java:66)
        at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:202)
        at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:33)
        at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.doExecuteQuery(JdbcSelectExecutorStandardImpl.java:209)
        at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.executeQuery(JdbcSelectExecutorStandardImpl.java:83)
        at org.hibernate.sql.exec.spi.JdbcSelectExecutor.list(JdbcSelectExecutor.java:76)
        at org.hibernate.sql.exec.spi.JdbcSelectExecutor.list(JdbcSelectExecutor.java:65)
        at org.hibernate.query.sqm.internal.ConcreteSqmSelectQueryPlan.lambda$new$2(ConcreteSqmSelectQueryPlan.java:137)
        at org.hibernate.query.sqm.internal.ConcreteSqmSelectQueryPlan.withCacheableSqmInterpretation(ConcreteSqmSelectQueryPlan.java:362)
        at org.hibernate.query.sqm.internal.ConcreteSqmSelectQueryPlan.performList(ConcreteSqmSelectQueryPlan.java:303)
        at org.hibernate.query.sqm.internal.QuerySqmImpl.doList(QuerySqmImpl.java:509)
        at org.hibernate.query.spi.AbstractSelectionQuery.list(AbstractSelectionQuery.java:427)
        at org.hibernate.query.Query.getResultList(Query.java:120)
        at io.quarkus.hibernate.orm.panache.common.runtime.CommonPanacheQueryImpl.list(CommonPanacheQueryImpl.java:280)
        at io.quarkus.hibernate.orm.panache.runtime.PanacheQueryImpl.list(PanacheQueryImpl.java:149)
        at org.acme.entity.Expense.findAllExpensesOfMonth(Expense.java:51)
        at org.acme.batch.bill.ExpenseItemReader.open(ExpenseItemReader.java:42)
        at org.jberet.runtime.runner.ChunkRunner.run(ChunkRunner.java:195)
        at org.jberet.runtime.runner.StepExecutionRunner.runBatchletOrChunk(StepExecutionRunner.java:223)
        at org.jberet.runtime.runner.StepExecutionRunner.run(StepExecutionRunner.java:142)
        at org.jberet.runtime.runner.CompositeExecutionRunner.runStep(CompositeExecutionRunner.java:170)
        at org.jberet.runtime.runner.CompositeExecutionRunner.runFromHeadOrRestartPoint(CompositeExecutionRunner.java:94)
        at org.jberet.runtime.runner.JobExecutionRunner.run(JobExecutionRunner.java:58)
        at org.jberet.spi.JobExecutor$1.run(JobExecutor.java:100)
        at io.smallrye.context.impl.wrappers.SlowContextualRunnable.run(SlowContextualRunnable.java:19)
        at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:582)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1512)
        at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
        at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.sql.SQLException: Enlisted connection used without active transaction
        at io.agroal.pool.ConnectionHandler.verifyEnlistment(ConnectionHandler.java:381)
        at io.agroal.pool.ConnectionHandler.transactionRollback(ConnectionHandler.java:352)
        at io.agroal.narayana.LocalXAResource.rollback(LocalXAResource.java:86)
        ... 52 more

My setup

The batch-job.xml looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<job id="billing-job" xmlns="http://xmlns.jcp.org/xml/ns/javaee" version="1.0">
    <step id="createBillItems" >
        <chunk>
            <reader ref="expenseItemReader"/>
            <processor ref="billItemGenerator"/>
            <writer ref="billItemWriter"/>
        </chunk>    
    </step>
</job>

My reader:

package org.acme.batch.bill;

import java.io.Serializable;
import java.util.List;
import java.util.Properties;

import org.acme.entity.Expense;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import jakarta.batch.api.chunk.AbstractItemReader;
import jakarta.batch.operations.JobOperator;
import jakarta.batch.runtime.context.JobContext;
import jakarta.enterprise.context.Dependent;
import jakarta.inject.Inject;
import jakarta.inject.Named;

@Dependent
@Named
public class ExpenseItemReader extends AbstractItemReader {

  @Inject
  private JobOperator jobOperator;

  @Inject
  private JobContext jobContext;

  @Inject
  @ConfigProperty(name = "month", defaultValue = "1")
  private int month;
  @Inject
  @ConfigProperty(name = "year", defaultValue = "2020")
  private int year;

  private List<Expense> items;
  private int currentIndex = 0;

  @Override
  public void open(Serializable checkpoint) throws Exception {
    Properties properties = jobOperator.getParameters(jobContext.getExecutionId());
    month = Integer.parseInt(properties.getProperty("month"));
    year = Integer.parseInt(properties.getProperty("year"));
    items = Expense.findAllExpensesOfMonth(month, year);
  }

  @Override
  public Object readItem() throws Exception {
    if (currentIndex < items.size()) {
      return items.get(currentIndex++);
    } else {
      return null;
    }
  }
}

My writer:

package org.acme.batch.bill;

import java.util.List;

import org.acme.entity.BillItem;

import jakarta.batch.api.chunk.AbstractItemWriter;
import jakarta.enterprise.context.Dependent;
import jakarta.inject.Named;
import jakarta.transaction.Transactional;

@Dependent
@Named
public class BillItemWriter extends AbstractItemWriter {

  @Transactional
  @Override
  public void writeItems(List<Object> items) throws Exception {
    for (Object item : items) {
      if (item instanceof BillItem billItem) {
        billItem.persist();
      } else if (item instanceof List list) {
        writeItems(list);
      }
    }
  }
}

[iocanel]

Just tried with Quarkus 3.10.0 and now I am getting a different kind of error. Looking ...

edit:: Once I worked around the 3.10.0 I end up with the original issue.

[radcortez]

Ok, I'll have a look.

[radcortez]

I've tried reproducing it with my batch project https://github.com/radcortez/wow-auctions but had no luck. Any chance of a reproducer? Unfortunately, the code in the issue is missing pieces.

[iocanel]

Will try to create a reproducer.

[dabukster]

I face the same issue with Quarkus 3.14.1 and JBeret extension version 2.40 when I have 'jdbc' as repository activated. This error does not occur with an in-memory repository. Maybe this helps to reproduce the error.

[radcortez]

@dabukster, is there any chance you can provide a reproducer? Thank you!