How to prevent Spring from trying to create indexes during MongoDB transactions?

Question

I am trying to utilize Spring Data Mongo with Transactions. I initially ran into an issue where my inserts would fail due to Spring trying to create the collection and/or indexes during the first insert of a document. I have since solved that issue by creating all of my collections and indexes on startup, prior to any transaction starting. However, I am still using the Spring Data annotations to define the indexes (ie, @Indexed, @CompoundIndexes, etc.). However, even though I have created all the indexes already, Spring is still trying to ensure/create the indexes during mongo processing.

In my situation, I am using inheritance with my data types. Ie., BasicUnit which other Unit's extend. If I go to store a particular type of Unit, and it is the first time I have tried to save that type of object during the current application run, Spring doesn't recognize it, and creates a new PersistentEntity for it. As part of creating a new PersistentEntity, Spring posts an event, which MongoPersistentEntityIndexCreator catches, which in turn tries to ensure all indexes are created, and thus an exception is thrown.

Here is the exception:

Caused by: com.mongodb.MongoCommandException: Command failed with error 263 (OperationNotSupportedInTransaction): 'It is illegal to run command createIndexes in a multi-document transaction.' on server 127.0.0.1:27017. The full response is { "operationTime" : { "$timestamp" : { "t" : 1560198052, "i" : 1 } }, "ok" : 0.0, "errmsg" : "It is illegal to run command createIndexes in a multi-document transaction.", "code" : 263, "codeName" : "OperationNotSupportedInTransaction", "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1560198052, "i" : 1 } }, "signature" : { "hash" : { "$binary" : "AAAAAAAAAAAAAAAAAAAAAAAAAAA=", "$type" : "00" }, "keyId" : { "$numberLong" : "0" } } } }
    at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:179)
    at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:293)
    at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255)
    at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99)
    at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:444)
    at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72)
    at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:200)
    at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269)
    at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131)
    at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:123)
    at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:242)
    at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:233)
    at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:170)
    at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:163)
    at com.mongodb.operation.CreateIndexesOperation$1.call(CreateIndexesOperation.java:174)
    at com.mongodb.operation.CreateIndexesOperation$1.call(CreateIndexesOperation.java:169)
    at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:453)
    at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:415)
    at com.mongodb.operation.CreateIndexesOperation.execute(CreateIndexesOperation.java:169)
    at com.mongodb.operation.CreateIndexesOperation.execute(CreateIndexesOperation.java:70)
    at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:193)
    at com.mongodb.client.internal.MongoCollectionImpl.executeCreateIndexes(MongoCollectionImpl.java:805)
    at com.mongodb.client.internal.MongoCollectionImpl.createIndexes(MongoCollectionImpl.java:800)
    at com.mongodb.client.internal.MongoCollectionImpl.createIndexes(MongoCollectionImpl.java:793)
    at com.mongodb.client.internal.MongoCollectionImpl.createIndex(MongoCollectionImpl.java:778)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:246)
    at org.springframework.data.mongodb.SessionAwareMethodInterceptor.invoke(SessionAwareMethodInterceptor.java:123)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy202.createIndex(Unknown Source)
    at org.springframework.data.mongodb.core.DefaultIndexOperations.lambda$ensureIndex$0(DefaultIndexOperations.java:135)
    at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:545)

Here is the call stack leanding up to the call to ensureIndex:

DefaultIndexOperations.lambda$ensureIndex$0(IndexDefinition, MongoCollection) line: 135 
1894975953.doInCollection(MongoCollection) line: not available  
MongoTemplate.execute(String, CollectionCallback<T>) line: 545  
DefaultIndexOperations.execute(CollectionCallback<T>) line: 218 
DefaultIndexOperations.ensureIndex(IndexDefinition) line: 121   
MongoPersistentEntityIndexCreator.createIndex(MongoPersistentEntityIndexResolver$IndexDefinitionHolder) line: 145   
MongoPersistentEntityIndexCreator.checkForAndCreateIndexes(MongoPersistentEntity<?>) line: 135  
MongoPersistentEntityIndexCreator.checkForIndexes(MongoPersistentEntity<?>) line: 127   
MongoPersistentEntityIndexCreator.onApplicationEvent(MappingContextEvent<?,?>) line: 111    
MongoPersistentEntityIndexCreator.onApplicationEvent(ApplicationEvent) line: 54 
SimpleApplicationEventMulticaster.doInvokeListener(ApplicationListener, ApplicationEvent) line: 172 
SimpleApplicationEventMulticaster.invokeListener(ApplicationListener<?>, ApplicationEvent) line: 165    
SimpleApplicationEventMulticaster.multicastEvent(ApplicationEvent, ResolvableType) line: 139    
AnnotationConfigWebApplicationContext(AbstractApplicationContext).publishEvent(Object, ResolvableType) line: 398    
AnnotationConfigWebApplicationContext(AbstractApplicationContext).publishEvent(ApplicationEvent) line: 355  
MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 405  
MongoMappingContext(AbstractMappingContext<E,P>).getPersistentEntity(TypeInformation<?>) line: 248  
MongoMappingContext(AbstractMappingContext<E,P>).getPersistentEntity(Class<?>) line: 191    
MongoMappingContext(AbstractMappingContext<E,P>).getPersistentEntity(Class) line: 85    
MongoMappingContext(MappingContext<E,P>).getRequiredPersistentEntity(Class<?>) line: 73 
EntityOperations$AdaptibleMappedEntity<T>.of(T, MappingContext<MongoPersistentEntity<?>,MongoPersistentProperty>, ConversionService) line: 600  
EntityOperations$AdaptibleMappedEntity<T>.access$100(Object, MappingContext, ConversionService) line: 580   
EntityOperations.forEntity(T, ConversionService) line: 105  
MongoTemplate.doInsert(String, T, MongoWriter<T>) line: 1237

You can see where Spring tries to get the persistent entity, then finally decides to add, and then fires off the event, and then the event is handled by attempting to create indexes.

I need to know how to prevent Spring from attempting to create those indexes during a transaction.

The things I have considered

1. Try to figure out how to pre-register all my persistent entities, but I'm not sure how to do that besides manually keeping a list, or writing code to scan the classpath to find all classes that are annotated by @Document or extend such a class.
2. Completely abandon Spring's index annotations and only use explicit calls to ensureIndex for every index I need.

Neither of these solutions is very attractive to me.

I find it hard to believe that no one else has run into similar index issues using Spring Data Mongo with Transactions, so any of you know a solution to this problem, I would love to hear it.

Thanks.

Answer 1

Automatic index creation will be disabled by default as of Spring Data MongoDB 3.x.     Please use 'MongoMappingContext#setAutoIndexCreation(boolean)' or override 'MongoConfigurationSupport#autoIndexCreation()' to be explicit.  However, we recommend setting up indices manually in an application ready block. You may use index derivation there as well.

> -----------------------------------------------------------------------------------------
> @EventListener(ApplicationReadyEvent.class)
> public void initIndicesAfterStartup() {
>
>     IndexOperations indexOps = mongoTemplate.indexOps(DomainType.class);
>
>     IndexResolver resolver = new MongoPersistentEntityIndexResolver(mongoMappingContext);
>     resolver.resolveIndexFor(DomainType.class).forEach(indexOps::ensureIndex);
> }
> -----------------------------------------------------------------------------------------

Try this

@EventListener(ApplicationReadyEvent.class)
public void initIndicesAfterStartup() {    
  log.info("Mongo InitIndicesAfterStartup init");
  var init = System.currentTimeMillis();

  var mappingContext = this.mongoConverter.getMappingContext();

  if (mappingContext instanceof MongoMappingContext) {
    var mongoMappingContext = (MongoMappingContext) mappingContext;

    for (BasicMongoPersistentEntity<?> persistentEntity : mongoMappingContext.getPersistentEntities()) {
      var clazz = persistentEntity.getType();
      if (clazz.isAnnotationPresent(Document.class)) {
        var indexOps = mongoTemplate.indexOps(clazz);
        var resolver = new MongoPersistentEntityIndexResolver(mongoMappingContext);                    
        resolver.resolveIndexFor(clazz).forEach(indexOps::ensureIndex);
      }
    }
  }
  log.info("Mongo InitIndicesAfterStartup take: {}", (System.currentTimeMillis() - init));
}

remember that var is only for java 11+