spring-data-jpa Spring JPA异步保存

lf5gs5x2  于 2022-11-10  发布在  Spring
关注(0)|答案(1)|浏览(327)

我尝试使用CrudRepository.saveAll()方法保存一个从XLS文件创建的大型List(超过100K个条目)。由于数据大小的原因,saveAll方法需要很长时间才能运行,所以我尝试使用Apache Commons Collections ListUtils.partition对列表进行分区,并在循环中调用saveAll方法,以提高性能:

public interface ClaimsSegmentRepository extends CrudRepository<ClaimsSegment, String> {}

public class FileLoaderService {
    @Async("asyncFileLoadExecutor")
    public CompletableFuture<List<String>> loadFile(String fileName) throws Exception {
        for(List<ClaimsSegment> claimsSegmentGrp : ListUtils.partition(claimsSegmentList, MAX_PARTITION_SIZE)) {
            CompletableFuture.supplyAsync(() -> claimSegmentRepo.saveAll(claimsSegmentGrp))
                .thenAccept(claimSegmentList -> {
                   //Processing child objects for Claim Segment List
                });
    }
}

这是抛出错误HHH000010: On release of batch it still contained JDBC statements,所以我读了一些更多的在线和看到这个建议,我的回购类:

public interface ClaimsSegmentRepository extends CrudRepository<ClaimsSegment, String> {
    @Async
    <S extends ClaimsSegment> CompletableFuture<S> saveAll()
}

但是,这会导致编译错误,并抱怨返回类型不匹配:

org.springframework.beans.factory.UnsatisfiedDependencyException:
    Error creating bean with name 'loadController':
        Unsatisfied dependency expressed through field 'loaderSvc'; nested exception is org.springframework.beans.factory.BeanCreationException:
            Error creating bean with name 'fileLoaderService' defined in file [Path\to\file\FileLoaderService.class]:
                Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException:
                    Failed to instantiate [nom.side.xls.service.FileLoaderService]:
                        Constructor threw exception; nested exception is java.lang.Error: Unresolved compilation problem: 
    The method saveAll(Iterable<ClaimsSegment>) is ambiguous for the type ClaimsSegmentRepository

    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:659) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:639) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955) ~[spring-beans-5.3.23.jar:5.3.23]
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) ~[spring-context-5.3.23.jar:5.3.23]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.23.jar:5.3.23]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:147) ~[spring-boot-2.7.4.jar:2.7.4]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734) ~[spring-boot-2.7.4.jar:2.7.4]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) ~[spring-boot-2.7.4.jar:2.7.4]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:308) ~[spring-boot-2.7.4.jar:2.7.4]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) ~[spring-boot-2.7.4.jar:2.7.4]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) ~[spring-boot-2.7.4.jar:2.7.4]
    at nom.side.xls.ExcelFileLoaderApplication.main(ExcelFileLoaderApplication.java:10) ~[classes/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) ~[spring-boot-devtools-2.7.4.jar:2.7.4]
    ...

@Async注解对于find方法是有效的,但是对于savesaveAll就不行了。
谢谢你,

ct3nt3jp

ct3nt3jp1#

要使用 Boot 和SpringDataJPA进行批量插入,您只需要做两件事:
1.将选项spring.jpa.properties. hib. jdbc.batch_size设置为所需的适当值(例如:第20段)
1.请使用存储库的saveAll()方法和准备插入的实体列表。
工作示例为here

相关问题