這裡對Spring Batch 進行批處理實踐。

介紹

本文将會講述SpringBatch 如何搭建并運作起來的。

本教程，将會介紹從磁盤讀取檔案，并寫入MySql 中。

什麼是Spring Batch

Spring Batch 是Spring的子項目，基于Spring的批處理的架構，通過其可以建構出批量的批處理架構。

官方位址：github.com/spring-projects/spring-batch

入門案例

建立Spring Boot 項目

https://www.iming.info/wp-content/uploads/2020/06/wp_editor_md_9c28d388a896a77f687c203187b32649.jpg https://www.iming.info/wp-content/uploads/2020/06/wp_editor_md_0bb923d23efbd97cc4dc0b21c8a4bb54.jpg

選擇Spring Batch

https://www.iming.info/wp-content/uploads/2020/06/wp_editor_md_686be7d7d954ce9a7b0a29a1f10cc88b.jpg https://www.iming.info/wp-content/uploads/2020/06/wp_editor_md_8feba222d8e08c3f936a2ae365569cbc.jpg

繼續等待

https://www.iming.info/wp-content/uploads/2020/06/wp_editor_md_09835c3036d64f09929d7b7c6d91a206.jpg

pom 依賴如下

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.1.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>demo</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.springframework.batch</groupId>
            <artifactId>spring-batch-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

什麼是 Spring Batch works

https://www.iming.info/wp-content/uploads/2020/06/wp_editor_md_c4147eec409408a4583fde40a96fb98f.jpg

一個job有讀寫處理這三個部分組成。

通過JobLauncher啟動Job

步驟為讀、處理、寫三個步驟

一個例子

初始化目錄結構如下

https://www.iming.info/wp-content/uploads/2020/06/wp_editor_md_c1311338e7aa443626512c2998ca30bb.jpg

讀取

從數組中讀取三個字元

package com.example.demo.step;

import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;

public class Reader implements ItemReader<String> {
    private String[] message = {"ming", "mingming", "mingmingming"};
    
    private int count = 0;
    
    @Override
    public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
        if(count < message.length){
            return message[count++];
        }else{
            count = 0;
        }
    }
}

每次将會調用reader 方法，進行讀取一個，

處理

字元串轉為大寫

package com.example.demo.step;

import org.springframework.batch.item.ItemProcessor;

public class Processor implements ItemProcessor<String, String> {
    @Override
    public String process(String s) throws Exception {
        return s.toUpperCase();
    }
}

字元串将會調用其方法将其處理為大寫

寫

package com.example.demo.step;

import org.springframework.batch.item.ItemWriter;

import java.util.List;

public class Writer implements ItemWriter<String> {
    @Override
    public void write(List<? extends String> list) throws Exception {
        for (String s : list) {
            System.out.println("Writing the data " + s);
        }
    }
}

監聽

任務成功完成後往控制台輸出一行字元串

package com.example.demo.step;

import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.listener.JobExecutionListenerSupport;

public class JobCompletionListener extends JobExecutionListenerSupport {
    @Override
    public void afterJob(JobExecution jobExecution) {
        // 項目完成以後調用
        if(jobExecution.getStatus() == BatchStatus.COMPLETED){
            System.out.println("項目已經完成");
        }
    }
}

Config

對項目進行配置

package com.example.demo.config;

import com.example.demo.step.JobCompletionListener;
import com.example.demo.step.Processor;
import com.example.demo.step.Reader;
import com.example.demo.step.Writer;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecutionListener;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;


@Configuration
public class BatchConfig {
    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Bean
    public Job processJob() {
        return jobBuilderFactory.get("processJob")
                .incrementer(new RunIdIncrementer()).listener(listener())// 監聽
                .flow(orderStep1()).end().build(); // 建立步驟1 
    }

    @Bean
    // 步驟1 bean 先讀再寫
    public Step orderStep1() {
        return stepBuilderFactory.get("orderStep1").<String, String> chunk(1)
                .reader(new Reader()).processor(new Processor())    // 讀取。處理
                .writer(new Writer()).build();  // 最後寫
    }

    @Bean
    public JobExecutionListener listener() {
        return new JobCompletionListener(); // 建立監聽
    }

}

配置Controller

用來啟動應用

package com.example.demo.controller;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;


@RestController
public class JobInvokerController {
    @Autowired
    JobLauncher jobLauncher;

    @Autowired
    Job processJob;

    @RequestMapping("/invokejob")
    public String handle() throws Exception {

        JobParameters jobParameters = new JobParametersBuilder().addLong("time", System.currentTimeMillis())
                .toJobParameters();
        jobLauncher.run(processJob, jobParameters);

        return "Batch job has been invoked";
    }

}

配置檔案

spring:
  batch:
    job:
      enabled: false
  datasource:
    url: jdbc:h2:file:./DB
  jpa:
    properties:
      hibernate:
        hbm2ddl:
          auto: update

Spring Batch在加載的時候job預設都會執行，把spring.batch.job.enabled置為false，即把job設定成不可用，應用便會根據jobLauncher.run來執行。下面2行是資料庫的配置，不配置也可以，使用的嵌入式資料庫h2

添加注解

Spring Boot入口類：加注解@EnableBatchProcessing

package com.example.demo;

import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
@EnableBatchProcessing
public class DemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

}

Run

此時項目run

通路

http://localhost:8080/invokejob

項目已經啟動

https://www.iming.info/wp-content/uploads/2020/06/wp_editor_md_2e0452ebfddfd46fd5cb753be1c54cbe.jpg https://www.iming.info/wp-content/uploads/2020/06/wp_editor_md_3b93ea370fc27aabf9e5c9501a39d403.jpg

Spring Batch之批處理實踐介紹什麼是Spring Batch入門案例什麼是 Spring Batch works一個例子配置Controller配置檔案添加注解Run微信公衆号

介紹

什麼是Spring Batch

入門案例

什麼是 Spring Batch works

一個例子

讀取

處理

寫

監聽

Config

配置Controller

配置檔案

添加注解

Run

微信公衆号

繼續閱讀

Java String.format方法的簡單使用

neo4j之cypher使用文檔

GitHub連夜封殺！這份阿裡 10W 字内部 Java 字面試手冊到底有多強？

spark/scala關于【資源檔案】加載方法概述外部檔案加載方案測試資源檔案打包入jar包中小結

NOSQL安全攻擊

mybatis_入門程式Mybatis入門

mysql使用source指令導入.sql檔案

AOP程式設計_Android優雅權限架構(1)概念基礎，2021金三銀四前言正文大綱正文

登入plsql 報錯 the account is locked --使用者被鎖

Effective Java 8:通用程式設計

SequoiaDB巨杉資料庫C++驅動概述

OOM三種類型

工廠模式-三種類型

【遞歸】高效率求2的n次幂

win10本地scala和spark安裝安裝scala安裝spark

scala (3) Function 和 Method