panbaolin/SourceTermAnalysisSystem_java
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix:

Browse Source 
1.按客户要求修改正演和反演逻辑
2.修改系统调用flexpart逻辑弃用docker，通过ssh调用宿主机安装的flexpart
3.通过docker-compose配置环境变量，系统自动读取flexpart所在主机
This commit is contained in:
panbaolin 2026-02-06 18:11:50 +08:00
parent 2fae10797a
commit 2a8c763b58
35 changed files with 2770 additions and 963 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

162
README.md
View File

@ -1,164 +1,6 @@
JeecgBoot 低代码开发平台
===============
当前最新版本： 3.8.0（发布日期：2025-05-16）
[![AUR](https://img.shields.io/badge/license-Apache%20License%202.0-blue.svg)](https://github.com/zhangdaiscott/jeecg-boot/blob/master/LICENSE)
[![](https://img.shields.io/badge/Author-北京国炬软件-orange.svg)](http://jeecg.com/aboutusIndex)
[![](https://img.shields.io/badge/version-3.8.0-brightgreen.svg)](https://github.com/zhangdaiscott/jeecg-boot)
[![GitHub stars](https://img.shields.io/github/stars/zhangdaiscott/jeecg-boot.svg?style=social&label=Stars)](https://github.com/zhangdaiscott/jeecg-boot)
[![GitHub forks](https://img.shields.io/github/forks/zhangdaiscott/jeecg-boot.svg?style=social&label=Fork)](https://github.com/zhangdaiscott/jeecg-boot)
项目介绍
-----------------------------------
<h3 align="center">Java Low Code Platform for Enterprise web applications</h3>
JeecgBoot 是一款基于代码生成器的`低代码开发平台`！前后端分离架构 SpringBoot2.x和3.x，SpringCloud，Ant Design Vue3，Mybatis-plus，Shiro，JWT，支持微服务。强大的代码生成器让前后端代码一键生成，实现低代码开发! JeecgBoot 引领新的低代码开发模式(OnlineCoding-> 代码生成器-> 手工MERGE)， 帮助解决Java项目70%的重复工作，让开发更多关注业务。既能快速提高效率，节省研发成本，同时又不失灵活性！
#### 项目说明
| 项目名 | 说明 |
|--------------------|------------------------|
| `jeecg-boot` | 后端源码JAVA（SpringBoot微服务架构） |
| `jeecgboot-vue3` | 前端源码VUE3（vue3+vite5+ts最新技术栈） |
技术文档
-----------------------------------
- 官方网站： [http://www.jeecg.com](http://www.jeecg.com)
- 新手指南： [快速入门](http://www.jeecg.com/doc/quickstart)
- QQ交流群 ： ⑩716488839、⑨808791225、其他(满)
- 在线演示 ： [在线演示](http://boot3.jeecg.com) | [APP演示](http://jeecg.com/appIndex)
> 演示系统的登录账号密码，请点击 [获取账号密码](http://jeecg.com/doc/demo) 获取
启动项目
-----------------------------------
- [IDEA启动前后端项目](https://help.jeecg.com/java/setup/idea/startup)
- [Docker一键启动前后端](https://help.jeecg.com/java/docker/quick)
微服务启动
-----------------------------------
- [单体快速切换微服务](https://help.jeecg.com/java/springcloud/switchcloud/monomer)
- [Docker启动微服务后台](https://help.jeecg.com/java/docker/springcloud)
技术架构：
-----------------------------------
#### 后端
- IDE建议： IDEA (必须安装lombok插件 )
- 语言：Java 8+ (支持17)
- 依赖管理：Maven
- 基础框架：Spring Boot 2.7.18
- 微服务框架： Spring Cloud Alibaba 2021.0.1.0
- 持久层框架：MybatisPlus 3.5.3.2
- 报表工具： JimuReport 1.9.4
- 安全框架：Apache Shiro 1.12.0，Jwt 3.11.0
- 微服务技术栈：Spring Cloud Alibaba、Nacos、Gateway、Sentinel、Skywalking
- 数据库连接池：阿里巴巴Druid 1.1.24
- 日志打印：logback
- 缓存：Redis
- 其他：autopoi, fastjson，poi，Swagger-ui，quartz, lombok（简化代码）等。
- 默认数据库脚本：MySQL5.7+
- [其他数据库，需要自己转](https://my.oschina.net/jeecg/blog/4905722)
#### 前端
- 前端IDE建议：WebStorm、Vscode
- 采用 Vue3.0+TypeScript+Vite+Ant-Design-Vue等新技术方案，包括二次封装组件、utils、hooks、动态菜单、权限校验、按钮级别权限控制等功能
- 最新技术栈：Vue3.0 + TypeScript + Vite5 + ant-design-vue4 + pinia + echarts + unocss + vxe-table + qiankun + es6
- 依赖管理：node、npm、pnpm
#### 支持库
| 数据库 | 支持 |
| --- | --- |
| MySQL | √ |
| Oracle11g | √ |
| Sqlserver2017 | √ |
| PostgreSQL | √ |
| MariaDB | √ |
| 达梦 | √ |
| 人大金仓 | √ |
| TiDB | √ |
## 微服务解决方案
- 1、服务注册和发现 Nacos √
- 2、统一配置中心 Nacos √
- 3、路由网关 gateway(三种加载方式) √
- 4、分布式 http feign √
- 5、熔断降级限流 Sentinel √
- 6、分布式文件 Minio、阿里OSS √
- 7、统一权限控制 JWT + Shiro √
- 8、服务监控 SpringBootAdmin√
- 9、链路跟踪 Skywalking [参考文档](https://help.jeecg.com/java/springcloud/super/skywarking)
- 10、消息中间件 RabbitMQ √
- 11、分布式任务 xxl-job √
- 12、分布式事务 Seata
- 13、轻量分布式日志 Loki+grafana套件
- 14、支持 docker-compose、k8s、jenkins
- 15、CAS 单点登录 √
- 16、路由限流 √
后台目录结构
-----------------------------------
```
项目结构
├─jeecg-boot-parent（父POM： 项目依赖、modules组织）
│ ├─jeecg-boot-base-core（共通模块： 工具类、config、权限、查询过滤器、注解等）
│ ├─jeecg-module-demo 示例代码
│ ├─jeecg-module-system System系统管理目录
│ │ ├─jeecg-system-biz System系统管理权限等功能
│ │ ├─jeecg-system-start System单体启动项目(8080）
│ │ ├─jeecg-system-api System系统管理模块对外api
│ │ │ ├─jeecg-system-cloud-api System模块对外提供的微服务接口
│ │ │ ├─jeecg-system-local-api System模块对外提供的单体接口
│ ├─jeecg-server-cloud --微服务模块
├─jeecg-cloud-gateway --微服务网关模块(9999)
├─jeecg-cloud-nacos --Nacos服务模块(8848)
├─jeecg-system-cloud-start --System微服务启动项目(7001)
├─jeecg-demo-cloud-start --Demo微服务启动项目(7002)
├─jeecg-visual
├─jeecg-cloud-monitor --微服务监控模块 (9111)
├─jeecg-cloud-xxljob --微服务xxljob定时任务服务端 (9080)
├─jeecg-cloud-sentinel --sentinel服务端 (9000)
├─jeecg-cloud-test -- 微服务测试示例（各种例子）
├─jeecg-cloud-test-more -- 微服务测试示例（feign、熔断降级、xxljob、分布式锁）
├─jeecg-cloud-test-rabbitmq -- 微服务测试示例（rabbitmq）
├─jeecg-cloud-test-seata -- 微服务测试示例（seata分布式事务）
├─jeecg-cloud-test-shardingsphere -- 微服务测试示例（分库分表）
```
#### 微服务架构图
![微服务架构图](https://jeecgos.oss-cn-beijing.aliyuncs.com/files/jeecgboot_springcloud2022.png "在这里输入图片标题")

23
jeecg-boot-base-core/src/main/java/org/jeecg/common/constant/enums/FlexpartSpeciesType.java Normal file
View File

@ -0,0 +1,23 @@
package org.jeecg.common.constant.enums;
/**
* 物种类型
*/
public enum FlexpartSpeciesType {
XE_131m(51),
XE_133(52),
XE_133m(53),
XE_135(54),
NOT_SPECIES(61);
private Integer value;
FlexpartSpeciesType(int value) {
this.value = value;
}
public Integer getValue() {
return value;
}
}

23
jeecg-boot-base-core/src/main/java/org/jeecg/common/constant/enums/TransportReleaseDataSource.java Normal file
View File

@ -0,0 +1,23 @@
package org.jeecg.common.constant.enums;
public enum TransportReleaseDataSource {
/**
* 自动填写
*/
AUTO_SELECT(1),
/**
* 手动输入
*/
MANUAL_ENTRY(2);
private Integer key;
TransportReleaseDataSource(Integer key) {
this.key = key;
}
public Integer getKey(){
return this.key;
}
}

16
jeecg-boot-base-core/src/main/java/org/jeecg/common/constant/enums/TransportSampleTypeEnum.java Normal file
View File

@ -0,0 +1,16 @@
package org.jeecg.common.constant.enums;
/**
* 输运任务样品类型说明枚举
*/
public enum TransportSampleTypeEnum {
/**
* 正向
*/
B,
/**
* 反向
*/
P;
}

6
jeecg-boot-base-core/src/main/java/org/jeecg/common/properties/DataFusionProperties.java
View File

@ -20,7 +20,11 @@ public class DataFusionProperties {
/** /**
* srs文件的上级目录有可能是flexpart.x.ecmwf.l1或flexpart.x.ncep.l1 * srs文件的上级目录有可能是flexpart.x.ecmwf.l1或flexpart.x.ncep.l1
*/ */
private String srmParentDir; private String idcSrmParentDir;
/**
* srs文件的上级目录有可能是flexpart.x.ecmwf.l1或flexpart.x.ncep.l1
*/
private String ndcSrmParentDir;
/** /**
* 浓度值过滤条件 * 浓度值过滤条件
*/ */

19
jeecg-boot-base-core/src/main/java/org/jeecg/common/properties/ServerProperties.java Normal file
View File

@ -0,0 +1,19 @@
package org.jeecg.common.properties;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
@Data
@Component
@ConfigurationProperties(prefix = "server-prop")
public class ServerProperties {
private String host;
private int port;
private String username;
private String password;
}

7
jeecg-boot-base-core/src/main/java/org/jeecg/common/properties/SystemStorageProperties.java
View File

@ -60,10 +60,15 @@ public class SystemStorageProperties {
private String formatScriptPythonEnv; private String formatScriptPythonEnv;
/** /**
* ai-models 安装地址 * pangu的ai-models 安装地址
*/ */
private String panguEnvPath; private String panguEnvPath;
/**
* graphcast的ai-models 安装地址
*/
private String graphcastEnvPath;
/** /**
* 盘古模型执行路径 * 盘古模型执行路径
*/ */

15
jeecg-boot-base-core/src/main/java/org/jeecg/common/properties/TransportSimulationProperties.java
View File

@ -9,11 +9,26 @@ import org.springframework.stereotype.Component;
@ConfigurationProperties(prefix = "transport-simulation") @ConfigurationProperties(prefix = "transport-simulation")
public class TransportSimulationProperties { public class TransportSimulationProperties {
/**
* 颗粒物样品大气输运级别
*/
private Integer transportLevel_p;
/**
* 惰性气体样品输运状态
*/
private Integer transportLevel_x;
/** /**
* 模型输出路径 * 模型输出路径
*/ */
private String outputPath; private String outputPath;
/**
* 正演各站点排放数据存储目录
*/
private String inputSiteHourPath;
/** /**
* 参数配置文件路径（和fnl气象数据有关系） * 参数配置文件路径（和fnl气象数据有关系）
*/ */

38
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/entity/TransportTask.java
View File

@ -5,7 +5,6 @@ import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName; import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonFormat; import com.fasterxml.jackson.annotation.JsonFormat;
import jakarta.validation.Valid;
import jakarta.validation.constraints.NotBlank; import jakarta.validation.constraints.NotBlank;
import jakarta.validation.constraints.NotNull; import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Null; import jakarta.validation.constraints.Null;
@ -13,6 +12,7 @@ import lombok.Data;
import org.jeecg.common.validgroup.InsertGroup; import org.jeecg.common.validgroup.InsertGroup;
import org.jeecg.common.validgroup.UpdateGroup; import org.jeecg.common.validgroup.UpdateGroup;
import org.springframework.format.annotation.DateTimeFormat; import org.springframework.format.annotation.DateTimeFormat;
import java.time.LocalDateTime;
import java.util.Date; import java.util.Date;
import java.util.List; import java.util.List;
@ -90,7 +90,7 @@ public class TransportTask{
* 创建时间 * 创建时间
*/ */
@TableField(value = "create_time") @TableField(value = "create_time")
@JsonFormat(timezone = "GMT+8", pattern = "yyyy-MM-dd HH:mm:ss") @JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss") @DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private Date createTime; private Date createTime;
@ -104,7 +104,7 @@ public class TransportTask{
* 更新时间 * 更新时间
*/ */
@TableField(value = "update_time") @TableField(value = "update_time")
@JsonFormat(timezone = "GMT+8", pattern = "yyyy-MM-dd HH:mm:ss") @JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss") @DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private Date updateTime; private Date updateTime;
@ -112,15 +112,19 @@ public class TransportTask{
* 模拟开始时间 * 模拟开始时间
*/ */
@NotNull(message = "模拟开始时间不能为空",groups = {InsertGroup.class, UpdateGroup.class}) @NotNull(message = "模拟开始时间不能为空",groups = {InsertGroup.class, UpdateGroup.class})
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
@TableField(value = "start_time") @TableField(value = "start_time")
private Date startTime; private LocalDateTime startTime;
/** /**
* 模拟结束时间 * 模拟结束时间
*/ */
@NotNull(message = "模拟结束时间不能为空",groups = {InsertGroup.class, UpdateGroup.class}) @NotNull(message = "模拟结束时间不能为空",groups = {InsertGroup.class, UpdateGroup.class})
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
@TableField(value = "end_time") @TableField(value = "end_time")
private Date endTime; private LocalDateTime endTime;
/** /**
* 释放下部高度 * 释放下部高度
@ -143,7 +147,27 @@ public class TransportTask{
@TableField(value = "particle_count") @TableField(value = "particle_count")
private Integer particleCount; private Integer particleCount;
@Valid /**
* 正演释放数据来源 1自动填写,2手动输入
*/
@TableField(value = "release_data_source")
private Integer releaseDataSource;
/**
* 反演子表信息
*/
@TableField(exist = false) @TableField(exist = false)
List<TransportTaskChild> childList; private List<TransportTaskBackwardChild> backwardChild;
/**
* 正演子表信息
*/
@TableField(exist = false)
private List<TransportTaskForwardChild> forwardChild;
/**
* 正演物种信息
*/
@TableField(exist = false)
private List<TransportTaskForwardSpecies> species;
} }

58
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/entity/TransportTaskChild.java → jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/entity/TransportTaskBackwardChild.java
View File

@ -5,75 +5,87 @@ import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId; import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName; import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonFormat; import com.fasterxml.jackson.annotation.JsonFormat;
import jakarta.validation.constraints.NotBlank;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Null;
import lombok.Data; import lombok.Data;
import org.jeecg.common.validgroup.InsertGroup;
import org.jeecg.common.validgroup.UpdateGroup;
import org.springframework.format.annotation.DateTimeFormat; import org.springframework.format.annotation.DateTimeFormat;
import java.time.LocalDateTime;
import java.util.Date; import java.util.Date;
/** /**
* 输运模拟任务子表 * 输运模拟反演任务配置子表
*/ */
@Data @Data
@TableName("stas_transport_task_child") @TableName("stas_transport_task_backward_child")
public class TransportTaskChild{ public class TransportTaskBackwardChild {
/** /**
* ID * ID
*/ */
@Null(message = "ID必须为空",groups = { InsertGroup.class})
@TableId(type = IdType.AUTO) @TableId(type = IdType.AUTO)
private Integer id; private Integer id;
/** /**
* 任务ID * 任务ID
*/ */
@Null(message = "任务id必须为空",groups = {InsertGroup.class, UpdateGroup.class})
@TableField(value = "task_id") @TableField(value = "task_id")
private Integer taskId; private Integer taskId;
/** /**
* 台站编目 * 台站编目
*/ */
@NotBlank(message = "台站编码不能为空",groups = {InsertGroup.class, UpdateGroup.class})
@TableField(value = "station_code") @TableField(value = "station_code")
private String stationCode; private String stationCode;
/** /**
* 经度1 * 经度
*/ */
@NotNull(message = "经度不能为空",groups = {InsertGroup.class, UpdateGroup.class})
@TableField(value = "lon") @TableField(value = "lon")
private Double lon; private Double lon;
/** /**
* 纬度1 * 纬度
*/ */
@NotNull(message = "经度不能为空",groups = {InsertGroup.class, UpdateGroup.class})
@TableField(value = "lat") @TableField(value = "lat")
private Double lat; private Double lat;
/** /**
* 释放量 * 释放量
*/ */
@NotNull(message = "释放量不能为空",groups = {InsertGroup.class, UpdateGroup.class})
@TableField(value = "release_amount") @TableField(value = "release_amount")
private String releaseAmount; private String releaseAmount;
/**
* 创建人
*/
@TableField(value = "create_by")
private String createBy;
/** /**
* 创建时间 * 创建时间
*/ */
@TableField(value = "create_time") @TableField(value = "create_time")
@JsonFormat(timezone = "GMT+8", pattern = "yyyy-MM-dd HH:mm:ss") @JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss") @DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private Date createTime; private Date createTime;
/**
* 样品id
*/
@TableField(value = "sample_id")
private Integer sampleId;
/**
* 样品类型
*/
@TableField(value = "sample_type")
private String sampleType;
/**
* 开始测量时间
*/
@TableField(value = "acq_start_time")
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private LocalDateTime acqStartTime;
/**
* 结束测量时间
*/
@TableField(value = "acq_end_time")
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private LocalDateTime acqEndTime;
} }

64
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/entity/TransportTaskForwardChild.java Normal file
View File

@ -0,0 +1,64 @@
package org.jeecg.modules.base.entity;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonFormat;
import lombok.Data;
import org.springframework.format.annotation.DateTimeFormat;
import java.util.Date;
import java.util.List;
/**
* 输运模拟正演任务配置子表
*/
@Data
@TableName("stas_transport_task_forward_child")
public class TransportTaskForwardChild {
/**
* ID
*/
@TableId(type = IdType.AUTO)
private Integer id;
/**
* 任务ID
*/
@TableField(value = "task_id")
private Integer taskId;
/**
* 台站编目
*/
@TableField(value = "station_code")
private String stationCode;
/**
* 经度
*/
@TableField(value = "lon")
private Double lon;
/**
* 纬度
*/
@TableField(value = "lat")
private Double lat;
/**
* 创建时间
*/
@TableField(value = "create_time")
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private Date createTime;
/**
* 正演子表信息
*/
@TableField(exist = false)
private List<TransportTaskForwardRelease> forwardReleaseChild;
}

68
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/entity/TransportTaskForwardRelease.java Normal file
View File

@ -0,0 +1,68 @@
package org.jeecg.modules.base.entity;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonFormat;
import lombok.Data;
import org.springframework.format.annotation.DateTimeFormat;
import java.time.LocalDateTime;
import java.util.Date;
/**
* 输运模拟正演任务配置子表
*/
@Data
@TableName("stas_transport_task_forward_release")
public class TransportTaskForwardRelease {
/**
* ID
*/
@TableId(type = IdType.AUTO)
private Integer id;
/**
* 任务ID
*/
@TableField(value = "task_id")
private Integer taskId;
/**
* 任务ID
*/
@TableField(value = "forward_child_id")
private Integer forwardChildId;
/**
* 开始时间
*/
@TableField(value = "start_time")
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private LocalDateTime startTime;
/**
* 结束时间
*/
@TableField(value = "end_time")
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private LocalDateTime endTime;
/**
* 释放量
*/
@TableField(value = "release_amount")
private String releaseAmount;
/**
* 创建时间
*/
@TableField(value = "create_time")
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private Date createTime;
}

37
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/entity/TransportTaskForwardSpecies.java Normal file
View File

@ -0,0 +1,37 @@
package org.jeecg.modules.base.entity;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.annotation.JsonFormat;
import lombok.Data;
import org.springframework.format.annotation.DateTimeFormat;
import java.util.Date;
/**
* 输运模拟正演任务配置子表
*/
@Data
@TableName("stas_transport_task_forward_species")
public class TransportTaskForwardSpecies {
/**
* 任务ID
*/
@TableField(value = "task_id")
private Integer taskId;
/**
* 物种id
*/
@TableField(value = "species_id")
private String speciesId;
/**
* 创建时间
*/
@TableField(value = "create_time")
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
private Date createTime;
}

7
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/mapper/TransportTaskBackwardChildMapper.java Normal file
View File

@ -0,0 +1,7 @@
package org.jeecg.modules.base.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.jeecg.modules.base.entity.TransportTaskBackwardChild;
public interface TransportTaskBackwardChildMapper extends BaseMapper<TransportTaskBackwardChild> {
}

7
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/mapper/TransportTaskChildMapper.java
View File

@ -1,7 +0,0 @@
package org.jeecg.modules.base.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.jeecg.modules.base.entity.TransportTaskChild;
public interface TransportTaskChildMapper extends BaseMapper<TransportTaskChild> {
}

7
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/mapper/TransportTaskForwardChildMapper.java Normal file
View File

@ -0,0 +1,7 @@
package org.jeecg.modules.base.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.jeecg.modules.base.entity.TransportTaskForwardChild;
public interface TransportTaskForwardChildMapper extends BaseMapper<TransportTaskForwardChild> {
}

7
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/mapper/TransportTaskForwardReleaseMapper.java Normal file
View File

@ -0,0 +1,7 @@
package org.jeecg.modules.base.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.jeecg.modules.base.entity.TransportTaskForwardRelease;
public interface TransportTaskForwardReleaseMapper extends BaseMapper<TransportTaskForwardRelease> {
}

8
jeecg-boot-base-core/src/main/java/org/jeecg/modules/base/mapper/TransportTaskForwardSpeciesMapper.java Normal file
View File

@ -0,0 +1,8 @@
package org.jeecg.modules.base.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.jeecg.modules.base.entity.TransportTaskForwardSpecies;
public interface TransportTaskForwardSpeciesMapper extends BaseMapper<TransportTaskForwardSpecies> {
}

4
jeecg-module-large-screen/src/main/java/org/jeecg/sample/mapper/IMSSampleAnalysesMapper.java
View File

@ -1,6 +1,7 @@
package org.jeecg.sample.mapper; package org.jeecg.sample.mapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper; import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Param;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
@ -23,5 +24,6 @@ public interface IMSSampleAnalysesMapper extends BaseMapper {
* 查询待输运的样品列表 * 查询待输运的样品列表
* @return * @return
*/ */
List<Map<String, Object>> getSamplesTransportList(); List<Map<String, Object>> getSamplesTransportList(@Param("transportLevel_p") Integer transportLevel_p,
@Param("transportLevel_x") Integer transportLevel_x);
} }

4
jeecg-module-large-screen/src/main/java/org/jeecg/sample/mapper/xml/IMSSampleAnalysesMapper.xml
View File

@ -40,9 +40,9 @@
INNER JOIN CONFIGURATION.GARDS_STATIONS gst on gsd.STATION_ID = gst.STATION_ID INNER JOIN CONFIGURATION.GARDS_STATIONS gst on gsd.STATION_ID = gst.STATION_ID
WHERE CLOSE_STATUS = 0 WHERE CLOSE_STATUS = 0
and ( and (
(gsd.SAMPLE_TYPE = 'P' AND ga.CATEGORY IN (4, 5)) (gsd.SAMPLE_TYPE = 'P' AND ga.CATEGORY > #{transportLevel_p})
or or
(gsd.SAMPLE_TYPE = 'B' AND ga.CATEGORY IN (3)) (gsd.SAMPLE_TYPE = 'B' AND ga.CATEGORY > #{transportLevel_x})
) )
ORDER BY gts.MODDATE desc ORDER BY gts.MODDATE desc
</select> </select>

4
jeecg-module-large-screen/src/main/java/org/jeecg/sample/service/impl/IMSSampleAnalysesServiceImpl.java
View File

@ -3,6 +3,7 @@ package org.jeecg.sample.service.impl;
import com.baomidou.dynamic.datasource.annotation.DS; import com.baomidou.dynamic.datasource.annotation.DS;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.jeecg.common.constant.enums.TransportTaskCloseStatusEnum; import org.jeecg.common.constant.enums.TransportTaskCloseStatusEnum;
import org.jeecg.common.properties.TransportSimulationProperties;
import org.jeecg.modules.base.entity.rnauto.GardsTransportStatus; import org.jeecg.modules.base.entity.rnauto.GardsTransportStatus;
import org.jeecg.modules.base.mapper.GardsTransportStatusMapper; import org.jeecg.modules.base.mapper.GardsTransportStatusMapper;
import org.jeecg.sample.mapper.IMSSampleAnalysesMapper; import org.jeecg.sample.mapper.IMSSampleAnalysesMapper;
@ -18,6 +19,7 @@ public class IMSSampleAnalysesServiceImpl implements IMSSampleAnalysesService {
private final IMSSampleAnalysesMapper sampleAnalysesMapper; private final IMSSampleAnalysesMapper sampleAnalysesMapper;
private final GardsTransportStatusMapper transportStatusMapper; private final GardsTransportStatusMapper transportStatusMapper;
private final TransportSimulationProperties transportSimulationProperties;
/** /**
* 统计气溶胶样品数量 * 统计气溶胶样品数量
@ -46,7 +48,7 @@ public class IMSSampleAnalysesServiceImpl implements IMSSampleAnalysesService {
@DS("ora") @DS("ora")
@Override @Override
public List<Map<String, Object>> getSamplesTransportList() { public List<Map<String, Object>> getSamplesTransportList() {
return sampleAnalysesMapper.getSamplesTransportList(); return sampleAnalysesMapper.getSamplesTransportList(transportSimulationProperties.getTransportLevel_p(),transportSimulationProperties.getTransportLevel_x());
} }
/** /**

4
jeecg-module-transport/src/main/java/org/jeecg/controller/TransportResultDataController.java
View File

@ -20,8 +20,8 @@ public class TransportResultDataController {
private final TransportResultDataService transportResultDataService; private final TransportResultDataService transportResultDataService;
@AutoLog(value = "分页查询输运任务数据") @AutoLog(value = "获取扩散数据")
@Operation(summary = "分页查询输运任务数据") @Operation(summary = "获取扩散数据")
@GetMapping("getDiffusionData") @GetMapping("getDiffusionData")
public Result<?> getDiffusionData(@Validated(value= QueryGroup.class) QueryDiffusionVO queryDiffusionVO) { public Result<?> getDiffusionData(@Validated(value= QueryGroup.class) QueryDiffusionVO queryDiffusionVO) {
return Result.OK(transportResultDataService.getDiffusionData(queryDiffusionVO)); return Result.OK(transportResultDataService.getDiffusionData(queryDiffusionVO));

692
jeecg-module-transport/src/main/java/org/jeecg/service/impl/TransportResultDataServiceImpl.java
View File

@ -2,28 +2,25 @@ package org.jeecg.service.impl;
import cn.hutool.core.collection.CollUtil; import cn.hutool.core.collection.CollUtil;
import cn.hutool.core.date.DateUtil; import cn.hutool.core.date.DateUtil;
import cn.hutool.core.date.LocalDateTimeUtil;
import cn.hutool.core.io.FileUtil; import cn.hutool.core.io.FileUtil;
import cn.hutool.core.util.StrUtil;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper; import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.jeecg.common.constant.CommonConstant; import org.jeecg.common.constant.CommonConstant;
import org.jeecg.common.constant.enums.TransportTaskModeEnum; import org.jeecg.common.constant.enums.TransportTaskModeEnum;
import org.jeecg.common.constant.enums.TransportTimingAnalysisEnum;
import org.jeecg.common.properties.TransportSimulationProperties; import org.jeecg.common.properties.TransportSimulationProperties;
import org.jeecg.common.util.NcUtil; import org.jeecg.common.util.NcUtil;
import org.jeecg.common.util.RedisUtil; import org.jeecg.common.util.RedisUtil;
import org.jeecg.modules.base.entity.TransportTask; import org.jeecg.modules.base.entity.TransportTask;
import org.jeecg.modules.base.entity.TransportTaskChild; import org.jeecg.modules.base.entity.TransportTaskBackwardChild;
import org.jeecg.modules.base.entity.configuration.GardsNuclearReactors; import org.jeecg.modules.base.entity.TransportTaskForwardChild;
import org.jeecg.modules.base.entity.configuration.GardsStations; import org.jeecg.modules.base.entity.configuration.GardsStations;
import org.jeecg.modules.base.mapper.TransportTaskChildMapper; import org.jeecg.modules.base.mapper.TransportTaskBackwardChildMapper;
import org.jeecg.modules.base.mapper.TransportTaskForwardChildMapper;
import org.jeecg.modules.base.mapper.TransportTaskMapper; import org.jeecg.modules.base.mapper.TransportTaskMapper;
import org.jeecg.service.StationDataService; import org.jeecg.service.StationDataService;
import org.jeecg.service.TransportResultDataService; import org.jeecg.service.TransportResultDataService;
import org.jeecg.util.BilinearInterpolatorWithMath; import org.jeecg.util.BilinearInterpolatorWithMath;
import org.jeecg.vo.ConcModValVo;
import org.jeecg.vo.ContributionAnalysisVO; import org.jeecg.vo.ContributionAnalysisVO;
import org.jeecg.vo.QueryDiffusionVO; import org.jeecg.vo.QueryDiffusionVO;
import org.jeecg.vo.TaskStationsVO; import org.jeecg.vo.TaskStationsVO;
@ -33,15 +30,10 @@ import ucar.ma2.DataType;
import ucar.ma2.Index; import ucar.ma2.Index;
import ucar.ma2.InvalidRangeException; import ucar.ma2.InvalidRangeException;
import ucar.nc2.Attribute; import ucar.nc2.Attribute;
import ucar.nc2.Dimension;
import ucar.nc2.NetcdfFile; import ucar.nc2.NetcdfFile;
import ucar.nc2.Variable; import ucar.nc2.Variable;
import java.io.File; import java.io.File;
import java.io.IOException; import java.io.IOException;
import java.math.BigDecimal;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.util.*; import java.util.*;
import java.util.stream.Collectors; import java.util.stream.Collectors;
@ -52,7 +44,8 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
private final TransportTaskMapper transportTaskMapper; private final TransportTaskMapper transportTaskMapper;
private final TransportSimulationProperties simulationProperties; private final TransportSimulationProperties simulationProperties;
private final TransportTaskChildMapper transportTaskChildMapper; private final TransportTaskBackwardChildMapper backwardChildMapper;
private final TransportTaskForwardChildMapper forwardChildMapper;
private final StationDataService stationDataService; private final StationDataService stationDataService;
private final RedisUtil redisUtil; private final RedisUtil redisUtil;
private final static String FORWARD="forward"; private final static String FORWARD="forward";
@ -123,18 +116,18 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
*/ */
private List<Map<String,Object>> getBackwardData(TransportTask transportTask,QueryDiffusionVO queryDiffusionVO){ private List<Map<String,Object>> getBackwardData(TransportTask transportTask,QueryDiffusionVO queryDiffusionVO){
//验证任务子信息 //验证任务子信息
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>(); LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskChild::getTaskId,queryDiffusionVO.getTaskId()); queryWrapper.eq(TransportTaskBackwardChild::getTaskId,queryDiffusionVO.getTaskId());
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode); queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
queryWrapper.orderByAsc(TransportTaskChild::getId); queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper); List<TransportTaskBackwardChild> transportTaskChildren = backwardChildMapper.selectList(queryWrapper);
if(CollUtil.isEmpty(transportTaskChildren)){ if(CollUtil.isEmpty(transportTaskChildren)){
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息"); throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
} }
List<Map<String,Object>> result = new ArrayList<>(); List<Map<String,Object>> result = new ArrayList<>();
for(Integer pointNum : queryDiffusionVO.getPointspecs()){ for(Integer pointNum : queryDiffusionVO.getPointspecs()){
//获取nc文件路径 //获取nc文件路径
TransportTaskChild transportTaskChild = transportTaskChildren.get(pointNum); TransportTaskBackwardChild transportTaskChild = transportTaskChildren.get(pointNum);
String path = this.getBackForwardTaskNCPath(transportTask,transportTaskChild.getStationCode()); String path = this.getBackForwardTaskNCPath(transportTask,transportTaskChild.getStationCode());
try (NetcdfFile ncFile = NetcdfFile.open(path.toString())) { try (NetcdfFile ncFile = NetcdfFile.open(path.toString())) {
List<Double> lonData = NcUtil.getNCList(ncFile, "longitude"); List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
@ -175,28 +168,22 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
if(Objects.isNull(transportTask)){ if(Objects.isNull(transportTask)){
throw new RuntimeException("此任务不存在"); throw new RuntimeException("此任务不存在");
} }
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskChild::getTaskId,taskId); List<String> stationCodes = this.getStationCodes(transportTask.getId(),transportTask.getTaskMode());
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode);
queryWrapper.orderByAsc(TransportTaskChild::getId);
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
if(CollUtil.isEmpty(transportTaskChildren)){
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
}
//获取nc文件路径 //获取nc文件路径
String path = ""; String path = "";
if(TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())){ if(TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())){
path = this.getForwardTaskNCPath(transportTask); path = this.getForwardTaskNCPath(transportTask);
}else if(TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())){ }else if(TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())){
path = this.getBackForwardTaskNCPath(transportTask,transportTaskChildren.get(0).getStationCode()); path = this.getBackForwardTaskNCPath(transportTask,stationCodes.get(0));
} }
if(!FileUtil.exist(path)){ if(!FileUtil.exist(path)){
throw new RuntimeException("此任务模拟结果不存在，请确认任务运行状态"); throw new RuntimeException("此任务模拟结果不存在，请确认任务运行状态");
} }
//本任务模拟的台站数据 //本任务模拟的台站数据
Map<Integer,Object> stationNumMap = new LinkedHashMap<>(); Map<Integer,Object> stationNumMap = new LinkedHashMap<>();
for (int i = 0; i<transportTaskChildren.size();i++) { for (int i = 0; i<stationCodes.size();i++) {
stationNumMap.put((i),transportTaskChildren.get(i).getStationCode()); stationNumMap.put((i),stationCodes.get(i));
} }
resultMap.put("stationNum",stationNumMap); resultMap.put("stationNum",stationNumMap);
//高度数据 //高度数据
@ -231,6 +218,38 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
return resultMap; return resultMap;
} }
/**
* 获取本次模拟台站信息
* @param taskId
* @param taskMode
* @return
*/
private List<String> getStationCodes(Integer taskId,Integer taskMode) {
List<String> stationCodes = new ArrayList<>();
if(TransportTaskModeEnum.FORWARD.getKey().equals(taskMode)){
LambdaQueryWrapper<TransportTaskForwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskForwardChild::getTaskId,taskId);
queryWrapper.select(TransportTaskForwardChild::getId, TransportTaskForwardChild::getStationCode);
queryWrapper.orderByAsc(TransportTaskForwardChild::getId);
List<TransportTaskForwardChild> transportTaskChildren = forwardChildMapper.selectList(queryWrapper);
if(CollUtil.isEmpty(transportTaskChildren)){
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
}
transportTaskChildren.forEach(taskChild -> stationCodes.add(taskChild.getStationCode()));
}else if(TransportTaskModeEnum.BACK_FORWARD.getKey().equals(taskMode)){
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
List<TransportTaskBackwardChild> transportTaskChildren = backwardChildMapper.selectList(queryWrapper);
if(CollUtil.isEmpty(transportTaskChildren)){
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
}
transportTaskChildren.forEach(taskChild -> stationCodes.add(taskChild.getStationCode()));
}
return stationCodes;
}
/** /**
* 查询符合度分析数据 * 查询符合度分析数据
* @param taskId * @param taskId
@ -241,99 +260,100 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
*/ */
@Override @Override
public List<Map<String, Object>> getConformityAnalysis(Integer taskId,Integer stationId,String nuclideName,String facilityName) { public List<Map<String, Object>> getConformityAnalysis(Integer taskId,Integer stationId,String nuclideName,String facilityName) {
TransportTask transportTask = transportTaskMapper.selectById(taskId); // TransportTask transportTask = transportTaskMapper.selectById(taskId);
if(Objects.isNull(transportTask)){ // if(Objects.isNull(transportTask)){
throw new RuntimeException("此任务不存在"); // throw new RuntimeException("此任务不存在");
} // }
//
List<Map<String, Object>> xeResults = stationDataService.getXeResults(stationId, transportTask.getStartTime(), transportTask.getEndTime(), nuclideName); // List<Map<String, Object>> xeResults = stationDataService.getXeResults(stationId, transportTask.getStartTime(), transportTask.getEndTime(), nuclideName);
//获取nc文件路径 // //获取nc文件路径
String path = this.getForwardTaskNCPath(transportTask); // String path = this.getForwardTaskNCPath(transportTask);
try (NetcdfFile ncFile = NetcdfFile.open(path.toString())) { // try (NetcdfFile ncFile = NetcdfFile.open(path.toString())) {
int pointNum = -1; // int pointNum = -1;
int maxPointNum = -1; // int maxPointNum = -1;
if (StrUtil.isNotBlank(facilityName)) { // if (StrUtil.isNotBlank(facilityName)) {
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>(); // LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskChild::getTaskId,taskId); // queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode); // queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
queryWrapper.orderByAsc(TransportTaskChild::getId); // queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper); // List<TransportTaskBackwardChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
for(int i=0;i<transportTaskChildren.size();i++){ // for(int i=0;i<transportTaskChildren.size();i++){
if (transportTaskChildren.get(i).getStationCode().equals(facilityName)){ // if (transportTaskChildren.get(i).getStationCode().equals(facilityName)){
pointNum = i; // pointNum = i;
maxPointNum = i; // maxPointNum = i;
} // }
} // }
}else{ // }else{
Dimension pointspec = ncFile.findDimension("pointspec"); // Dimension pointspec = ncFile.findDimension("pointspec");
maxPointNum = pointspec.getLength() -1; // maxPointNum = pointspec.getLength() -1;
} // }
//
//找到最接近台站经纬度的值，作为对比点位 // //找到最接近台站经纬度的值，作为对比点位
List<Double> lonData = NcUtil.getNCList(ncFile, "longitude"); // List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
List<Double> latData = NcUtil.getNCList(ncFile, "latitude"); // List<Double> latData = NcUtil.getNCList(ncFile, "latitude");
List<Double> timeData = NcUtil.getNCList(ncFile, "time"); // List<Double> timeData = NcUtil.getNCList(ncFile, "time");
GardsStations station = this.stationDataService.getStationById(stationId); // GardsStations station = this.stationDataService.getStationById(stationId);
Map<String,Object> lonRecentValMap = this.getRecentValue(lonData, station.getLon()); // Map<String,Object> lonRecentValMap = this.getRecentValue(lonData, station.getLon());
Map<String,Object> latRecentValMap = this.getRecentValue(latData, station.getLat()); // Map<String,Object> latRecentValMap = this.getRecentValue(latData, station.getLat());
Integer lonBestIndex = Integer.parseInt(lonRecentValMap.get("bestIndex").toString()); // Integer lonBestIndex = Integer.parseInt(lonRecentValMap.get("bestIndex").toString());
Integer latBestIndex = Integer.parseInt(latRecentValMap.get("bestIndex").toString()); // Integer latBestIndex = Integer.parseInt(latRecentValMap.get("bestIndex").toString());
//
Variable spec001Mr = ncFile.findVariable("spec001_mr"); // Variable spec001Mr = ncFile.findVariable("spec001_mr");
List<ConcModValVo> modValList = new ArrayList<>(); // List<ConcModValVo> modValList = new ArrayList<>();
for(int i=pointNum;i<=maxPointNum;i++){ // for(int i=pointNum;i<=maxPointNum;i++){
for(int k=0;k<timeData.size();k++){ // for(int k=0;k<timeData.size();k++){
//nageclass=1, pointspec=1, time=30, height=6, latitude=710, longitude=1430 // //nageclass=1, pointspec=1, time=30, height=6, latitude=710, longitude=1430
int[] origin = {0, i,k,0, latBestIndex,lonBestIndex}; // int[] origin = {0, i,k,0, latBestIndex,lonBestIndex};
int[] section = {1, 1,1,1,1,1}; // int[] section = {1, 1,1,1,1,1};
Array levelData = spec001Mr.read(origin,section); // Array levelData = spec001Mr.read(origin,section);
double[] pointData = (double[]) levelData.get1DJavaArray(DataType.DOUBLE); // double[] pointData = (double[]) levelData.get1DJavaArray(DataType.DOUBLE);
double pointDataVal = pointData[0]; // double pointDataVal = pointData[0];
if(pointDataVal > 0){ // if(pointDataVal > 0){
Instant instant = transportTask.getStartTime().toInstant(); // Instant instant = transportTask.getStartTime().toInstant();
LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault()); // LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());
localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue()); // localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue());
long second = localDateTime.atZone(ZoneId.systemDefault()).toEpochSecond(); // long second = localDateTime.atZone(ZoneId.systemDefault()).toEpochSecond();
ConcModValVo concModValVo = new ConcModValVo(); // ConcModValVo concModValVo = new ConcModValVo();
concModValVo.setSecond(second); // concModValVo.setSecond(second);
concModValVo.setPointDataVal(pointDataVal); // concModValVo.setPointDataVal(pointDataVal);
modValList.add(concModValVo); // modValList.add(concModValVo);
}else { // }else {
ConcModValVo concModValVo = new ConcModValVo(); // ConcModValVo concModValVo = new ConcModValVo();
concModValVo.setSecond(0L); // concModValVo.setSecond(0L);
concModValVo.setPointDataVal(pointDataVal); // concModValVo.setPointDataVal(pointDataVal);
modValList.add(concModValVo); // modValList.add(concModValVo);
} // }
} // }
} // }
if(CollUtil.isNotEmpty(modValList)){ // if(CollUtil.isNotEmpty(modValList)){
xeResults.parallelStream().forEach(resultMap -> { // xeResults.parallelStream().forEach(resultMap -> {
Date acqStart = (Date)resultMap.get("acquisitionStart"); // Date acqStart = (Date)resultMap.get("acquisitionStart");
Date acqEnd = (Date)resultMap.get("acquisitionStop"); // Date acqEnd = (Date)resultMap.get("acquisitionStop");
int acqStartSecond = (int)(acqStart.getTime()/1000); // int acqStartSecond = (int)(acqStart.getTime()/1000);
int acqStopSecond = (int)(acqEnd.getTime()/1000); // int acqStopSecond = (int)(acqEnd.getTime()/1000);
int count = 0; // int count = 0;
Double sumVodValue = 0D; // Double sumVodValue = 0D;
for (ConcModValVo modVal : modValList){ // for (ConcModValVo modVal : modValList){
if (modVal.getSecond() >= acqStartSecond && modVal.getSecond() <= acqStopSecond) { // if (modVal.getSecond() >= acqStartSecond && modVal.getSecond() <= acqStopSecond) {
sumVodValue += modVal.getPointDataVal(); // sumVodValue += modVal.getPointDataVal();
count +=1; // count +=1;
} // }
} // }
if (sumVodValue != 0){ // if (sumVodValue != 0){
//纳克转换为毫克需除以1000000 // //纳克转换为毫克需除以1000000
BigDecimal vodValue = new BigDecimal(sumVodValue); // BigDecimal vodValue = new BigDecimal(sumVodValue);
BigDecimal finalVodValue = vodValue.divide(new BigDecimal(count)).divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP); // BigDecimal finalVodValue = vodValue.divide(new BigDecimal(count)).divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP);
resultMap.put("modValue", finalVodValue); // resultMap.put("modValue", finalVodValue);
}else { // }else {
resultMap.put("modValue", sumVodValue); // resultMap.put("modValue", sumVodValue);
} // }
}); // });
} // }
} catch (IOException | InvalidRangeException e) { // } catch (IOException | InvalidRangeException e) {
throw new RuntimeException(e); // throw new RuntimeException(e);
} // }
return xeResults; // return xeResults;
return null;
} }
/** /**
@ -344,12 +364,12 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
*/ */
@Override @Override
public List<String> getTaskFacility(Integer taskId) { public List<String> getTaskFacility(Integer taskId) {
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>(); LambdaQueryWrapper<TransportTaskForwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskChild::getTaskId,taskId); queryWrapper.eq(TransportTaskForwardChild::getTaskId,taskId);
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode); queryWrapper.select(TransportTaskForwardChild::getId, TransportTaskForwardChild::getStationCode);
queryWrapper.orderByAsc(TransportTaskChild::getId); queryWrapper.orderByAsc(TransportTaskForwardChild::getId);
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper); List<TransportTaskForwardChild> transportTaskChildren = forwardChildMapper.selectList(queryWrapper);
return transportTaskChildren.stream().map(TransportTaskChild::getStationCode).collect(Collectors.toList()); return transportTaskChildren.stream().map(TransportTaskForwardChild::getStationCode).collect(Collectors.toList());
} }
/** /**
@ -378,104 +398,104 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
*/ */
@Override @Override
public void handleContributionAnalysis(Integer taskId) { public void handleContributionAnalysis(Integer taskId) {
//查询任务数据 // //查询任务数据
TransportTask transportTask = transportTaskMapper.selectById(taskId); // TransportTask transportTask = transportTaskMapper.selectById(taskId);
//查询需要处理贡献分析数据的台站 // //查询需要处理贡献分析数据的台站
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>(); // LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskChild::getTaskId,taskId); // queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
List<TransportTaskChild> stationInfos = this.transportTaskChildMapper.selectList(queryWrapper); // List<TransportTaskBackwardChild> stationInfos = this.transportTaskChildMapper.selectList(queryWrapper);
//
//所以核设施数据 // //所以核设施数据
List<GardsNuclearReactors> facilitys = (List<GardsNuclearReactors>) redisUtil.get(CommonConstant.ALL_NUCLEARFACILITY); // List<GardsNuclearReactors> facilitys = (List<GardsNuclearReactors>) redisUtil.get(CommonConstant.ALL_NUCLEARFACILITY);
//
NetcdfFile ncFile = null; // NetcdfFile ncFile = null;
try { // try {
for(TransportTaskChild stationInfo :stationInfos) { // for(TransportTaskBackwardChild stationInfo :stationInfos) {
//每个台站的结果数据 // //每个台站的结果数据
ContributionAnalysisVO contributionAnalysisVO = new ContributionAnalysisVO(); // ContributionAnalysisVO contributionAnalysisVO = new ContributionAnalysisVO();
//存储台站每天的浓度值数据 // //存储台站每天的浓度值数据
Map<String,Double> stationEveryDayConcDatas = new LinkedHashMap<>(); // Map<String,Double> stationEveryDayConcDatas = new LinkedHashMap<>();
//存储核设施每天的浓度值数据 // //存储核设施每天的浓度值数据
Map<String,Map<String,Double>> facilityEveryDayConcDatas = new HashMap<>(); // Map<String,Map<String,Double>> facilityEveryDayConcDatas = new HashMap<>();
//饼图数据 // //饼图数据
Map<String,Double> pipeChartData = new HashMap<>(); // Map<String,Double> pipeChartData = new HashMap<>();
//总浓度值 // //总浓度值
Double totalConc = 0D; // Double totalConc = 0D;
//获取nc文件路径 // //获取nc文件路径
String path = this.getBackForwardTaskNCPath(transportTask,stationInfo.getStationCode()); // String path = this.getBackForwardTaskNCPath(transportTask,stationInfo.getStationCode());
ncFile = NetcdfFile.open(path.toString()); // ncFile = NetcdfFile.open(path.toString());
List<Double> lonData = NcUtil.getNCList(ncFile, "longitude"); // List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
List<Double> latData = NcUtil.getNCList(ncFile, "latitude"); // List<Double> latData = NcUtil.getNCList(ncFile, "latitude");
List<Double> timeData = NcUtil.getNCList(ncFile, "time"); // List<Double> timeData = NcUtil.getNCList(ncFile, "time");
Variable spec001Mr = ncFile.findVariable("spec001_mr"); // Variable spec001Mr = ncFile.findVariable("spec001_mr");
for(int k=0;k<timeData.size();k++){ // for(int k=0;k<timeData.size();k++){
System.out.println(stationInfo.getStationCode()+"循环:"+k+"，共"+timeData.size()+"次"); // System.out.println(stationInfo.getStationCode()+"循环:"+k+"，共"+timeData.size()+"次");
//处理日期数据 // //处理日期数据
Instant instant = transportTask.getEndTime().toInstant(); // Instant instant = transportTask.getEndTime().toInstant();
LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault()); // LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());
localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue()); // localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue());
String dayStr = LocalDateTimeUtil.format(localDateTime, "yyyy-MM-dd"); // String dayStr = LocalDateTimeUtil.format(localDateTime, "yyyy-MM-dd");
if (!stationEveryDayConcDatas.containsKey(dayStr)) { // if (!stationEveryDayConcDatas.containsKey(dayStr)) {
stationEveryDayConcDatas.put(dayStr,0D); // stationEveryDayConcDatas.put(dayStr,0D);
} // }
//获取台站点位取整后±1.5度内的点坐标及数据，使用插值算法求，台站点位的值 // //获取台站点位取整后±1.5度内的点坐标及数据，使用插值算法求，台站点位的值
Double stationConc = this.getTargetSiteConc(lonData,latData,stationInfo.getLon(),stationInfo.getLat(),k,spec001Mr); // Double stationConc = this.getTargetSiteConc(lonData,latData,stationInfo.getLon(),stationInfo.getLat(),k,spec001Mr);
//累加台站位置当前天的浓度数据 // //累加台站位置当前天的浓度数据
stationEveryDayConcDatas.put(dayStr,stationEveryDayConcDatas.get(dayStr)+stationConc); // stationEveryDayConcDatas.put(dayStr,stationEveryDayConcDatas.get(dayStr)+stationConc);
//
for (GardsNuclearReactors facility : facilitys){ // for (GardsNuclearReactors facility : facilitys){
Double facilityConc = this.getTargetSiteConc(lonData,latData,facility.getLongitude(),facility.getLatitude(),k,spec001Mr); // Double facilityConc = this.getTargetSiteConc(lonData,latData,facility.getLongitude(),facility.getLatitude(),k,spec001Mr);
if (facilityConc>0){ // if (facilityConc>0){
if (!facilityEveryDayConcDatas.containsKey(dayStr)) { // if (!facilityEveryDayConcDatas.containsKey(dayStr)) {
Map<String,Double> facilityConcMap = new HashMap<>(); // Map<String,Double> facilityConcMap = new HashMap<>();
facilityConcMap.put(facility.getUnitName(),facilityConc); // facilityConcMap.put(facility.getUnitName(),facilityConc);
facilityEveryDayConcDatas.put(dayStr,facilityConcMap); // facilityEveryDayConcDatas.put(dayStr,facilityConcMap);
}else { // }else {
Map<String,Double> facilityConcMap = facilityEveryDayConcDatas.get(dayStr); // Map<String,Double> facilityConcMap = facilityEveryDayConcDatas.get(dayStr);
if (!facilityConcMap.containsKey(facility.getUnitName())) { // if (!facilityConcMap.containsKey(facility.getUnitName())) {
facilityConcMap.put(facility.getUnitName(),facilityConc); // facilityConcMap.put(facility.getUnitName(),facilityConc);
}else { // }else {
facilityConcMap.put(facility.getUnitName(),facilityConcMap.get(facility.getUnitName())+facilityConc); // facilityConcMap.put(facility.getUnitName(),facilityConcMap.get(facility.getUnitName())+facilityConc);
} // }
} // }
} // }
} // }
} // }
//计算本模拟时间内总浓度数据 // //计算本模拟时间内总浓度数据
totalConc = stationEveryDayConcDatas.values().stream().mapToDouble(Double::doubleValue).sum(); // totalConc = stationEveryDayConcDatas.values().stream().mapToDouble(Double::doubleValue).sum();
//处理饼图数据 // //处理饼图数据
if(CollUtil.isNotEmpty(facilityEveryDayConcDatas)){ // if(CollUtil.isNotEmpty(facilityEveryDayConcDatas)){
for (Map<String,Double> facilityConcMap : facilityEveryDayConcDatas.values()) { // for (Map<String,Double> facilityConcMap : facilityEveryDayConcDatas.values()) {
Set<Map.Entry<String, Double>> entries = facilityConcMap.entrySet(); // Set<Map.Entry<String, Double>> entries = facilityConcMap.entrySet();
for (Map.Entry<String, Double> entry : entries) { // for (Map.Entry<String, Double> entry : entries) {
if (!pipeChartData.containsKey(entry.getKey())) { // if (!pipeChartData.containsKey(entry.getKey())) {
pipeChartData.put(entry.getKey(),entry.getValue()); // pipeChartData.put(entry.getKey(),entry.getValue());
}else { // }else {
pipeChartData.put(entry.getKey(),pipeChartData.get(entry.getKey())+entry.getValue()); // pipeChartData.put(entry.getKey(),pipeChartData.get(entry.getKey())+entry.getValue());
} // }
} // }
} // }
} // }
//处理返回值 // //处理返回值
contributionAnalysisVO.setTotalConc(totalConc); // contributionAnalysisVO.setTotalConc(totalConc);
contributionAnalysisVO.setStationEveryDayConcDatas(stationEveryDayConcDatas); // contributionAnalysisVO.setStationEveryDayConcDatas(stationEveryDayConcDatas);
contributionAnalysisVO.setFacilityEveryDayConcDatas(facilityEveryDayConcDatas); // contributionAnalysisVO.setFacilityEveryDayConcDatas(facilityEveryDayConcDatas);
contributionAnalysisVO.setNuclideNames(new ArrayList<>(pipeChartData.keySet())); // contributionAnalysisVO.setNuclideNames(new ArrayList<>(pipeChartData.keySet()));
contributionAnalysisVO.setPipeChartData(pipeChartData); // contributionAnalysisVO.setPipeChartData(pipeChartData);
redisUtil.set(CommonConstant.TRANSPORT_CONTRIBUTION_ANALYSIS+transportTask.getId()+":"+stationInfo.getStationCode(), contributionAnalysisVO); // redisUtil.set(CommonConstant.TRANSPORT_CONTRIBUTION_ANALYSIS+transportTask.getId()+":"+stationInfo.getStationCode(), contributionAnalysisVO);
ncFile.close(); // ncFile.close();
} // }
}catch (IOException | InvalidRangeException e) { // }catch (IOException | InvalidRangeException e) {
throw new RuntimeException(e); // throw new RuntimeException(e);
}finally { // }finally {
try { // try {
if(ncFile !=null){ // if(ncFile !=null){
ncFile.close(); // ncFile.close();
} // }
} catch (IOException e) { // } catch (IOException e) {
throw new RuntimeException(e); // throw new RuntimeException(e);
} // }
} // }
} }
/** /**
@ -489,11 +509,11 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
if(Objects.isNull(transportTask)){ if(Objects.isNull(transportTask)){
throw new RuntimeException("此任务不存在"); throw new RuntimeException("此任务不存在");
} }
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>(); LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskChild::getTaskId,taskId); queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode); queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
queryWrapper.orderByAsc(TransportTaskChild::getId); queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper); List<TransportTaskBackwardChild> transportTaskChildren = backwardChildMapper.selectList(queryWrapper);
if(CollUtil.isEmpty(transportTaskChildren)){ if(CollUtil.isEmpty(transportTaskChildren)){
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息"); throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
} }
@ -527,19 +547,19 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
if(Objects.isNull(transportTask)){ if(Objects.isNull(transportTask)){
throw new RuntimeException("此任务不存在"); throw new RuntimeException("此任务不存在");
} }
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskChild::getTaskId,taskId);
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode);
queryWrapper.orderByAsc(TransportTaskChild::getId);
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
if(CollUtil.isEmpty(transportTaskChildren)){
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
}
//获取nc文件路径 //获取nc文件路径
String path = ""; String path = "";
if(TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())){ if(TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())){
path = this.getForwardTaskNCPath(transportTask); path = this.getForwardTaskNCPath(transportTask);
}else if(TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())){ }else if(TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())){
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
List<TransportTaskBackwardChild> transportTaskChildren = backwardChildMapper.selectList(queryWrapper);
if(CollUtil.isEmpty(transportTaskChildren)){
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
}
path = this.getBackForwardTaskNCPath(transportTask,transportTaskChildren.get(0).getStationCode()); path = this.getBackForwardTaskNCPath(transportTask,transportTaskChildren.get(0).getStationCode());
} }
if(!FileUtil.exist(path)){ if(!FileUtil.exist(path)){
@ -569,46 +589,46 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
*/ */
@Override @Override
public Map<String,Double> getTimingAnalysis(Integer taskId,String stationCode,String facilityName,Integer timeNum) { public Map<String,Double> getTimingAnalysis(Integer taskId,String stationCode,String facilityName,Integer timeNum) {
TransportTask transportTask = transportTaskMapper.selectById(taskId); // TransportTask transportTask = transportTaskMapper.selectById(taskId);
if(Objects.isNull(transportTask)){ // if(Objects.isNull(transportTask)){
throw new RuntimeException("此任务不存在"); // throw new RuntimeException("此任务不存在");
} // }
//包含从缓存中拿 // //包含从缓存中拿
String key = CommonConstant.TRANSPORT_TIMING_ANALYSIS+transportTask.getId()+":"+stationCode+":"+facilityName; // String key = CommonConstant.TRANSPORT_TIMING_ANALYSIS+transportTask.getId()+":"+stationCode+":"+facilityName;
if(redisUtil.hasKey(key)){ // if(redisUtil.hasKey(key)){
Map<String,Double> concValMap = (LinkedHashMap)redisUtil.get(key); // Map<String,Double> concValMap = (LinkedHashMap)redisUtil.get(key);
//如果是3小时纬度，直接范围，因为最小是3小时 // //如果是3小时纬度，直接范围，因为最小是3小时
if (TransportTimingAnalysisEnum.THREE_HOURS.getKey().equals(timeNum)) { // if (TransportTimingAnalysisEnum.THREE_HOURS.getKey().equals(timeNum)) {
concValMap.forEach((k,v)->{ // concValMap.forEach((k,v)->{
//纳克转换为毫克需除以1000000 // //纳克转换为毫克需除以1000000
BigDecimal concValue = new BigDecimal(v); // BigDecimal concValue = new BigDecimal(v);
BigDecimal finalValue = concValue.divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP); // BigDecimal finalValue = concValue.divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP);
concValMap.put(k,finalValue.doubleValue()); // concValMap.put(k,finalValue.doubleValue());
}); // });
return concValMap; // return concValMap;
}else{ // }else{
Map<String,Double> resultMap = new LinkedHashMap<>(); // Map<String,Double> resultMap = new LinkedHashMap<>();
Instant endTimeInstant = transportTask.getEndTime().toInstant(); // Instant endTimeInstant = transportTask.getEndTime().toInstant();
LocalDateTime endTime = LocalDateTime.ofInstant(endTimeInstant, ZoneId.systemDefault()); // LocalDateTime endTime = LocalDateTime.ofInstant(endTimeInstant, ZoneId.systemDefault());
//
Instant startTimeInstant = transportTask.getStartTime().toInstant(); // Instant startTimeInstant = transportTask.getStartTime().toInstant();
LocalDateTime startTime = LocalDateTime.ofInstant(startTimeInstant, ZoneId.systemDefault()); // LocalDateTime startTime = LocalDateTime.ofInstant(startTimeInstant, ZoneId.systemDefault());
//
boolean flag = true; // boolean flag = true;
while(flag){ // while(flag){
String dayTimeStr = LocalDateTimeUtil.format(startTime, "yyyy-MM-dd HH:mm:ss"); // String dayTimeStr = LocalDateTimeUtil.format(startTime, "yyyy-MM-dd HH:mm:ss");
//纳克转换为毫克需除以1000000 // //纳克转换为毫克需除以1000000
BigDecimal concValue = new BigDecimal(concValMap.get(dayTimeStr)); // BigDecimal concValue = new BigDecimal(concValMap.get(dayTimeStr));
BigDecimal finalValue = concValue.divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP); // BigDecimal finalValue = concValue.divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP);
resultMap.put(dayTimeStr,finalValue.doubleValue()); // resultMap.put(dayTimeStr,finalValue.doubleValue());
startTime = startTime.plusHours(timeNum); // startTime = startTime.plusHours(timeNum);
if(startTime.isEqual(endTime)){ // if(startTime.isEqual(endTime)){
flag = false; // flag = false;
} // }
} // }
return resultMap; // return resultMap;
} // }
} // }
return Map.of(); return Map.of();
} }
@ -619,62 +639,62 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
*/ */
@Override @Override
public void handleTimingAnalysis(Integer taskId) { public void handleTimingAnalysis(Integer taskId) {
TransportTask transportTask = this.transportTaskMapper.selectById(taskId); // TransportTask transportTask = this.transportTaskMapper.selectById(taskId);
//
//查询需要处理数据的台站 // //查询需要处理数据的台站
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>(); // LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskChild::getTaskId,taskId); // queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
List<TransportTaskChild> stationInfos = this.transportTaskChildMapper.selectList(queryWrapper); // List<TransportTaskBackwardChild> stationInfos = this.transportTaskChildMapper.selectList(queryWrapper);
//
//所以核设施数据 // //所以核设施数据
List<GardsNuclearReactors> facilitys = (List<GardsNuclearReactors>) redisUtil.get(CommonConstant.ALL_NUCLEARFACILITY); // List<GardsNuclearReactors> facilitys = (List<GardsNuclearReactors>) redisUtil.get(CommonConstant.ALL_NUCLEARFACILITY);
//
NetcdfFile ncFile = null; // NetcdfFile ncFile = null;
try { // try {
for(TransportTaskChild stationInfo :stationInfos) { // for(TransportTaskBackwardChild stationInfo :stationInfos) {
log.info("处理"+stationInfo.getStationCode()+"台站数据"); // log.info("处理"+stationInfo.getStationCode()+"台站数据");
Map<String,Map<String,Double>> everyFacilityConcDatas = new HashMap<>(); // Map<String,Map<String,Double>> everyFacilityConcDatas = new HashMap<>();
//获取nc文件路径 // //获取nc文件路径
String path = this.getBackForwardTaskNCPath(transportTask,stationInfo.getStationCode()); // String path = this.getBackForwardTaskNCPath(transportTask,stationInfo.getStationCode());
ncFile = NetcdfFile.open(path.toString()); // ncFile = NetcdfFile.open(path.toString());
List<Double> lonData = NcUtil.getNCList(ncFile, "longitude"); // List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
List<Double> latData = NcUtil.getNCList(ncFile, "latitude"); // List<Double> latData = NcUtil.getNCList(ncFile, "latitude");
List<Double> timeData = NcUtil.getNCList(ncFile, "time"); // List<Double> timeData = NcUtil.getNCList(ncFile, "time");
Variable spec001Mr = ncFile.findVariable("spec001_mr"); // Variable spec001Mr = ncFile.findVariable("spec001_mr");
for (GardsNuclearReactors facility : facilitys){ // for (GardsNuclearReactors facility : facilitys){
//存储台站每步的浓度值数据 // //存储台站每步的浓度值数据
Map<String,Double> everyStepConcDatas = new LinkedHashMap<>(); // Map<String,Double> everyStepConcDatas = new LinkedHashMap<>();
for(int k=0;k<timeData.size();k++){ // for(int k=0;k<timeData.size();k++){
//处理日期数据 // //处理日期数据
Instant instant = transportTask.getEndTime().toInstant(); // Instant instant = transportTask.getEndTime().toInstant();
LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault()); // LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());
localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue()); // localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue());
String dayTimeStr = LocalDateTimeUtil.format(localDateTime, "yyyy-MM-dd HH:mm:ss"); // String dayTimeStr = LocalDateTimeUtil.format(localDateTime, "yyyy-MM-dd HH:mm:ss");
//获取台站点位取整后±1.5度内的点坐标及数据，使用插值算法求，台站点位的值 // //获取台站点位取整后±1.5度内的点坐标及数据，使用插值算法求，台站点位的值
Double facilityConc = this.getTargetSiteConc(lonData,latData,facility.getLongitude(),facility.getLatitude(),k,spec001Mr); // Double facilityConc = this.getTargetSiteConc(lonData,latData,facility.getLongitude(),facility.getLatitude(),k,spec001Mr);
everyStepConcDatas.put(dayTimeStr,facilityConc); // everyStepConcDatas.put(dayTimeStr,facilityConc);
} // }
everyFacilityConcDatas.put(facility.getUnitName(),everyStepConcDatas); // everyFacilityConcDatas.put(facility.getUnitName(),everyStepConcDatas);
} // }
if(CollUtil.isNotEmpty(everyFacilityConcDatas)){ // if(CollUtil.isNotEmpty(everyFacilityConcDatas)){
everyFacilityConcDatas.forEach((facilityName,facilityConcData)->{ // everyFacilityConcDatas.forEach((facilityName,facilityConcData)->{
String key = CommonConstant.TRANSPORT_TIMING_ANALYSIS+transportTask.getId()+":"+stationInfo.getStationCode()+":"+facilityName; // String key = CommonConstant.TRANSPORT_TIMING_ANALYSIS+transportTask.getId()+":"+stationInfo.getStationCode()+":"+facilityName;
redisUtil.set(key, facilityConcData); // redisUtil.set(key, facilityConcData);
}); // });
} // }
ncFile.close(); // ncFile.close();
} // }
}catch (IOException | InvalidRangeException e) { // }catch (IOException | InvalidRangeException e) {
throw new RuntimeException(e); // throw new RuntimeException(e);
}finally { // }finally {
try { // try {
if(ncFile !=null){ // if(ncFile !=null){
ncFile.close(); // ncFile.close();
} // }
} catch (IOException e) { // } catch (IOException e) {
throw new RuntimeException(e); // throw new RuntimeException(e);
} // }
} // }
} }
/** /**

204
jeecg-module-transport/src/main/java/org/jeecg/service/impl/TransportTaskServiceImpl.java
View File

@ -7,22 +7,22 @@ import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl; import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import org.jeecg.common.constant.CommonConstant; import org.jeecg.common.constant.enums.TransportReleaseDataSource;
import org.jeecg.common.constant.enums.TransportTaskModeEnum;
import org.jeecg.common.constant.enums.TransportTaskStatusEnum; import org.jeecg.common.constant.enums.TransportTaskStatusEnum;
import org.jeecg.common.constant.enums.TransportTaskTypeEnum; import org.jeecg.common.constant.enums.TransportTaskTypeEnum;
import org.jeecg.common.properties.DataFusionProperties;
import org.jeecg.common.properties.ServerProperties;
import org.jeecg.common.properties.SystemStorageProperties; import org.jeecg.common.properties.SystemStorageProperties;
import org.jeecg.common.properties.TransportSimulationProperties; import org.jeecg.common.properties.TransportSimulationProperties;
import org.jeecg.common.system.query.PageRequest; import org.jeecg.common.system.query.PageRequest;
import org.jeecg.common.util.RedisUtil; import org.jeecg.common.util.RedisUtil;
import org.jeecg.modules.base.entity.TransportTask; import org.jeecg.modules.base.entity.*;
import org.jeecg.modules.base.entity.TransportTaskChild; import org.jeecg.modules.base.mapper.*;
import org.jeecg.modules.base.entity.TransportTaskLog;
import org.jeecg.modules.base.mapper.TransportTaskChildMapper;
import org.jeecg.modules.base.mapper.TransportTaskLogMapper;
import org.jeecg.modules.base.mapper.TransportTaskMapper;
import org.jeecg.modules.base.mapper.WeatherDataMapper;
import org.jeecg.service.TransportTaskService; import org.jeecg.service.TransportTaskService;
import org.jeecg.task.flexparttask.TransportTaskExec; import org.jeecg.task.flexparttask.AbstractTaskExec;
import org.jeecg.task.flexparttask.BackwardTaskExec;
import org.jeecg.task.flexparttask.ForwardTaskExec;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional; import org.springframework.transaction.annotation.Transactional;
import java.time.LocalDate; import java.time.LocalDate;
@ -37,12 +37,17 @@ import java.util.Objects;
@Service @Service
public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,TransportTask> implements TransportTaskService { public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,TransportTask> implements TransportTaskService {
private final TransportTaskChildMapper transportTaskChildMapper; private final TransportTaskBackwardChildMapper taskBackwardChildMapper;
private final TransportTaskLogMapper transportTaskLogMapper; private final TransportTaskLogMapper transportTaskLogMapper;
private final WeatherDataMapper weatherDataMapper; private final WeatherDataMapper weatherDataMapper;
private final TransportSimulationProperties simulationProperties; private final TransportSimulationProperties simulationProperties;
private final SystemStorageProperties systemStorageProperties; private final SystemStorageProperties systemStorageProperties;
private final RedisUtil redisUtil; private final RedisUtil redisUtil;
private final DataFusionProperties dataFusionProperties;
private final TransportTaskForwardReleaseMapper taskForwardReleaseMapper;
private final TransportTaskForwardChildMapper taskForwardChildMapper;
private final TransportTaskForwardSpeciesMapper taskForwardSpeciesMapper;
private final ServerProperties serverProperties;
/** /**
* 分页查询任务列表 * 分页查询任务列表
@ -92,11 +97,37 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
transportTask.setTaskType(TransportTaskTypeEnum.MANUALLY.getKey()); transportTask.setTaskType(TransportTaskTypeEnum.MANUALLY.getKey());
transportTask.setTimeConsuming(0D); transportTask.setTimeConsuming(0D);
this.baseMapper.insert(transportTask); this.baseMapper.insert(transportTask);
if(CollUtil.isNotEmpty(transportTask.getChildList())){ if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode()) &&
transportTask.getChildList().forEach(transportTaskChild -> { CollUtil.isNotEmpty(transportTask.getBackwardChild())) {
transportTask.getBackwardChild().forEach(transportTaskChild -> {
transportTaskChild.setTaskId(transportTask.getId()); transportTaskChild.setTaskId(transportTask.getId());
}); });
transportTaskChildMapper.insert(transportTask.getChildList()); taskBackwardChildMapper.insert(transportTask.getBackwardChild());
}else if (TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())) {
if(CollUtil.isNotEmpty(transportTask.getForwardChild())){
if(TransportReleaseDataSource.MANUAL_ENTRY.getKey().equals(transportTask.getReleaseDataSource())){
for (TransportTaskForwardChild child : transportTask.getForwardChild()) {
child.setTaskId(transportTask.getId());
taskForwardChildMapper.insert(child);
child.getForwardReleaseChild().forEach(forwardReleaseChild -> {
forwardReleaseChild.setForwardChildId(child.getId());
forwardReleaseChild.setTaskId(transportTask.getId());
});
taskForwardReleaseMapper.insert(child.getForwardReleaseChild());
}
}else {
transportTask.getForwardChild().forEach(transportTaskChild -> {
transportTaskChild.setTaskId(transportTask.getId());
});
taskForwardChildMapper.insert(transportTask.getForwardChild());
}
}
if(CollUtil.isNotEmpty(transportTask.getSpecies())){
transportTask.getSpecies().forEach(transportTaskSpecies -> {
transportTaskSpecies.setTaskId(transportTask.getId());
});
taskForwardSpeciesMapper.insert(transportTask.getSpecies());
}
} }
} }
@ -112,11 +143,37 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
if(Objects.isNull(transportTask)){ if(Objects.isNull(transportTask)){
throw new RuntimeException("此任务不存在"); throw new RuntimeException("此任务不存在");
} }
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>(); if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())) {
queryWrapper.eq(TransportTaskChild::getTaskId,transportTask.getId()); LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper); queryWrapper.eq(TransportTaskBackwardChild::getTaskId,transportTask.getId());
List<TransportTaskBackwardChild> transportTaskChildren = taskBackwardChildMapper.selectList(queryWrapper);
if(CollUtil.isNotEmpty(transportTaskChildren)){ if(CollUtil.isNotEmpty(transportTaskChildren)){
transportTask.setChildList(transportTaskChildren); transportTask.setBackwardChild(transportTaskChildren);
}
}else if (TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())) {
LambdaQueryWrapper<TransportTaskForwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskForwardChild::getTaskId,transportTask.getId());
List<TransportTaskForwardChild> taskChildren = taskForwardChildMapper.selectList(queryWrapper);
if(CollUtil.isNotEmpty(taskChildren)){
transportTask.setForwardChild(taskChildren);
if(TransportReleaseDataSource.MANUAL_ENTRY.getKey().equals(transportTask.getReleaseDataSource())){
taskChildren.forEach(taskChild -> {
LambdaQueryWrapper<TransportTaskForwardRelease> releaseQueryWrapper = new LambdaQueryWrapper<>();
releaseQueryWrapper.eq(TransportTaskForwardRelease::getForwardChildId,taskChild.getId());
releaseQueryWrapper.orderByAsc(TransportTaskForwardRelease::getStartTime);
List<TransportTaskForwardRelease> forwardReleases = taskForwardReleaseMapper.selectList(releaseQueryWrapper);
if(CollUtil.isNotEmpty(forwardReleases)){
taskChild.setForwardReleaseChild(forwardReleases);
}
});
}
}
LambdaQueryWrapper<TransportTaskForwardSpecies> speciesQueryWrapper = new LambdaQueryWrapper<>();
speciesQueryWrapper.eq(TransportTaskForwardSpecies::getTaskId,transportTask.getId());
List<TransportTaskForwardSpecies> species = taskForwardSpeciesMapper.selectList(speciesQueryWrapper);
if(CollUtil.isNotEmpty(species)){
transportTask.setSpecies(species);
}
} }
return transportTask; return transportTask;
} }
@ -147,16 +204,55 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
checkIdResult.setZ1(transportTask.getZ1()); checkIdResult.setZ1(transportTask.getZ1());
checkIdResult.setZ2(transportTask.getZ2()); checkIdResult.setZ2(transportTask.getZ2());
checkIdResult.setParticleCount(transportTask.getParticleCount()); checkIdResult.setParticleCount(transportTask.getParticleCount());
checkIdResult.setReleaseDataSource(transportTask.getReleaseDataSource());
this.baseMapper.updateById(checkIdResult); this.baseMapper.updateById(checkIdResult);
//先删除再保存 //先删除再保存
LambdaQueryWrapper<TransportTaskChild> delQueryWrapper = new LambdaQueryWrapper<>(); if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())) {
delQueryWrapper.eq(TransportTaskChild::getTaskId,checkIdResult.getId()); LambdaQueryWrapper<TransportTaskBackwardChild> delQueryWrapper = new LambdaQueryWrapper<>();
transportTaskChildMapper.delete(delQueryWrapper); delQueryWrapper.eq(TransportTaskBackwardChild::getTaskId,checkIdResult.getId());
if(CollUtil.isNotEmpty(transportTask.getChildList())){ taskBackwardChildMapper.delete(delQueryWrapper);
transportTask.getChildList().forEach(transportTaskChild -> { if(CollUtil.isNotEmpty(transportTask.getBackwardChild())){
transportTask.getBackwardChild().forEach(transportTaskChild -> {
transportTaskChild.setTaskId(transportTask.getId()); transportTaskChild.setTaskId(transportTask.getId());
}); });
transportTaskChildMapper.insert(transportTask.getChildList()); taskBackwardChildMapper.insert(transportTask.getBackwardChild());
}
}else if (TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())) {
//先删除再新增
LambdaQueryWrapper<TransportTaskForwardChild> delForwardChildQueryWrapper = new LambdaQueryWrapper<>();
delForwardChildQueryWrapper.eq(TransportTaskForwardChild::getTaskId,checkIdResult.getId());
taskForwardChildMapper.delete(delForwardChildQueryWrapper);
LambdaQueryWrapper<TransportTaskForwardRelease> delForwardReleaseQueryWrapper = new LambdaQueryWrapper<>();
delForwardReleaseQueryWrapper.eq(TransportTaskForwardRelease::getTaskId,checkIdResult.getId());
taskForwardReleaseMapper.delete(delForwardReleaseQueryWrapper);
LambdaQueryWrapper<TransportTaskForwardSpecies> delForwardSpeciesQueryWrapper = new LambdaQueryWrapper<>();
delForwardSpeciesQueryWrapper.eq(TransportTaskForwardSpecies::getTaskId,checkIdResult.getId());
taskForwardSpeciesMapper.delete(delForwardSpeciesQueryWrapper);
//重新新增
if(CollUtil.isNotEmpty(transportTask.getForwardChild())){
if(TransportReleaseDataSource.MANUAL_ENTRY.getKey().equals(transportTask.getReleaseDataSource())){
for (TransportTaskForwardChild child : transportTask.getForwardChild()) {
child.setTaskId(transportTask.getId());
taskForwardChildMapper.insert(child);
child.getForwardReleaseChild().forEach(forwardReleaseChild -> {
forwardReleaseChild.setForwardChildId(child.getId());
forwardReleaseChild.setTaskId(transportTask.getId());
});
taskForwardReleaseMapper.insert(child.getForwardReleaseChild());
}
}else {
transportTask.getForwardChild().forEach(transportTaskChild -> {
transportTaskChild.setTaskId(transportTask.getId());
});
taskForwardChildMapper.insert(transportTask.getForwardChild());
}
}
if(CollUtil.isNotEmpty(transportTask.getSpecies())){
transportTask.getSpecies().forEach(transportTaskSpecies -> {
transportTaskSpecies.setTaskId(transportTask.getId());
});
taskForwardSpeciesMapper.insert(transportTask.getSpecies());
}
} }
} }
@ -168,13 +264,15 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
@Transactional(rollbackFor = RuntimeException.class) @Transactional(rollbackFor = RuntimeException.class)
@Override @Override
public void delete(Integer id) { public void delete(Integer id) {
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>(); TransportTask transportTask = this.baseMapper.selectById(id);
queryWrapper.eq(TransportTaskChild::getTaskId,id); if(Objects.nonNull(transportTask)){
transportTaskChildMapper.delete(queryWrapper); if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())) {
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,id);
taskBackwardChildMapper.delete(queryWrapper);
}
this.baseMapper.deleteById(id); this.baseMapper.deleteById(id);
//删除存储的贡献分析数据和时序分析数据 }
redisUtil.del(CommonConstant.TRANSPORT_CONTRIBUTION_ANALYSIS+id);
redisUtil.del(CommonConstant.TRANSPORT_TIMING_ANALYSIS+id);
} }
/** /**
@ -190,19 +288,51 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
if(Objects.isNull(checkIdResult)){ if(Objects.isNull(checkIdResult)){
throw new RuntimeException("此任务不存在"); throw new RuntimeException("此任务不存在");
} }
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>(); if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(checkIdResult.getTaskMode())){
queryWrapper.eq(TransportTaskChild::getTaskId,id); LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.orderByAsc(TransportTaskChild::getId); queryWrapper.eq(TransportTaskBackwardChild::getTaskId,id);
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper); queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
List<TransportTaskBackwardChild> transportTaskChildren = taskBackwardChildMapper.selectList(queryWrapper);
if(Objects.isNull(transportTaskChildren)){ if(Objects.isNull(transportTaskChildren)){
throw new RuntimeException("此任务对应的flexpart模型配置信息不存在，请确认"); throw new RuntimeException("此任务对应的flexpart反演配置信息不存在，请确认");
} }
TransportTaskExec taskExec = new TransportTaskExec(); AbstractTaskExec taskExec = new BackwardTaskExec();
taskExec.init(weatherDataMapper,this, taskExec.init(weatherDataMapper,this,
checkIdResult,transportTaskChildren, checkIdResult,transportTaskChildren,
simulationProperties,systemStorageProperties); simulationProperties,systemStorageProperties,
taskExec.setName("输运模拟执行线程"); dataFusionProperties,serverProperties);
taskExec.setName("大气输运反演任务执行线程");
taskExec.start(); taskExec.start();
}else if (TransportTaskModeEnum.FORWARD.getKey().equals(checkIdResult.getTaskMode())){
//查询物种
LambdaQueryWrapper<TransportTaskForwardSpecies> speciesQueryWrapper = new LambdaQueryWrapper<>();
speciesQueryWrapper.eq(TransportTaskForwardSpecies::getTaskId,id);
List<TransportTaskForwardSpecies> species = this.taskForwardSpeciesMapper.selectList(speciesQueryWrapper);
if (CollUtil.isNotEmpty(species)) {
checkIdResult.setSpecies(species);
}
//查询台站信息
LambdaQueryWrapper<TransportTaskForwardChild> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(TransportTaskForwardChild::getTaskId,id);
List<TransportTaskForwardChild> stationInfo = taskForwardChildMapper.selectList(queryWrapper);
if (CollUtil.isNotEmpty(stationInfo)) {
stationInfo.forEach(station -> {
LambdaQueryWrapper<TransportTaskForwardRelease> releaseQueryWrapper = new LambdaQueryWrapper<>();
releaseQueryWrapper.eq(TransportTaskForwardRelease::getForwardChildId,station.getId());
releaseQueryWrapper.orderByAsc(TransportTaskForwardRelease::getStartTime);
List<TransportTaskForwardRelease> releaseList = taskForwardReleaseMapper.selectList(releaseQueryWrapper);
if (CollUtil.isNotEmpty(releaseList)) {
station.setForwardReleaseChild(releaseList);
}
});
checkIdResult.setForwardChild(stationInfo);
}
AbstractTaskExec taskExec = new ForwardTaskExec();
taskExec.init(weatherDataMapper,this,checkIdResult,
simulationProperties,systemStorageProperties,serverProperties);
taskExec.setName("大气输运反演任务执行线程");
taskExec.start();
}
} }
/** /**

219
jeecg-module-transport/src/main/java/org/jeecg/task/flexparttask/AbstractTaskExec.java Normal file
View File

@ -0,0 +1,219 @@
package org.jeecg.task.flexparttask;
import cn.hutool.core.util.StrUtil;
import com.jcraft.jsch.ChannelExec;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import lombok.Setter;
import org.apache.logging.log4j.util.Strings;
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
import org.jeecg.common.properties.DataFusionProperties;
import org.jeecg.common.properties.ServerProperties;
import org.jeecg.common.properties.SystemStorageProperties;
import org.jeecg.common.properties.TransportSimulationProperties;
import org.jeecg.modules.base.entity.TransportTask;
import org.jeecg.modules.base.entity.TransportTaskBackwardChild;
import org.jeecg.modules.base.mapper.WeatherDataMapper;
import org.jeecg.service.TransportTaskService;
import java.io.*;
import java.util.List;
import java.util.Properties;
public abstract class AbstractTaskExec extends Thread{
protected WeatherDataMapper weatherDataMapper;
protected TransportTaskService transportTaskService;
protected TransportTask transportTask;
protected List<TransportTaskBackwardChild> transportTaskChildren;
protected TransportSimulationProperties simulationProperties;
protected SystemStorageProperties systemStorageProperties;
protected DataFusionProperties dataFusionProperties;
protected ServerProperties serverProperties;
/**
* 初始化
*/
public void init(WeatherDataMapper weatherDataMapper,
TransportTaskService transportTaskService,
TransportTask transportTask,
TransportSimulationProperties simulationProperties,
SystemStorageProperties systemStorageProperties,
ServerProperties serverProperties){
this.weatherDataMapper = weatherDataMapper;
this.transportTaskService = transportTaskService;
this.transportTask = transportTask;
this.simulationProperties = simulationProperties;
this.systemStorageProperties = systemStorageProperties;
this.serverProperties = serverProperties;
}
/**
* 初始化
*/
public void init(WeatherDataMapper weatherDataMapper,
TransportTaskService transportTaskService,
TransportTask transportTask,
List<TransportTaskBackwardChild> transportTaskChildren,
TransportSimulationProperties simulationProperties,
SystemStorageProperties systemStorageProperties,
DataFusionProperties dataFusionProperties,
ServerProperties serverProperties){
this.weatherDataMapper = weatherDataMapper;
this.transportTaskService = transportTaskService;
this.transportTask = transportTask;
this.simulationProperties = simulationProperties;
this.systemStorageProperties = systemStorageProperties;
this.transportTaskChildren = transportTaskChildren;
this.dataFusionProperties = dataFusionProperties;
this.serverProperties = serverProperties;
}
public abstract void execute();
protected abstract void checkMetData();
protected abstract void execSimulation();
/**
* 获取气象数据路径
* @param dataSource
* @return
*/
protected String getMetDataPath(Integer dataSource){
StringBuilder path = new StringBuilder();
path.append(systemStorageProperties.getRootPath());
path.append(File.separator);
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
path.append(systemStorageProperties.getPangu());
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
path.append(systemStorageProperties.getGraphcast());
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
path.append(systemStorageProperties.getCra40());
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
path.append(systemStorageProperties.getNcep());
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
path.append(systemStorageProperties.getFnl());
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
path.append(systemStorageProperties.getT1h());
}
return path.toString();
}
/**
* 获取参数配置文件路径
* @param dataSource
* @return
*/
protected String getParamConfigPath(Integer dataSource){
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
return simulationProperties.getPanguParamConfigPath();
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
return simulationProperties.getGraphcastParamConfigPath();
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
return simulationProperties.getCra40ParamConfigPath();
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
return simulationProperties.getNcepParamConfigPath();
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
return simulationProperties.getFnlParamConfigPath();
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
return simulationProperties.getT1hParamConfigPath();
}
return Strings.EMPTY;
}
/**
* 获取台站参数配置文件路径
* @param dataSource
* @return
*/
protected String getStationsConfigPath(Integer dataSource){
StringBuilder path = new StringBuilder();
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
path.append(simulationProperties.getPanguStationsConfigPath());
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
path.append(simulationProperties.getCra40StationsConfigPath());
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
path.append(simulationProperties.getCra40StationsConfigPath());
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
path.append(simulationProperties.getNcepStationsConfigPath());
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
path.append(simulationProperties.getFnlStationsConfigPath());
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
path.append(simulationProperties.getT1hStationsConfigPath());
}
return path.toString();
}
protected class JSchRemoteRunner{
private JSch jsch = new JSch();
private Session session = null;
private ChannelExec channel = null;
@Setter
private String command;
/**
* 登录
* @param host
* @param port
* @param username
* @param password
* @return
* @throws JSchException
*/
protected Session login(String host,int port,String username,String password) throws JSchException {
Properties config = new Properties();
config.put("StrictHostKeyChecking","no");
session = jsch.getSession(username, host, port);
session.setPassword(password);
session.setConfig(config);
session.connect();
return session;
}
/**
* 持续读取日志
*/
protected void execCommand() throws JSchException, IOException {
try{
//打开一个执行通道
channel = (ChannelExec) session.openChannel("exec");
String fullCommand = command + " 2>&1";
channel.setCommand(fullCommand);
// 获取脚本的标准输出流，包含错误输出流
InputStream in = channel.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
// 连接通道
channel.connect();
String line;
while ((line = reader.readLine()) != null) {
if(StrUtil.isNotBlank(line)){
ProgressQueue.getInstance().offer(new ProgressEvent(transportTask.getId(),line));
}
}
// // 等待脚本执行完成
// while (channel.isConnected()) {
// Thread.sleep(1000);
// }
}catch(JSchException |IOException e){
throw new RuntimeException(e);
}finally {
// 关闭资源
if (channel != null) {
channel.disconnect();
}
if (session != null) {
session.disconnect();
}
}
}
}
}

187
jeecg-module-transport/src/main/java/org/jeecg/task/flexparttask/BackwardTaskExec.java Normal file
View File

@ -0,0 +1,187 @@
package org.jeecg.task.flexparttask;
import cn.hutool.core.collection.CollUtil;
import cn.hutool.core.date.LocalDateTimeUtil;
import cn.hutool.core.io.FileUtil;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import org.apache.commons.lang3.time.StopWatch;
import org.apache.logging.log4j.util.Strings;
import org.jeecg.common.constant.enums.FlexpartSpeciesType;
import org.jeecg.common.constant.enums.TransportTaskStatusEnum;
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
import org.jeecg.modules.base.entity.*;
import java.io.*;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.text.DecimalFormat;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
public class BackwardTaskExec extends AbstractTaskExec{
@Override
public void run() {
this.execute();
}
/**
* 执行任务
*/
public void execute() {
StopWatch stopWatch = new StopWatch();
stopWatch.start();
try{
//修改任务状态为执行中
super.transportTaskService.updateTaskStatus(super.transportTask.getId(), TransportTaskStatusEnum.IN_OPERATION.getValue());
//如果此任务已存在历史日志，先清除
super.transportTaskService.deleteTaskLog(super.transportTask.getId());
//检查气象数据
this.checkMetData();
//执行模拟
this.execSimulation();
//生成SRS文件
// this.generateSRSFile();
}catch (Exception e){
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),taskErrorLog));
super.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
throw e;
}finally {
//添加任务耗时
stopWatch.stop();
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
double min = seconds/60D;
BigDecimal bgMin = new BigDecimal(min);
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
super.transportTaskService.updateTaskStatusToCompleted(super.transportTask.getId(),result.doubleValue());
String taskCompletedLog = "任务执行完成，耗时："+min+"分钟";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),taskCompletedLog));
}
}
/**
* 检查气象数据
*/
protected void checkMetData(){
String msg = "检查气象数据";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),msg));
LocalDateTime startTime = super.transportTask.getStartTime();
LocalDateTime endTime = super.transportTask.getEndTime();
LambdaQueryWrapper<WeatherData> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(WeatherData::getDataSource,super.transportTask.getUseMetType());
queryWrapper.between(WeatherData::getDataStartTime,startTime,endTime);
List<WeatherData> dataList = super.weatherDataMapper.selectList(queryWrapper);
long hours = ChronoUnit.HOURS.between(startTime, endTime)/6;
//一天至少4个气象数据，至少是00,06,12,18
if(CollUtil.isEmpty(dataList) || dataList.size() < hours){
String exceptionMsg = "此任务时间范围(%s - %s)参数所涉及的气象数据不完整，请核对气象数据";
String formatMsg = String.format(exceptionMsg, LocalDateTimeUtil.format(startTime, "yyyy-MM-dd hh:mm:ss"), LocalDateTimeUtil.format(endTime, "yyyy-MM-dd hh:mm:ss"));
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),formatMsg));
throw new RuntimeException(formatMsg);
}
String finalMsg = "检查气象数据完毕";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),finalMsg));
}
/**
* 执行模拟
*/
protected void execSimulation(){
// Process process = null;
try {
String paramMsg = "生成flexpart所需参数文件：param.config,stations.config";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),paramMsg));
//处理参数配置文件
String paramConfigPath = super.getParamConfigPath(super.transportTask.getUseMetType());
String metDataPath = super.getMetDataPath(super.transportTask.getUseMetType());
if(!FileUtil.exist(paramConfigPath)){
FileUtil.touch(paramConfigPath);
}
StringBuilder paramContent = new StringBuilder();
paramContent.append(super.transportTask.getStartTime().format(DateTimeFormatter.ofPattern("yyyyMMdd"))).append("\n");
paramContent.append(super.transportTask.getEndTime().format(DateTimeFormatter.ofPattern("yyyyMMdd"))).append("\n");
paramContent.append(super.transportTask.getStartTime().format(DateTimeFormatter.ofPattern("HHmmss"))).append("\n");
paramContent.append(super.transportTask.getEndTime().format(DateTimeFormatter.ofPattern("HHmmss"))).append("\n");
paramContent.append(super.transportTask.getParticleCount()).append("\n");
paramContent.append(super.transportTask.getZ1()).append("\n");
paramContent.append(super.transportTask.getZ2()).append("\n");
paramContent.append(metDataPath).append("\n");
paramContent.append(super.simulationProperties.getOutputPath()+File.separator+super.transportTask.getTaskName()).append("\n");
paramContent.append(FlexpartSpeciesType.NOT_SPECIES.getValue()).append("\n");//反演固定61，不显示具体核素，只显示Xe
FileUtil.writeString(paramContent.toString(),paramConfigPath,"UTF-8");
//处理台站数据文件
List<String> stationConfigInfo = new ArrayList<>();
String stationsConfigPath = super.getStationsConfigPath(super.transportTask.getUseMetType())+".backward";
if(!FileUtil.exist(stationsConfigPath)){
FileUtil.touch(stationsConfigPath);
}
super.transportTaskChildren.forEach(taskChild -> {
String format = "%s,%f,%f,%s,%s,%s,%s,%s";
BigDecimal srcReleaseAmount = new BigDecimal(taskChild.getReleaseAmount());
BigDecimal constant = new BigDecimal("1000");
DecimalFormat scientificFormat = new DecimalFormat("0.00E0");
double value = srcReleaseAmount.divide(constant).doubleValue();
String releaseAmount = scientificFormat.format(value);
String row = String.format(format,taskChild.getStationCode(),taskChild.getLat(),taskChild.getLon(),releaseAmount,
taskChild.getAcqStartTime().format(DateTimeFormatter.ofPattern("yyyyMMdd")),
taskChild.getAcqEndTime().format(DateTimeFormatter.ofPattern("yyyyMMdd")),
taskChild.getAcqStartTime().format(DateTimeFormatter.ofPattern("HHmmss")),
taskChild.getAcqEndTime().format(DateTimeFormatter.ofPattern("HHmmss")));
stationConfigInfo.add(row);
});
//最后一行需要换行，否则启动flexpart报错
stationConfigInfo.add("\n");
FileUtil.writeLines(stationConfigInfo,stationsConfigPath,"UTF-8");
//获取反演脚本路径
String scriptPath = this.getBackForwardScriptPath(super.transportTask.getUseMetType());
String execScriptMsg = "执行任务脚本，开始模拟，路径为："+scriptPath;
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),execScriptMsg));
//ssh连接宿主机调用flexpart
JSchRemoteRunner jschRemoteRunner = new JSchRemoteRunner();
jschRemoteRunner.setCommand(scriptPath);
jschRemoteRunner.login(super.serverProperties.getHost(),super.serverProperties.getPort(),
super.serverProperties.getUsername(),super.serverProperties.getPassword());
jschRemoteRunner.execCommand();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/**
* 获取反演脚本路径
* @param dataSource
* @return
*/
private String getBackForwardScriptPath(Integer dataSource){
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
return simulationProperties.getPanguBackwardScriptPath();
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
return simulationProperties.getGraphcastBackwardScriptPath();
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
return simulationProperties.getCra40BackwardScriptPath();
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
return simulationProperties.getNcepBackwardScriptPath();
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
return simulationProperties.getFnlBackwardScriptPath();
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
return simulationProperties.getT1hBackwardScriptPath();
}
return Strings.EMPTY;
}
/**
* 生成SRS文件
*/
private void generateSRSFile(){
for (TransportTaskBackwardChild transportTaskChild : this.transportTaskChildren){
BuildNcToSrsFile ncToSrsFile = new BuildNcToSrsFile();
ncToSrsFile.init(dataFusionProperties,simulationProperties,transportTask,transportTaskChild,transportTaskChild.getReleaseAmount());
ncToSrsFile.execute();
}
}
}

272
jeecg-module-transport/src/main/java/org/jeecg/task/flexparttask/BuildNcToSrsFile.java Normal file
View File

@ -0,0 +1,272 @@
package org.jeecg.task.flexparttask;
import cn.hutool.core.date.DateUtil;
import cn.hutool.core.io.FileUtil;
import lombok.extern.slf4j.Slf4j;
import org.apache.logging.log4j.util.Strings;
import org.jeecg.common.constant.enums.TransportSampleTypeEnum;
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
import org.jeecg.common.properties.DataFusionProperties;
import org.jeecg.common.properties.TransportSimulationProperties;
import org.jeecg.modules.base.entity.TransportTask;
import org.jeecg.modules.base.entity.TransportTaskBackwardChild;
import ucar.ma2.Array;
import ucar.ma2.DataType;
import ucar.ma2.InvalidRangeException;
import ucar.nc2.Attribute;
import ucar.nc2.NetcdfFile;
import ucar.nc2.Variable;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.text.DecimalFormat;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.List;
import java.util.zip.GZIPOutputStream;
/**
* 根据NC文件构建SRS文件
*/
@Slf4j
public class BuildNcToSrsFile {
private final static String BACK_FORWARD="backward";
private DataFusionProperties dataFusionProperties;
private TransportSimulationProperties simulationProperties;
private TransportTask transportTask;
private TransportTaskBackwardChild transportTaskChild;
private final int radius=6371*1000;
private final double pi=3.14;
private double factorMul;
public void init(DataFusionProperties dataFusionProperties,
TransportSimulationProperties simulationProperties,
TransportTask transportTask,
TransportTaskBackwardChild transportTaskChild,
String releaseAmount) {
this.dataFusionProperties = dataFusionProperties;
this.simulationProperties = simulationProperties;
this.transportTask = transportTask;
this.transportTaskChild = transportTaskChild;
this.factorMul = Double.parseDouble(releaseAmount);
}
/**
* 执行转换任务
*/
public void execute(){
String ncFilePath = this.getBackForwardTaskNCPath();
if(!FileUtil.exist(ncFilePath)){
throw new RuntimeException("此任务模拟结果不存在，请确认任务运行状态");
}
try (NetcdfFile ncFile = NetcdfFile.open(ncFilePath.toString())) {
String header = this.buildHeader(ncFile);
List<String> body = buildBody(ncFile);
body.add(0,header);
File file = new File(this.getSrsStoragePath());
if(!file.exists()){
if(!file.getParentFile().exists()){
file.getParentFile().mkdirs();
}
file.createNewFile();
}
Path outputPath = Paths.get(file.getParent(), file.getName());
this.compressContentToFile(body,outputPath);
log.info("SRS文件生成完毕，路径为：{}", file.getAbsolutePath());
} catch (IOException | InvalidRangeException e) {
String errLog = "SRS文件生成失败，原因："+e.getMessage();
log.error(errLog,e);
throw new RuntimeException(e);
}
}
private String buildHeader(NetcdfFile ncFile){
//处理台站经纬度
BigDecimal lonBigDecimal = new BigDecimal(this.transportTaskChild.getLon());
BigDecimal latBigDecimal = new BigDecimal(this.transportTaskChild.getLat());
Double lon = lonBigDecimal.setScale(2, RoundingMode.HALF_UP).doubleValue();
Double lat = latBigDecimal.setScale(2, RoundingMode.HALF_UP).doubleValue();
//处理测量开始和结束时间参数
LocalDateTime acqStartTime = transportTaskChild.getAcqStartTime();
LocalDateTime acqEndTime = transportTaskChild.getAcqEndTime();
String acqStartTimeStr = acqStartTime.format(DateTimeFormatter.ofPattern("yyyyMMdd"));
String acqEndTimeStr = acqEndTime.format(DateTimeFormatter.ofPattern("yyyyMMdd"));
//处理模拟总时长
LocalDateTime taskStartTime = transportTask.getStartTime();
LocalDateTime taskEndTime = transportTask.getEndTime();
long startHour = taskStartTime.atZone(ZoneId.of("Asia/Shanghai")).getHour();
long endHour = taskEndTime.atZone(ZoneId.of("Asia/Shanghai")).getHour();
long totalHour = endHour - startHour;
//处理输出频率[小时] 和 平均时间[小时]参数
Attribute loutstep = ncFile.findGlobalAttribute("loutstep");
Attribute loutaver = ncFile.findGlobalAttribute("loutaver");
int outputFreHour = Math.abs(loutstep.getNumericValue().intValue())/3600;
int averageTime = Math.abs(loutaver.getNumericValue().intValue())/3600;
//获取网格精度
Attribute dxoutVar = ncFile.findGlobalAttribute("dxout");
Attribute dyoutVar = ncFile.findGlobalAttribute("dyout");
String header = String.format(" %s %s %s %s %s %s %s %s %s %s %s %s %s",
lon,lat,acqStartTimeStr,acqStartTime.getHour(),acqEndTimeStr,acqEndTime.getHour(),
this.transportTaskChild.getReleaseAmount(),totalHour,outputFreHour,averageTime,
dxoutVar.getNumericValue().doubleValue(),dyoutVar.getNumericValue().doubleValue(),
this.transportTaskChild.getStationCode());
return header;
}
/**
* 计算body数据
* @param ncFile
* @return
* @throws InvalidRangeException
* @throws IOException
*/
private List<String> buildBody(NetcdfFile ncFile) throws InvalidRangeException, IOException {
//获取时间序号
Variable time = ncFile.findVariable("time");
int timeLen = time.getShape()[0];
//获取经纬度网格数量
Variable longitude = ncFile.findVariable("longitude");
Variable latitude = ncFile.findVariable("latitude");
int lonLen = longitude.getShape()[0];
int latLen = latitude.getShape()[0];
Array latArray = latitude.read();
Array lonArray = longitude.read();
double[] lats = (double[]) latArray.get1DJavaArray(DataType.DOUBLE);
double[] lons = (double[]) lonArray.get1DJavaArray(DataType.DOUBLE);
//获取网格精度
Attribute dxoutVar = ncFile.findGlobalAttribute("dxout");
Attribute dyoutVar = ncFile.findGlobalAttribute("dyout");
double dxout = dxoutVar.getNumericValue().doubleValue();
double dyout = dyoutVar.getNumericValue().doubleValue();
//处理输出频率[小时]
Attribute loutstep = ncFile.findGlobalAttribute("loutstep");
double outputFreHour = loutstep.getNumericValue().doubleValue();
List<String> body = new ArrayList<>();
Variable spec001Mr = ncFile.findVariable("spec001_mr");
for (int t=0;t<timeLen;t++){
int[] origin = {0, 0,t,0, 0, 0};
int[] section = {1, 1,1,1,latLen,lonLen};
Array data = spec001Mr.read(origin,section);
double[] pointData = (double[]) data.get1DJavaArray(DataType.DOUBLE);
for (int y=0;y<latLen;y++){
for (int x=0;x<lonLen;x++){
int index = y * lonLen + x;
double pointVal = pointData[index];
if (pointVal>0){
double currentLat = lats[y];
double currentLon = lons[x];
double dx_1d=cal_dx_1d(currentLat);
double dy_1d = radius * pi / 180;
double area=dyout * dxout * dy_1d * dx_1d;
double volume=area*100;//height = 100 米（固定的高度层厚度）
double factor = pointVal / (outputFreHour * 3600) / volume * factorMul;
DecimalFormat scientificFormat = new DecimalFormat("0.0000000E0");
String result = scientificFormat.format(factor);
String line = String.format(" %-7s %8s %-4s %s",currentLat,currentLon,t+1,result);
body.add(line);
}
}
}
}
return body;
}
/**
* 获取反演NC结果文件路径
* @return
*/
private String getBackForwardTaskNCPath(){
//拼接nc文件路径
StringBuilder path = new StringBuilder();
path.append(simulationProperties.getOutputPath());
path.append(File.separator);
path.append(this.transportTask.getTaskName());
path.append(File.separator);
path.append(BACK_FORWARD);
path.append(File.separator);
path.append(this.transportTaskChild.getStationCode());
path.append(File.separator);
path.append("grid_time_").append(DateUtil.format(transportTask.getEndTime(), "yyyyMMddHHmmss")).append(".nc");
return path.toString();
}
/**
* 获取SRS文件存储路径
* @return
*/
private String getSrsStoragePath(){
StringBuilder path = new StringBuilder();
path.append(dataFusionProperties.getNdcSRSPath());
path.append(File.separator);
path.append(transportTaskChild.getAcqEndTime().format(DateTimeFormatter.ofPattern("yyyyMMdd")));
path.append(File.separator);
if(TransportSampleTypeEnum.B.name().equals(transportTaskChild.getSampleType())){
path.append("flexpart.x."+this.getMetType()+".l1");
}else if(TransportSampleTypeEnum.P.name().equals(transportTaskChild.getSampleType())){
path.append("flexpart."+this.getMetType()+".l1");
}
path.append(File.separator);
path.append(transportTaskChild.getStationCode()+".fp.");
path.append(transportTaskChild.getAcqEndTime().format(DateTimeFormatter.ofPattern("yyyyMMddHH")));
path.append(".f9.srm.gz");
return path.toString();
}
/**
* 获取气象类型
* @return
*/
private String getMetType(){
if (WeatherDataSourceEnum.PANGU.getKey().equals(transportTask.getUseMetType())){
return WeatherDataSourceEnum.PANGU.name().toLowerCase();
}else if (WeatherDataSourceEnum.GRAPHCAST.getKey().equals(transportTask.getUseMetType())){
return WeatherDataSourceEnum.GRAPHCAST.name().toLowerCase();
}else if (WeatherDataSourceEnum.CRA40.getKey().equals(transportTask.getUseMetType())){
return WeatherDataSourceEnum.CRA40.name().toLowerCase();
}else if (WeatherDataSourceEnum.NCEP.getKey().equals(transportTask.getUseMetType())){
return WeatherDataSourceEnum.NCEP.name().toLowerCase();
}else if (WeatherDataSourceEnum.FNL.getKey().equals(transportTask.getUseMetType())){
return WeatherDataSourceEnum.FNL.name().toLowerCase();
}else if (WeatherDataSourceEnum.T1H.getKey().equals(transportTask.getUseMetType())){
return WeatherDataSourceEnum.T1H.name().toLowerCase();
}
return Strings.EMPTY;
}
/**
* 计算经度方向的实际距离（米）
* 考虑地球曲率效应，经度1°的距离随纬度变化
* @param lat_input
* @return
*/
private Double cal_dx_1d(Double lat_input){
return radius*2*pi*Math.cos(lat_input/180*pi)/360;
}
private void compressContentToFile(List<String> contents, Path outputFile) throws IOException {
try (FileOutputStream fileOut = new FileOutputStream(outputFile.toFile());
GZIPOutputStream gzipOut = new GZIPOutputStream(fileOut)) {
for (int i = 0; i < contents.size(); i++) {
String str = contents.get(i);
byte[] bytes = str.getBytes(StandardCharsets.UTF_8);
gzipOut.write(bytes);
if (i < contents.size() - 1) {
gzipOut.write('\n');
}
}
}
}
}

222
jeecg-module-transport/src/main/java/org/jeecg/task/flexparttask/ForwardTaskExec.java Normal file
View File

@ -0,0 +1,222 @@
package org.jeecg.task.flexparttask;
import cn.hutool.core.collection.CollUtil;
import cn.hutool.core.date.LocalDateTimeUtil;
import cn.hutool.core.io.FileUtil;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import org.apache.commons.lang3.time.StopWatch;
import org.apache.logging.log4j.util.Strings;
import org.jeecg.common.constant.enums.TransportTaskStatusEnum;
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
import org.jeecg.modules.base.entity.*;
import java.io.File;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.text.DecimalFormat;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.*;
import java.util.concurrent.TimeUnit;
public class ForwardTaskExec extends AbstractTaskExec{
@Override
public void run() {
this.execute();
}
/**
* 执行任务
*/
public void execute() {
StopWatch stopWatch = new StopWatch();
stopWatch.start();
try{
//修改任务状态为执行中
super.transportTaskService.updateTaskStatus(super.transportTask.getId(), TransportTaskStatusEnum.IN_OPERATION.getValue());
//如果此任务已存在历史日志，先清除
super.transportTaskService.deleteTaskLog(super.transportTask.getId());
//检查气象数据
this.checkMetData();
//执行模拟
this.execSimulation();
}catch (Exception e){
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),taskErrorLog));
this.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
throw e;
}finally {
//添加任务耗时
stopWatch.stop();
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
double min = seconds/60D;
BigDecimal bgMin = new BigDecimal(min);
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
super.transportTaskService.updateTaskStatusToCompleted(super.transportTask.getId(),result.doubleValue());
String taskCompletedLog = "任务执行完成，耗时："+min+"分钟";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),taskCompletedLog));
}
}
/**
* 检查气象数据
*/
protected void checkMetData(){
String msg = "检查气象数据";
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),msg));
LocalDateTime startTime = super.transportTask.getStartTime();
LocalDateTime endTime = super.transportTask.getEndTime();
LambdaQueryWrapper<WeatherData> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(WeatherData::getDataSource,super.transportTask.getUseMetType());
queryWrapper.between(WeatherData::getDataStartTime,startTime,endTime);
List<WeatherData> dataList = super.weatherDataMapper.selectList(queryWrapper);
long hours = ChronoUnit.HOURS.between(startTime, endTime)/6;
//一天至少4个气象数据，至少是00,06,12,18
if(CollUtil.isEmpty(dataList) || dataList.size() < hours){
String exceptionMsg = "此任务时间范围(%s - %s)参数所涉及的气象数据不完整，请核对气象数据";
String formatMsg = String.format(exceptionMsg, LocalDateTimeUtil.format(startTime, "yyyy-MM-dd hh:mm:ss"), LocalDateTimeUtil.format(endTime, "yyyy-MM-dd hh:mm:ss"));
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),formatMsg));
throw new RuntimeException(formatMsg);
}
String finalMsg = "检查气象数据完毕";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),finalMsg));
}
/**
* 执行模拟
*/
protected void execSimulation(){
Process process = null;
try {
String paramMsg = "生成flexpart所需配置文件：param.config,stations.config,input_site_hour";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),paramMsg));
//处理参数配置文件-param.config
String paramConfigPath = super.getParamConfigPath(super.transportTask.getUseMetType());
String metDataPath = super.getMetDataPath(super.transportTask.getUseMetType());
if(!FileUtil.exist(paramConfigPath)){
FileUtil.touch(paramConfigPath);
}
StringBuilder paramContent = new StringBuilder();
paramContent.append(super.transportTask.getStartTime().format(DateTimeFormatter.ofPattern("yyyyMMdd"))).append("\n");
paramContent.append(super.transportTask.getEndTime().format(DateTimeFormatter.ofPattern("yyyyMMdd"))).append("\n");
paramContent.append(super.transportTask.getStartTime().format(DateTimeFormatter.ofPattern("HHmmss"))).append("\n");
paramContent.append(super.transportTask.getEndTime().format(DateTimeFormatter.ofPattern("HHmmss"))).append("\n");
paramContent.append(super.transportTask.getParticleCount()).append("\n");
paramContent.append(super.transportTask.getZ1()).append("\n");
paramContent.append(super.transportTask.getZ2()).append("\n");
paramContent.append(metDataPath).append("\n");
paramContent.append(super.simulationProperties.getOutputPath()+File.separator+super.transportTask.getTaskName()).append("\n");
//配置物种信息
List<TransportTaskForwardSpecies> species = super.transportTask.getSpecies();
if(CollUtil.isEmpty(species)){
String speciesMissingLog = super.transportTask.getTaskName()+"任务物种信息不存在，任务停止";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),speciesMissingLog));
super.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
return;
}
for(int i=0;i<species.size();i++){
paramContent.append(species.get(i).getSpeciesId());
if (i != species.size()-1){
paramContent.append(",");
}else {
paramContent.append("\n");
}
}
FileUtil.writeString(paramContent.toString(),paramConfigPath,"UTF-8");
//处理台站数据文件-stations.config
if (CollUtil.isEmpty(super.transportTask.getForwardChild())){
String stationsMissingLog = super.transportTask.getTaskName()+"任务站点信息不存在，任务停止";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),stationsMissingLog));
super.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
return;
}
List<String> stationConfigInfo = new ArrayList<>();
String stationsConfigPath = super.getStationsConfigPath(super.transportTask.getUseMetType());
if(!FileUtil.exist(stationsConfigPath)){
FileUtil.touch(stationsConfigPath);
}
super.transportTask.getForwardChild().forEach(station ->{
String format = "%s,%f,%f,%s";
//最后0是以前版本的总释放量，改版后无用，给个默认值
String row = String.format(format,station.getStationCode(),station.getLat(),station.getLon(),"0");
stationConfigInfo.add(row);
});
//最后一行需要换行，否则启动flexpart报错
stationConfigInfo.add("\n");
FileUtil.writeLines(stationConfigInfo,stationsConfigPath,"UTF-8");
//配置各站点排放数据
//如果input_site_hour目录不存在先创建
if(!FileUtil.exist(super.simulationProperties.getInputSiteHourPath())){
FileUtil.mkdir(super.simulationProperties.getInputSiteHourPath());
}else {
FileUtil.clean(super.simulationProperties.getInputSiteHourPath());
}
Map<String,List<TransportTaskForwardRelease>> releaseInfoMap = new HashMap<>();
for (TransportTaskForwardChild station : super.transportTask.getForwardChild()){
if (CollUtil.isEmpty(station.getForwardReleaseChild())){
String releaseMissingLog = station.getStationCode()+"站点释放信息缺失，任务停止";
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),releaseMissingLog));
super.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
return;
}
releaseInfoMap.put(station.getStationCode(),station.getForwardReleaseChild());
}
releaseInfoMap .forEach((stationCode,releaseInfoList)->{
String stationReleaseFile = super.simulationProperties.getInputSiteHourPath() + File.separator + stationCode + ".txt";
FileUtil.touch(stationReleaseFile);
List<String> stationReleaseInfo = new ArrayList<>();
releaseInfoList.forEach(releaseInfo ->{
String format = "%s,%s,%s";
String startTime = releaseInfo.getStartTime().format(DateTimeFormatter.ofPattern("yyyyMMddHH"));
String endTime = releaseInfo.getEndTime().format(DateTimeFormatter.ofPattern("yyyyMMddHH"));
BigDecimal releaseAmount = new BigDecimal(releaseInfo.getReleaseAmount());
BigDecimal constant = new BigDecimal("1000");
DecimalFormat scientificFormat = new DecimalFormat("0.00E0");
double value = releaseAmount.divide(constant).doubleValue();
String releaseAmountFormat = scientificFormat.format(value);
String row = String.format(format, startTime, endTime, releaseAmountFormat);
stationReleaseInfo.add(row);
});
FileUtil.writeLines(stationReleaseInfo,stationReleaseFile,"UTF-8");
});
//获取正演脚本路径
String scriptPath = this.getForwardScriptPath(super.transportTask.getUseMetType());
String execScriptMsg = "执行任务脚本，开始模拟，路径为："+scriptPath;
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),execScriptMsg));
//ssh连接宿主机调用flexpart
JSchRemoteRunner jschRemoteRunner = new JSchRemoteRunner();
jschRemoteRunner.setCommand(scriptPath);
jschRemoteRunner.login(super.serverProperties.getHost(),super.serverProperties.getPort(),
super.serverProperties.getUsername(),super.serverProperties.getPassword());
jschRemoteRunner.execCommand();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/**
* 获取正演脚本路径
* @param dataSource
* @return
*/
private String getForwardScriptPath(Integer dataSource){
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
return simulationProperties.getPanguForwardScriptPath();
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
return simulationProperties.getGraphcastForwardScriptPath();
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
return simulationProperties.getCra40ForwardScriptPath();
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
return simulationProperties.getNcepForwardScriptPath();
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
return simulationProperties.getFnlForwardScriptPath();
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
return simulationProperties.getT1hForwardScriptPath();
}
return Strings.EMPTY;
}
}

307
jeecg-module-transport/src/main/java/org/jeecg/task/flexparttask/TransportTaskExec.java
View File

@ -1,307 +0,0 @@
package org.jeecg.task.flexparttask;
import cn.hutool.core.date.DateUtil;
import cn.hutool.core.io.FileUtil;
import cn.hutool.core.util.StrUtil;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import org.apache.commons.lang3.time.StopWatch;
import org.apache.logging.log4j.util.Strings;
import org.jeecg.common.constant.enums.TransportTaskModeEnum;
import org.jeecg.common.constant.enums.TransportTaskStatusEnum;
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
import org.jeecg.common.properties.SystemStorageProperties;
import org.jeecg.common.properties.TransportSimulationProperties;
import org.jeecg.modules.base.entity.*;
import org.jeecg.modules.base.mapper.WeatherDataMapper;
import org.jeecg.service.TransportTaskService;
import java.io.*;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.TimeUnit;
public class TransportTaskExec extends Thread{
private WeatherDataMapper weatherDataMapper;
private TransportTaskService transportTaskService;
private TransportTask transportTask;
private List<TransportTaskChild> transportTaskChildren;
private TransportSimulationProperties simulationProperties;
private SystemStorageProperties systemStorageProperties;
/**
* 初始化
*/
public void init(WeatherDataMapper weatherDataMapper,
TransportTaskService transportTaskService,
TransportTask transportTask,
List<TransportTaskChild> transportTaskChildren,
TransportSimulationProperties simulationProperties,
SystemStorageProperties systemStorageProperties){
this.weatherDataMapper = weatherDataMapper;
this.transportTaskService = transportTaskService;
this.transportTask = transportTask;
this.transportTaskChildren = transportTaskChildren;
this.simulationProperties = simulationProperties;
this.systemStorageProperties = systemStorageProperties;
}
@Override
public void run() {
this.execute();
}
/**
* 执行任务
*/
public void execute() {
StopWatch stopWatch = new StopWatch();
stopWatch.start();
try{
//修改任务状态为执行中
this.transportTaskService.updateTaskStatus(this.transportTask.getId(), TransportTaskStatusEnum.IN_OPERATION.getValue());
//如果此任务已存在历史日志，先清除
this.transportTaskService.deleteTaskLog(this.transportTask.getId());
//检查气象数据
this.checkMetData();
//执行模拟
this.execSimulation();
}catch (Exception e){
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),taskErrorLog));
this.transportTaskService.updateTaskStatus(this.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
throw e;
}finally {
//添加任务耗时
stopWatch.stop();
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
double min = seconds/60D;
BigDecimal bgMin = new BigDecimal(min);
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
this.transportTaskService.updateTaskStatusToCompleted(this.transportTask.getId(),result.doubleValue());
String taskCompletedLog = "任务执行完成，耗时："+min+"分钟";
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),taskCompletedLog));
}
}
/**
* 检查气象数据
*/
private void checkMetData(){
String msg = "检查气象数据";
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),msg));
LocalDateTime startTime = this.transportTask.getStartTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();
LocalDateTime endTime = this.transportTask.getEndTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();
startTime = startTime.minusDays(3);
endTime = endTime.plusDays(3);
LambdaQueryWrapper<WeatherData> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(WeatherData::getDataSource,this.transportTask.getUseMetType());
queryWrapper.between(WeatherData::getDataStartTime,startTime,endTime);
List<WeatherData> dataList = weatherDataMapper.selectList(queryWrapper);
long days = ChronoUnit.DAYS.between(startTime, endTime);
//一天至少4个气象数据，至少是00,06,12,18
// if(CollUtil.isEmpty(dataList) || dataList.size() < (days*4)){
// String exceptionMsg = "此任务时间范围(%s - %s)参数所涉及的气象数据不完整，请核对气象数据";
// String formatMsg = String.format(exceptionMsg, LocalDateTimeUtil.format(startTime, "yyyy-MM-dd hh:mm:ss"), LocalDateTimeUtil.format(endTime, "yyyy-MM-dd hh:mm:ss"));
// ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),formatMsg));
// throw new RuntimeException(formatMsg);
// }
String finalMsg = "检查气象数据完毕";
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),finalMsg));
}
/**
* 执行模拟
*/
private void execSimulation(){
Process process = null;
try {
String paramMsg = "生成flexpart所需参数文件：param.config,stations.config";
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),paramMsg));
//处理参数配置文件
String paramConfigPath = this.getParamConfigPath(this.transportTask.getUseMetType());
String metDataPath = this.getMetDataPath(this.transportTask.getUseMetType());
if(!FileUtil.exist(paramConfigPath)){
FileUtil.touch(paramConfigPath);
}
StringBuilder paramContent = new StringBuilder();
paramContent.append(DateUtil.format(this.transportTask.getStartTime(),"yyyyMMdd")).append("\n");
paramContent.append(DateUtil.format(this.transportTask.getEndTime(),"yyyyMMdd")).append("\n");
paramContent.append(DateUtil.format(this.transportTask.getStartTime(),"HHmmss")).append("\n");
paramContent.append(DateUtil.format(this.transportTask.getEndTime(),"HHmmss")).append("\n");
paramContent.append(this.transportTask.getParticleCount()).append("\n");
paramContent.append(this.transportTask.getZ1()).append("\n");
paramContent.append(this.transportTask.getZ2()).append("\n");
paramContent.append(metDataPath).append("\n");
paramContent.append(this.simulationProperties.getOutputPath()+File.separator+this.transportTask.getTaskName()).append("\n");
if(TransportTaskModeEnum.FORWARD.getKey().equals(this.transportTask.getTaskMode())){
paramContent.append(54).append("\n");//物种先固定写54对应XE135
} else if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(this.transportTask.getTaskMode())) {
paramContent.append(64).append("\n");//物种先固定对应XE135
}
FileUtil.writeString(paramContent.toString(),paramConfigPath,"UTF-8");
//处理台站数据文件
List<String> stationConfigInfo = new ArrayList<>();
String stationsConfigPath = this.getStationsConfigPath(this.transportTask.getUseMetType());
if(!FileUtil.exist(stationsConfigPath)){
FileUtil.touch(stationsConfigPath);
}
this.transportTaskChildren.forEach(taskChild -> {
String format = "%s,%f,%f,%s";
String row = String.format(format,taskChild.getStationCode(),taskChild.getLat(),taskChild.getLon(),taskChild.getReleaseAmount());
stationConfigInfo.add(row);
});
//最后一行需要换行，否则启动flexpart报错
stationConfigInfo.add("\n");
FileUtil.writeLines(stationConfigInfo,stationsConfigPath,"UTF-8");
//获取脚本路径
String scriptPath = null;
if(TransportTaskModeEnum.FORWARD.getKey().equals(this.transportTask.getTaskMode())){
scriptPath = this.getForwardScriptPath(this.transportTask.getUseMetType());
} else if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(this.transportTask.getTaskMode())) {
scriptPath = this.getBackForwardScriptPath(this.transportTask.getUseMetType());
}
String execScriptMsg = "执行任务脚本，开始模拟，路径为："+scriptPath;
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),execScriptMsg));
ProcessBuilder processBuilder = new ProcessBuilder(scriptPath);
processBuilder.directory(new File(new File(scriptPath).getParent()));
processBuilder.redirectErrorStream(true);
process = processBuilder.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream(), "UTF-8"));
//读取输出日志
String line;
while ((line = reader.readLine()) != null) {
if(StrUtil.isNotBlank(line)){
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),line));
}
}
//等待进程结束
process.waitFor();
} catch (Exception e) {
throw new RuntimeException(e);
}finally {
if(Objects.nonNull(process)){
process.destroy();
}
}
}
/**
* 获取气象数据路径
* @param dataSource
* @return
*/
private String getMetDataPath(Integer dataSource){
StringBuilder path = new StringBuilder();
path.append(systemStorageProperties.getRootPath());
path.append(File.separator);
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
path.append(systemStorageProperties.getPangu());
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
path.append(systemStorageProperties.getGraphcast());
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
path.append(systemStorageProperties.getCra40());
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
path.append(systemStorageProperties.getNcep());
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
path.append(systemStorageProperties.getFnl());
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
path.append(systemStorageProperties.getT1h());
}
return path.toString();
}
/**
* 获取参数配置文件路径
* @param dataSource
* @return
*/
private String getParamConfigPath(Integer dataSource){
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
return simulationProperties.getPanguParamConfigPath();
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
return simulationProperties.getGraphcastParamConfigPath();
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
return simulationProperties.getCra40ParamConfigPath();
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
return simulationProperties.getNcepParamConfigPath();
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
return simulationProperties.getFnlParamConfigPath();
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
return simulationProperties.getT1hParamConfigPath();
}
return Strings.EMPTY;
}
/**
* 获取台站参数配置文件路径
* @param dataSource
* @return
*/
private String getStationsConfigPath(Integer dataSource){
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
return simulationProperties.getPanguStationsConfigPath();
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
return simulationProperties.getCra40StationsConfigPath();
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
return simulationProperties.getCra40StationsConfigPath();
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
return simulationProperties.getNcepStationsConfigPath();
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
return simulationProperties.getFnlStationsConfigPath();
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
return simulationProperties.getT1hStationsConfigPath();
}
return Strings.EMPTY;
}
/**
* 获取正演脚本路径
* @param dataSource
* @return
*/
private String getForwardScriptPath(Integer dataSource){
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
return simulationProperties.getPanguForwardScriptPath();
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
return simulationProperties.getGraphcastForwardScriptPath();
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
return simulationProperties.getCra40ForwardScriptPath();
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
return simulationProperties.getNcepForwardScriptPath();
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
return simulationProperties.getFnlForwardScriptPath();
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
return simulationProperties.getT1hForwardScriptPath();
}
return Strings.EMPTY;
}
/**
* 获取正演脚本路径
* @param dataSource
* @return
*/
private String getBackForwardScriptPath(Integer dataSource){
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
return simulationProperties.getPanguBackwardScriptPath();
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
return simulationProperties.getGraphcastBackwardScriptPath();
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
return simulationProperties.getCra40BackwardScriptPath();
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
return simulationProperties.getNcepBackwardScriptPath();
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
return simulationProperties.getFnlBackwardScriptPath();
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
return simulationProperties.getT1hBackwardScriptPath();
}
return Strings.EMPTY;
}
}

460
jeecg-module-transport/src/main/java/org/jeecg/task/nuclearfacilitiestask/AssociatedNuclearFacilitiesTaskExec.java Normal file
View File

@ -0,0 +1,460 @@
package org.jeecg.task.nuclearfacilitiestask;
import cn.hutool.core.collection.CollUtil;
import cn.hutool.core.date.DateUtil;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.time.StopWatch;
import org.apache.logging.log4j.util.Strings;
import org.jeecg.common.constant.enums.AssociatedWaveformTaskEnum;
import org.jeecg.common.constant.enums.WaveformEventResultTypeEnum;
import org.jeecg.common.properties.DataFusionProperties;
import org.jeecg.config.datasource.DataSourceSwitcher;
import org.jeecg.modules.base.entity.copsdb.Origin;
import org.jeecg.modules.base.entity.rnauto.GardsWaveformEvent;
import org.jeecg.modules.base.entity.rnauto.GardsWaveformEventResult;
import org.jeecg.modules.base.mapper.GardsWaveformEventMapper;
import org.jeecg.modules.base.mapper.GardsWaveformEventResultMapper;
import org.jeecg.modules.base.mapper.OriginMapper;
import org.jeecg.vo.SRSRecord;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.transaction.TransactionDefinition;
import org.springframework.transaction.TransactionStatus;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStreamReader;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.nio.charset.StandardCharsets;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import java.util.zip.GZIPInputStream;
@Slf4j
public class AssociatedNuclearFacilitiesTaskExec extends Thread{
private DataSourceTransactionManager transactionManager;
private TransactionDefinition transactionDefinition;
private DataFusionProperties dataFusionProperties;
private GardsWaveformEventMapper waveformEventMapper;
private GardsWaveformEventResultMapper waveformEventResultMapper;
private OriginMapper originMapper;
private Integer sampleId;
private Date acqEndTime;
private String stationCode;
private File idcSrsFile = null;
private File ndcSrsFile = null;
private List<String> idcSrsContents;
private List<String> ndcSrsContents;
/**
* 初始化
*/
public void init(
DataSourceTransactionManager transactionManager,
TransactionDefinition transactionDefinition,
DataFusionProperties dataFusionProperties,
GardsWaveformEventMapper waveformEventMapper,
OriginMapper originMapper,
GardsWaveformEventResultMapper waveformEventResultMapper,
Integer sampleId,
Date acqEndTime,
String stationCode){
this.transactionManager = transactionManager;
this.transactionDefinition = transactionDefinition;
this.dataFusionProperties = dataFusionProperties;
this.waveformEventMapper = waveformEventMapper;
this.originMapper = originMapper;
this.waveformEventResultMapper = waveformEventResultMapper;
this.sampleId = sampleId;
this.acqEndTime = acqEndTime;
this.stationCode = stationCode;
}
@Override
public void run() {
this.execute();
}
/**
* 执行任务
*/
public void execute() {
StopWatch stopWatch = new StopWatch();
stopWatch.start();
try{
//清除历史关联结果
this.cleanHistoryResult(this.sampleId);
//修改任务状态为执行中
this.updateTaskStatus(this.sampleId, AssociatedWaveformTaskEnum.IN_PROCESS.getValue());
//检查srs文件是否存在
this.checkSRSExist();
//解析SRS文件数据
this.parseSrsFile();
//关联波形数据
this.associationWaveform();
this.execComplete(stopWatch,AssociatedWaveformTaskEnum.COMPLETE.getValue(),null);
}catch (Exception e){
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
this.execComplete(stopWatch,AssociatedWaveformTaskEnum.UNSUCCESSFUL.getValue(),taskErrorLog);
throw e;
}
}
/**
* 检查IDC和NDC SRS文件是否存在
*/
private void checkSRSExist(){
String acqEndDate = DateUtil.format(this.acqEndTime,"yyyyMMdd");
this.checkIdcSrsExist(acqEndDate);
this.checkNdcSrsExist(acqEndDate);
}
/**
* 解析SRS文件
*/
private void parseSrsFile(){
if(Objects.nonNull(this.idcSrsFile)){
this.idcSrsContents = this.readSrmFromGzStreaming(this.idcSrsFile);
}
if(Objects.nonNull(this.ndcSrsFile)){
this.ndcSrsContents = this.readSrmFromGzStreaming(this.ndcSrsFile);
}
}
/**
* 关联波形数据
*/
private void associationWaveform(){
Set<Integer> idcEvents = this.associationSrs(this.idcSrsContents);
Set<Integer> ndcEvents = this.associationSrs(this.ndcSrsContents);
DataSourceSwitcher.switchToOracle();
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
try {
//保存关联结果
if (CollUtil.isNotEmpty(idcEvents)){
this.saveWaveformEventResult(idcEvents,WaveformEventResultTypeEnum.IDC.getKey());
}
if (CollUtil.isNotEmpty(ndcEvents)){
this.saveWaveformEventResult(ndcEvents,WaveformEventResultTypeEnum.NDC.getKey());
}
//修改关联结果统计
GardsWaveformEvent waveformEvent = this.waveformEventMapper.selectById(sampleId);
waveformEvent.setIdcEvents(idcEvents.size());
waveformEvent.setNdcEvents(ndcEvents.size());
waveformEvent.setDescription(Strings.EMPTY);
waveformEvent.setModdate(new Date());
waveformEvent.setStatus(AssociatedWaveformTaskEnum.COMPLETE.getValue());
this.waveformEventMapper.updateById(waveformEvent);
this.transactionManager.commit(transactionStatus);
}finally {
DataSourceSwitcher.clearDataSource();
}
}
/**
* 关联idc srs文件
*/
private Set<Integer> associationSrs(List<String> srsContents){
// 台站经度 台站纬度 开始测量日期 小时 结束测量时间 小时 系数 总共模拟时长 小时数 网格大小 台站编码
// 139.08 36.30 20241203 12 20241203 18 0.1300000E+16 336 1 1 0.50 0.50 "JPX38"
String[] firstLine = srsContents.get(0).split("\\s+");
Integer totalHour = Integer.parseInt(firstLine[8]);
Integer hourlyCoefficient = Integer.parseInt(firstLine[9]);
Double gridLonResolution = Double.parseDouble(firstLine[11]);
Double gridLatResolution = Double.parseDouble(firstLine[12]);
//根据测量结束时间，生成时间范围，用于查询范围内的波形事件
LocalDateTime endTime = LocalDateTime.ofInstant(this.acqEndTime.toInstant(), ZoneId.of("Asia/Shanghai"));
endTime = endTime.withMinute(0).withSecond(0);
LocalDateTime startTime = endTime.minusHours(totalHour).minusHours(dataFusionProperties.getTraceabilityTime()*24);
System.out.println(startTime.format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")));
System.out.println(endTime.format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")));
//查询范围内波形事件
List<Origin> origins = this.selectOriginByTime(startTime,endTime);
//处理srs文件中的坐标记录格式化成对象
List<SRSRecord> srsRecords = buildSRSRecord(srsContents, hourlyCoefficient,endTime);
//怎么对srsRecords进行按经纬度分组，分组后，对每组经纬度只求一个网格边界，然后再根据时间做origins查询，统计关联数
Map<Map.Entry<Double, Double>, List<SRSRecord>> srsRecordsGroup = srsRecords.stream()
.collect(Collectors.groupingBy(record ->
new AbstractMap.SimpleImmutableEntry<>(record.getLon(), record.getLat())
));
return this.queryWaveform(origins, srsRecordsGroup, gridLonResolution,gridLatResolution);
}
/**
* 检查IDC SRS文件是否存在
*/
private void checkIdcSrsExist(String acqEndDate){
//构建srs文件路径
StringBuilder srsFilePath = new StringBuilder();
srsFilePath.append(dataFusionProperties.getIdcSRSPath());
srsFilePath.append(File.separator);
srsFilePath.append(acqEndDate);
srsFilePath.append(File.separator);
srsFilePath.append(dataFusionProperties.getIdcSrmParentDir());
//构建srm.gz文件名称
StringBuilder srmFileName = new StringBuilder();
srmFileName.append(stationCode);
srmFileName.append(".fp.");
srmFileName.append(DateUtil.format(this.acqEndTime,"yyyyMMddHH"));
srmFileName.append(".f9.srm.gz");
String fullFilePath = srsFilePath + File.separator + srmFileName;
File idcSrsFile = new File(fullFilePath);
if (idcSrsFile.exists()){
this.idcSrsFile = idcSrsFile;
}else {
this.updateTaskNotSuccessful("IDC SRS文件不存在");
}
}
/**
* 检查NDC SRS文件是否存在
*/
private void checkNdcSrsExist(String acqEndDate){
//构建srs文件路径
StringBuilder srsFilePath = new StringBuilder();
srsFilePath.append(dataFusionProperties.getNdcSRSPath());
srsFilePath.append(File.separator);
srsFilePath.append(acqEndDate);
srsFilePath.append(File.separator);
srsFilePath.append(dataFusionProperties.getNdcSrmParentDir());
//构建srm.gz文件名称
StringBuilder srmFileName = new StringBuilder();
srmFileName.append(stationCode);
srmFileName.append(".fp.");
srmFileName.append(DateUtil.format(this.acqEndTime,"yyyyMMddHH"));
srmFileName.append(".f9.srm.gz");
String fullFilePath = srsFilePath + File.separator + srmFileName;
File ndcSrsFile = new File(fullFilePath);
if (ndcSrsFile.exists()){
this.ndcSrsFile = idcSrsFile;
}else {
this.updateTaskNotSuccessful("NDC SRS文件不存在");
}
}
/**
* 处理srs文件中的坐标记录格式化成对象
* @param contents
* @param hourlyCoefficient
* @param acqEndTime 测量结束时间
* @return
*/
private List<SRSRecord> buildSRSRecord(List<String> contents,Integer hourlyCoefficient,LocalDateTime acqEndTime){
List<SRSRecord> list = contents.parallelStream().skip(1).map(content->{
String[] line = content.split("\\s+");
BigDecimal val1 = new BigDecimal(line[4]);
BigDecimal val2 = new BigDecimal(dataFusionProperties.getFilterConditions());
//过滤掉比较小的值
if(val1.compareTo(val2) > 0){
SRSRecord srsRecord = new SRSRecord();
srsRecord.setLat(Double.parseDouble(line[1]));
srsRecord.setLon(Double.parseDouble(line[2]));
srsRecord.setHour(Integer.parseInt(line[3])*hourlyCoefficient);
srsRecord.setConc(line[4]);
LocalDateTime endTime = acqEndTime.minusHours(srsRecord.getHour());
LocalDateTime startTime = acqEndTime.minusHours(srsRecord.getHour()+dataFusionProperties.getTraceabilityTime()*24);
long startSecond = startTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
long endSecond = endTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
srsRecord.setStartSecond(startSecond);
srsRecord.setEndSecond(endSecond);
return srsRecord;
}
return null;
}).filter(Objects::nonNull)
.toList();
return list;
}
/**
* 根据经纬度及时间查询范围内的波形数据，并进行计数，最后的计数就是关联的波形结果
* @param srsRecordsGroup
*/
private Set<Integer> queryWaveform(List<Origin> origins,Map<Map.Entry<Double, Double>, List<SRSRecord>> srsRecordsGroup,Double gridLonResolution,Double gridLatResolution){
Set<Integer> result = ConcurrentHashMap.newKeySet();
if (CollUtil.isNotEmpty(srsRecordsGroup) && CollUtil.isNotEmpty(origins)){
srsRecordsGroup.forEach((key,srsRecords)->{
double leftLon = Math.floor(key.getKey() / gridLonResolution) * gridLonResolution;
double bottomLat = Math.floor(key.getValue() / gridLatResolution) * gridLatResolution;
double rightLon = leftLon + gridLonResolution;
double topLat = bottomLat + gridLatResolution;
//统计srs中每条记录内关联的波形数据
srsRecords.parallelStream().forEach(record->{
Set<Integer> orids = origins.stream()
.filter(origin ->origin.getLon() >= leftLon && origin.getLon() < rightLon &&
origin.getLat() >= bottomLat && origin.getLat() < topLat &&
origin.getTime() >= record.getStartSecond() && origin.getTime() <= record.getEndSecond())
.map(Origin::getOrid).collect(Collectors.toSet());
if (CollUtil.isNotEmpty(orids)){
result.addAll(orids);
}
});
});
return result;
}
return result;
}
/**
* 修改任务状态
* @param sampleId
* @param status
*/
private void updateTaskStatus(Integer sampleId, Integer status) {
DataSourceSwitcher.switchToOracle();
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
try {
GardsWaveformEvent waveformEvent = this.waveformEventMapper.selectById(sampleId);
waveformEvent.setStatus(status);
waveformEvent.setModdate(new Date());
this.waveformEventMapper.updateById(waveformEvent);
this.transactionManager.commit(transactionStatus);
}catch (Exception e){
this.transactionManager.rollback(transactionStatus);
log.error(e.getMessage(),e);
}finally {
DataSourceSwitcher.clearDataSource();
}
}
/**
* 修改任务关联未成功
* @param description
*/
private void updateTaskNotSuccessful(String description) {
DataSourceSwitcher.switchToOracle();
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
try {
GardsWaveformEvent waveformEvent = this.waveformEventMapper.selectById(this.sampleId);
waveformEvent.setStatus(AssociatedWaveformTaskEnum.UNSUCCESSFUL.getValue());
waveformEvent.setDescription(waveformEvent.getDescription()+"\n"+description);
waveformEvent.setModdate(new Date());
this.waveformEventMapper.updateById(waveformEvent);
this.transactionManager.commit(transactionStatus);
}catch (Exception e){
this.transactionManager.rollback(transactionStatus);
log.error(e.getMessage(),e);
}finally {
DataSourceSwitcher.clearDataSource();
}
}
/**
* 清除历史关联结果
* @param sampleId
*/
private void cleanHistoryResult(Integer sampleId) {
DataSourceSwitcher.switchToOracle();
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
try {
LambdaQueryWrapper<GardsWaveformEventResult> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(GardsWaveformEventResult::getSampleId, sampleId);
this.waveformEventResultMapper.delete(queryWrapper);
this.transactionManager.commit(transactionStatus);
}catch (Exception e){
this.transactionManager.rollback(transactionStatus);
log.error(e.getMessage(),e);
}finally {
DataSourceSwitcher.clearDataSource();
}
}
/**
* 获取时间范围内的波形数据
* @param startTime
* @param endTime
* @return
*/
private List<Origin> selectOriginByTime(LocalDateTime startTime,LocalDateTime endTime){
DataSourceSwitcher.switchToOracle();
try {
long startSecond = startTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
long endSecond = endTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
return originMapper.selectOriginByTime(startSecond,endSecond);
}finally {
DataSourceSwitcher.clearDataSource();
}
}
/**
* 保存关联结果
* @param orids
*/
private void saveWaveformEventResult(Set<Integer> orids,Integer type){
DataSourceSwitcher.switchToOracle();
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
try {
if(CollUtil.isNotEmpty(orids)){
List<GardsWaveformEventResult> waveformEventResults = new ArrayList<>();
orids.forEach(orid->{
GardsWaveformEventResult waveformEventResult = new GardsWaveformEventResult();
waveformEventResult.setSampleId(this.sampleId);
waveformEventResult.setOrid(orid);
waveformEventResult.setType(type);
waveformEventResult.setModdate(new Date());
waveformEventResults.add(waveformEventResult);
});
this.waveformEventResultMapper.insert(waveformEventResults);
this.transactionManager.commit(transactionStatus);
}
}finally {
DataSourceSwitcher.clearDataSource();
}
}
/**
* 读取srm文件
* @param gzFile
* @return
*/
public List<String> readSrmFromGzStreaming(File gzFile){
try {
List<String> contents = new ArrayList<>();
try (FileInputStream fis = new FileInputStream(gzFile);
GZIPInputStream gis = new GZIPInputStream(fis);
BufferedReader reader = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8))) {
String line;
while ((line = reader.readLine()) != null) {
contents.add(line);
}
}
return contents;
}catch (Exception e){
log.error("SRS文件解析异常，路径为：{},原因为：{}",gzFile.getAbsolutePath(),e.getMessage());
return List.of();
}
}
/**
* 任务执行完成
* @param stopWatch
* @param taskStatus
* @param taskErrorLog
*/
private void execComplete(StopWatch stopWatch,Integer taskStatus,String taskErrorLog){
//添加任务耗时
stopWatch.stop();
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
double min = seconds/60D;
BigDecimal bgMin = new BigDecimal(min);
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
if (AssociatedWaveformTaskEnum.COMPLETE.getValue().equals(taskStatus)){
log.info("任务执行完成，耗时{}分钟",result);
}else if (AssociatedWaveformTaskEnum.UNSUCCESSFUL.getValue().equals(taskStatus)){
log.error(taskErrorLog);
}
this.updateTaskStatus(this.sampleId,taskStatus);
}
}

32
jeecg-module-transport/src/main/java/org/jeecg/task/waveformtask/AssociatedWaveformTaskExec.java
View File

@ -164,7 +164,8 @@ public class AssociatedWaveformTaskExec extends Thread{
String[] firstLine = srsContents.get(0).split("\\s+"); String[] firstLine = srsContents.get(0).split("\\s+");
Integer totalHour = Integer.parseInt(firstLine[8]); Integer totalHour = Integer.parseInt(firstLine[8]);
Integer hourlyCoefficient = Integer.parseInt(firstLine[9]); Integer hourlyCoefficient = Integer.parseInt(firstLine[9]);
Double gridSize = Double.parseDouble(firstLine[11]); Double gridLonResolution = Double.parseDouble(firstLine[11]);
Double gridLatResolution = Double.parseDouble(firstLine[12]);
//根据测量结束时间，生成时间范围，用于查询范围内的波形事件 //根据测量结束时间，生成时间范围，用于查询范围内的波形事件
LocalDateTime endTime = LocalDateTime.ofInstant(this.acqEndTime.toInstant(), ZoneId.of("Asia/Shanghai")); LocalDateTime endTime = LocalDateTime.ofInstant(this.acqEndTime.toInstant(), ZoneId.of("Asia/Shanghai"));
endTime = endTime.withMinute(0).withSecond(0); endTime = endTime.withMinute(0).withSecond(0);
@ -180,7 +181,7 @@ public class AssociatedWaveformTaskExec extends Thread{
.collect(Collectors.groupingBy(record -> .collect(Collectors.groupingBy(record ->
new AbstractMap.SimpleImmutableEntry<>(record.getLon(), record.getLat()) new AbstractMap.SimpleImmutableEntry<>(record.getLon(), record.getLat())
)); ));
return this.queryWaveform(origins, srsRecordsGroup, gridSize); return this.queryWaveform(origins, srsRecordsGroup, gridLonResolution,gridLatResolution);
} }
/** /**
@ -193,7 +194,7 @@ public class AssociatedWaveformTaskExec extends Thread{
srsFilePath.append(File.separator); srsFilePath.append(File.separator);
srsFilePath.append(acqEndDate); srsFilePath.append(acqEndDate);
srsFilePath.append(File.separator); srsFilePath.append(File.separator);
srsFilePath.append(dataFusionProperties.getSrmParentDir()); srsFilePath.append(dataFusionProperties.getIdcSrmParentDir());
//构建srm.gz文件名称 //构建srm.gz文件名称
StringBuilder srmFileName = new StringBuilder(); StringBuilder srmFileName = new StringBuilder();
@ -221,7 +222,7 @@ public class AssociatedWaveformTaskExec extends Thread{
srsFilePath.append(File.separator); srsFilePath.append(File.separator);
srsFilePath.append(acqEndDate); srsFilePath.append(acqEndDate);
srsFilePath.append(File.separator); srsFilePath.append(File.separator);
srsFilePath.append(dataFusionProperties.getSrmParentDir()); srsFilePath.append(dataFusionProperties.getNdcSrmParentDir());
//构建srm.gz文件名称 //构建srm.gz文件名称
StringBuilder srmFileName = new StringBuilder(); StringBuilder srmFileName = new StringBuilder();
@ -276,14 +277,14 @@ public class AssociatedWaveformTaskExec extends Thread{
* 根据经纬度及时间查询范围内的波形数据，并进行计数，最后的计数就是关联的波形结果 * 根据经纬度及时间查询范围内的波形数据，并进行计数，最后的计数就是关联的波形结果
* @param srsRecordsGroup * @param srsRecordsGroup
*/ */
private Set<Integer> queryWaveform(List<Origin> origins,Map<Map.Entry<Double, Double>, List<SRSRecord>> srsRecordsGroup,Double gridSize){ private Set<Integer> queryWaveform(List<Origin> origins,Map<Map.Entry<Double, Double>, List<SRSRecord>> srsRecordsGroup,Double gridLonResolution,Double gridLatResolution){
Set<Integer> result = ConcurrentHashMap.newKeySet(); Set<Integer> result = ConcurrentHashMap.newKeySet();
if (CollUtil.isNotEmpty(srsRecordsGroup) && CollUtil.isNotEmpty(origins)){ if (CollUtil.isNotEmpty(srsRecordsGroup) && CollUtil.isNotEmpty(origins)){
srsRecordsGroup.forEach((key,srsRecords)->{ srsRecordsGroup.forEach((key,srsRecords)->{
double leftLon = Math.floor(key.getKey() / gridSize) * gridSize; double leftLon = Math.floor(key.getKey() / gridLonResolution) * gridLonResolution;
double bottomLat = Math.floor(key.getValue() / gridSize) * gridSize; double bottomLat = Math.floor(key.getValue() / gridLatResolution) * gridLatResolution;
double rightLon = leftLon + gridSize; double rightLon = leftLon + gridLonResolution;
double topLat = bottomLat + gridSize; double topLat = bottomLat + gridLatResolution;
//统计srs中每条记录内关联的波形数据 //统计srs中每条记录内关联的波形数据
srsRecords.parallelStream().forEach(record->{ srsRecords.parallelStream().forEach(record->{
Set<Integer> orids = origins.stream() Set<Integer> orids = origins.stream()
@ -453,17 +454,4 @@ public class AssociatedWaveformTaskExec extends Thread{
} }
this.updateTaskStatus(this.sampleId,taskStatus); this.updateTaskStatus(this.sampleId,taskStatus);
} }
public static void main(String[] args) {
LocalDateTime endTime = LocalDateTime.of(2024,12, 3,18, 0, 0);
LocalDateTime startTime = endTime.minusDays(74);
long startSecond = startTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
long endSecond = endTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
System.out.println(startSecond);
System.out.println(endSecond);
}
} }

3
jeecg-module-weather/src/main/java/org/jeecg/service/impl/WeatherDataServiceImpl.java
View File

@ -321,7 +321,8 @@ public class WeatherDataServiceImpl extends ServiceImpl<WeatherDataMapper, Weath
queryWrapper.between((Objects.nonNull(startTime) && Objects.nonNull(endTime)),WeatherData::getDataStartTime,startTime,endTime); queryWrapper.between((Objects.nonNull(startTime) && Objects.nonNull(endTime)),WeatherData::getDataStartTime,startTime,endTime);
queryWrapper.eq(StringUtils.isNotBlank(fileExt),WeatherData::getFileExt, fileExt); queryWrapper.eq(StringUtils.isNotBlank(fileExt),WeatherData::getFileExt, fileExt);
queryWrapper.like(StringUtils.isNotBlank(fileName),WeatherData::getFileName, fileName); queryWrapper.like(StringUtils.isNotBlank(fileName),WeatherData::getFileName, fileName);
queryWrapper.select(WeatherData::getId,WeatherData::getFileName,WeatherData::getFileSize,WeatherData::getDataSource,WeatherData::getFileExt,WeatherData::getDataStartTime,WeatherData::getFilePath); queryWrapper.select(WeatherData::getId,WeatherData::getFileName,WeatherData::getFileSize,WeatherData::getDataSource,
WeatherData::getFileExt,WeatherData::getDataStartTime,WeatherData::getFilePath,WeatherData::getTimeBatch);
IPage<WeatherData> iPage = new Page<>(pageRequest.getPageNum(),pageRequest.getPageSize()); IPage<WeatherData> iPage = new Page<>(pageRequest.getPageNum(),pageRequest.getPageSize());
return this.baseMapper.selectPage(iPage, queryWrapper); return this.baseMapper.selectPage(iPage, queryWrapper);
} }

13
jeecg-module-weather/src/main/java/org/jeecg/service/impl/WeatherTaskServiceImpl.java
View File

@ -9,6 +9,7 @@ import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import org.apache.logging.log4j.util.Strings; import org.apache.logging.log4j.util.Strings;
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
import org.jeecg.common.constant.enums.WeatherFileSuffixEnum; import org.jeecg.common.constant.enums.WeatherFileSuffixEnum;
import org.jeecg.common.constant.enums.WeatherForecastDatasourceEnum; import org.jeecg.common.constant.enums.WeatherForecastDatasourceEnum;
import org.jeecg.common.constant.enums.WeatherTaskStatusEnum; import org.jeecg.common.constant.enums.WeatherTaskStatusEnum;
@ -20,7 +21,8 @@ import org.jeecg.modules.base.mapper.WeatherDataMapper;
import org.jeecg.modules.base.mapper.WeatherTaskLogMapper; import org.jeecg.modules.base.mapper.WeatherTaskLogMapper;
import org.jeecg.modules.base.mapper.WeatherTaskMapper; import org.jeecg.modules.base.mapper.WeatherTaskMapper;
import org.jeecg.service.WeatherTaskService; import org.jeecg.service.WeatherTaskService;
import org.jeecg.task.WeatherTaskExec; import org.jeecg.task.GraphcastWeatherTaskExec;
import org.jeecg.task.PanGuWeatherTaskExec;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional; import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.multipart.MultipartFile; import org.springframework.web.multipart.MultipartFile;
@ -225,10 +227,17 @@ public class WeatherTaskServiceImpl extends ServiceImpl<WeatherTaskMapper, Weath
if(Objects.isNull(task)){ if(Objects.isNull(task)){
throw new RuntimeException("此任务不存在"); throw new RuntimeException("此任务不存在");
} }
WeatherTaskExec exec = new WeatherTaskExec(); if (WeatherDataSourceEnum.PANGU.getKey().equals(task.getPredictionModel())){
PanGuWeatherTaskExec exec = new PanGuWeatherTaskExec();
exec.init(task,this,systemStorageProperties,weatherDataMapper); exec.init(task,this,systemStorageProperties,weatherDataMapper);
exec.setName("天气预测任务线程"); exec.setName("天气预测任务线程");
exec.start(); exec.start();
} else if (WeatherDataSourceEnum.GRAPHCAST.getKey().equals(task.getPredictionModel())) {
GraphcastWeatherTaskExec exec = new GraphcastWeatherTaskExec();
exec.init(task,this,systemStorageProperties,weatherDataMapper);
exec.setName("天气预测任务线程");
exec.start();
}
} }
/** /**

74
jeecg-module-weather/src/main/java/org/jeecg/task/WeatherTaskExec.java → jeecg-module-weather/src/main/java/org/jeecg/task/GraphcastWeatherTaskExec.java
View File

@ -1,7 +1,5 @@
package org.jeecg.task; package org.jeecg.task;
import cn.hutool.core.collection.CollUtil;
import cn.hutool.core.date.DateUtil;
import cn.hutool.core.date.LocalDateTimeUtil; import cn.hutool.core.date.LocalDateTimeUtil;
import cn.hutool.core.io.FileUtil; import cn.hutool.core.io.FileUtil;
import cn.hutool.core.util.StrUtil; import cn.hutool.core.util.StrUtil;
@ -17,16 +15,13 @@ import org.jeecg.common.properties.SystemStorageProperties;
import org.jeecg.common.util.NcUtil; import org.jeecg.common.util.NcUtil;
import org.jeecg.modules.base.entity.WeatherData; import org.jeecg.modules.base.entity.WeatherData;
import org.jeecg.modules.base.entity.WeatherTask; import org.jeecg.modules.base.entity.WeatherTask;
import org.jeecg.modules.base.entity.WeatherTaskLog;
import org.jeecg.modules.base.mapper.WeatherDataMapper; import org.jeecg.modules.base.mapper.WeatherDataMapper;
import org.jeecg.service.WeatherDataService;
import org.jeecg.service.WeatherTaskService; import org.jeecg.service.WeatherTaskService;
import java.io.*; import java.io.*;
import java.math.BigDecimal; import java.math.BigDecimal;
import java.math.RoundingMode; import java.math.RoundingMode;
import java.time.Instant; import java.time.Instant;
import java.time.LocalDate;
import java.time.LocalDateTime; import java.time.LocalDateTime;
import java.time.ZoneId; import java.time.ZoneId;
import java.time.format.DateTimeFormatter; import java.time.format.DateTimeFormatter;
@ -38,7 +33,7 @@ import java.util.concurrent.TimeUnit;
/** /**
* 气象预测任务执行线程 * 气象预测任务执行线程
*/ */
public class WeatherTaskExec extends Thread{ public class GraphcastWeatherTaskExec extends Thread{
private WeatherTask weatherTask; private WeatherTask weatherTask;
private WeatherTaskService weatherTaskService; private WeatherTaskService weatherTaskService;
@ -116,11 +111,11 @@ public class WeatherTaskExec extends Thread{
//输入文件 //输入文件
String inputFileName = this.weatherTask.getInputFile(); String inputFileName = this.weatherTask.getInputFile();
//预测文件 //预测文件
String outputFileName = "panguweather_output_"+this.weatherTask.getId()+".grib"; String outputFileName = "graphcast_output_"+this.weatherTask.getId()+".grib";
//定义名称后缀,执行grib_set命令时去除，否则命名会冲突 //定义名称后缀,执行grib_set命令时去除，否则命名会冲突
String gribFileSuffix = "_not_grib_set"; String gribFileSuffix = "_not_grib_set";
//临时目录grib_copy,grib_set,python转换都在这个目录里，最后删除 //临时目录grib_copy,grib_set,python转换都在这个目录里，最后删除
String tempDir = systemStorageProperties.getPanguModelExecPath()+File.separator+this.weatherTask.getId(); String tempDir = systemStorageProperties.getGraphcastModelExecPath()+File.separator+this.weatherTask.getId();
if(!FileUtil.exist(tempDir)){ if(!FileUtil.exist(tempDir)){
FileUtil.mkdir(tempDir); FileUtil.mkdir(tempDir);
} }
@ -129,23 +124,23 @@ public class WeatherTaskExec extends Thread{
//切割文件，6小时一个 //切割文件，6小时一个
this.splitGrib(outputFileName,gribFileSuffix,tempDir); this.splitGrib(outputFileName,gribFileSuffix,tempDir);
//适配flexpart //适配flexpart
this.convertGrib(gribFileSuffix,tempDir); // this.convertGrib(gribFileSuffix,tempDir);
//保存文件数据入库 //保存文件数据入库
this.saveGribInfoToDB(tempDir,this.getPanguWeatherPath()); this.saveGribInfoToDB(tempDir,this.getPanguWeatherPath(),gribFileSuffix);
//删除预测文件 //删除预测文件
File inputFile = new File(systemStorageProperties.getPanguModelExecPath()+File.separator+inputFileName); // File inputFile = new File(systemStorageProperties.getGraphcastModelExecPath()+File.separator+inputFileName);
if (inputFile.exists()){ // if (inputFile.exists()){
inputFile.delete(); // inputFile.delete();
} // }
//删除输出的总文件 // //删除输出的总文件
File outputFile = new File(systemStorageProperties.getPanguModelExecPath()+File.separator+outputFileName); // File outputFile = new File(systemStorageProperties.getGraphcastModelExecPath()+File.separator+outputFileName);
if (outputFile.exists()){ // if (outputFile.exists()){
outputFile.delete(); // outputFile.delete();
} // }
//适配flexpart结束，删除目录 // //适配flexpart结束，删除目录
if(FileUtil.exist(tempDir)){ // if(FileUtil.exist(tempDir)){
FileUtil.del(tempDir); // FileUtil.del(tempDir);
} // }
} catch (Exception e) { } catch (Exception e) {
throw new RuntimeException(e); throw new RuntimeException(e);
} }
@ -159,15 +154,10 @@ public class WeatherTaskExec extends Thread{
private void forecast(String inputFileAddr,String outputFileAddr){ private void forecast(String inputFileAddr,String outputFileAddr){
try { try {
//处理开始日志 //处理开始日志
String forecastModel = "";
String startLogFormat = ""; String startLogFormat = "";
String startTimeFormat = LocalDateTimeUtil.format(this.weatherTask.getStartDate(),"yyyy-MM-dd HH:mm:ss"); String startTimeFormat = LocalDateTimeUtil.format(this.weatherTask.getStartDate(),"yyyy-MM-dd HH:mm:ss");
String forecastTime = this.weatherTask.getLeadTime().toString(); String forecastTime = this.weatherTask.getLeadTime().toString();
if (WeatherDataSourceEnum.GRAPHCAST.getKey().equals(this.weatherTask.getPredictionModel())){ String forecastModel = WeatherDataSourceEnum.GRAPHCAST.getValue();
forecastModel = WeatherDataSourceEnum.GRAPHCAST.getValue();
}else if(WeatherDataSourceEnum.PANGU.getKey().equals(this.weatherTask.getPredictionModel())){
forecastModel = WeatherDataSourceEnum.PANGU.getValue();
}
if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){ if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){
startLogFormat = "任务开始，本次使用预测模型为：%s，预测开始时间为：%s，预测时长为：%s，前置数据采用CDS在线数据"; startLogFormat = "任务开始，本次使用预测模型为：%s，预测开始时间为：%s，预测时长为：%s，前置数据采用CDS在线数据";
}else { }else {
@ -177,7 +167,7 @@ public class WeatherTaskExec extends Thread{
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),startLog)); ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),startLog));
//处理运行命令 //处理运行命令
List<String> command = new ArrayList<>(); List<String> command = new ArrayList<>();
String aiModelsPath = systemStorageProperties.getPanguEnvPath()+File.separator+"ai-models"; String aiModelsPath = systemStorageProperties.getGraphcastEnvPath()+File.separator+"ai-models";
if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){ if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){
command.add(aiModelsPath); command.add(aiModelsPath);
@ -192,8 +182,8 @@ public class WeatherTaskExec extends Thread{
command.add("--path"); command.add("--path");
command.add(outputFileAddr); command.add(outputFileAddr);
command.add("--assets"); command.add("--assets");
command.add("assets-panguweather"); command.add("assets-graphcast");
command.add("panguweather"); command.add("--class od graphcast");
}else if (this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.LOCATION_FILE.getKey())){ }else if (this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.LOCATION_FILE.getKey())){
//离线文件还需处理 //离线文件还需处理
command.add(aiModelsPath); command.add(aiModelsPath);
@ -204,13 +194,13 @@ public class WeatherTaskExec extends Thread{
command.add("--path"); command.add("--path");
command.add(outputFileAddr); command.add(outputFileAddr);
command.add("--assets"); command.add("--assets");
command.add("assets-panguweather"); command.add("assets-graphcast");
command.add("panguweather"); command.add("--class od graphcast");
} }
String execLog = "执行任务命令，开始模拟，命令为："+command; String execLog = "执行任务命令，开始模拟，命令为："+command;
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),execLog)); ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),execLog));
ProcessBuilder processBuilder = new ProcessBuilder(command); ProcessBuilder processBuilder = new ProcessBuilder(command);
processBuilder.directory(new File(systemStorageProperties.getPanguModelExecPath())); processBuilder.directory(new File(systemStorageProperties.getGraphcastModelExecPath()));
processBuilder.redirectErrorStream(true); processBuilder.redirectErrorStream(true);
Process process = processBuilder.start(); Process process = processBuilder.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream(), "UTF-8")); BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream(), "UTF-8"));
@ -240,17 +230,17 @@ public class WeatherTaskExec extends Thread{
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog)); ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog));
LocalDateTime startTime = this.weatherTask.getStartDate().atTime(this.weatherTask.getStartTime(), 0, 0); LocalDateTime startTime = this.weatherTask.getStartDate().atTime(this.weatherTask.getStartTime(), 0, 0);
//grib_copy命令 //grib_copy命令
String gribCopyCommandPath = systemStorageProperties.getPanguEnvPath()+File.separator+"grib_copy"; String gribCopyCommandPath = systemStorageProperties.getGraphcastEnvPath()+File.separator+"grib_copy";
//grib_set命令 //grib_set命令
String gribSetCommandPath = systemStorageProperties.getPanguEnvPath()+File.separator+"grib_set"; String gribSetCommandPath = systemStorageProperties.getGraphcastEnvPath()+File.separator+"grib_set";
//公用gribProcessBuilder //公用gribProcessBuilder
ProcessBuilder gribProcessBuilder = new ProcessBuilder(); ProcessBuilder gribProcessBuilder = new ProcessBuilder();
gribProcessBuilder.directory(new File(systemStorageProperties.getPanguModelExecPath())); gribProcessBuilder.directory(new File(systemStorageProperties.getGraphcastModelExecPath()));
//把grib文件切割成每6小时一份 //把grib文件切割成每6小时一份
int step = 6; int step = 6;
int i=this.weatherTask.getStartTime(); int i=this.weatherTask.getStartTime();
while(i <= this.weatherTask.getLeadTime()){ while(i <= this.weatherTask.getLeadTime()){
String gribCopyFileAddr = gribCopyTargetDir+File.separator+"panguweather_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+gribFileSuffix+".grib"; String gribCopyFileAddr = gribCopyTargetDir+File.separator+"graphcast_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+gribFileSuffix+".grib";
//切割grib文件命令 //切割grib文件命令
List<String> gribCopyCommand = new ArrayList<>(); List<String> gribCopyCommand = new ArrayList<>();
gribCopyCommand.add(gribCopyCommandPath); gribCopyCommand.add(gribCopyCommandPath);
@ -264,7 +254,7 @@ public class WeatherTaskExec extends Thread{
gribCopyProcess.waitFor(); gribCopyProcess.waitFor();
//重新设置reftime信息 //重新设置reftime信息
String gribSetFileAddr = gribCopyTargetDir+File.separator+"panguweather_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+".grib"; String gribSetFileAddr = gribCopyTargetDir+File.separator+"graphcast_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+".grib";
String date = LocalDateTimeUtil.format(startTime,"yyyyMMdd"); String date = LocalDateTimeUtil.format(startTime,"yyyyMMdd");
String time = LocalDateTimeUtil.format(startTime,"HHmm"); String time = LocalDateTimeUtil.format(startTime,"HHmm");
List<String> gribSetCommand = new ArrayList<>(); List<String> gribSetCommand = new ArrayList<>();
@ -338,11 +328,11 @@ public class WeatherTaskExec extends Thread{
* @param inputPath * @param inputPath
* @param targetPath * @param targetPath
*/ */
private void saveGribInfoToDB(String inputPath,String targetPath){ private void saveGribInfoToDB(String inputPath,String targetPath,String gribFileSuffix){
String handleGribLog = "格式转换结束，处理grib文件数据入库"; String handleGribLog = "格式转换结束，处理grib文件数据入库";
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog)); ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog));
List<File> files = FileUtil.loopFiles(inputPath,file -> file.getName().startsWith("pangu_")); List<File> files = FileUtil.loopFiles(inputPath,file -> file.getName().startsWith("graphcast_") && !file.getName().contains(gribFileSuffix));
List<File> targetFiles = new ArrayList<>(); List<File> targetFiles = new ArrayList<>();
if(!files.isEmpty()){ if(!files.isEmpty()){
for(File file:files){ for(File file:files){

438
jeecg-module-weather/src/main/java/org/jeecg/task/PanGuWeatherTaskExec.java Normal file
View File

@ -0,0 +1,438 @@
package org.jeecg.task;
import cn.hutool.core.date.LocalDateTimeUtil;
import cn.hutool.core.io.FileUtil;
import cn.hutool.core.util.StrUtil;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.time.StopWatch;
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
import org.jeecg.common.constant.enums.WeatherForecastDatasourceEnum;
import org.jeecg.common.constant.enums.WeatherTaskStatusEnum;
import org.jeecg.common.exception.JeecgFileUploadException;
import org.jeecg.common.properties.SystemStorageProperties;
import org.jeecg.common.util.NcUtil;
import org.jeecg.modules.base.entity.WeatherData;
import org.jeecg.modules.base.entity.WeatherTask;
import org.jeecg.modules.base.mapper.WeatherDataMapper;
import org.jeecg.service.WeatherTaskService;
import java.io.*;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.TimeUnit;
/**
* 气象预测任务执行线程
*/
@Slf4j
public class PanGuWeatherTaskExec extends Thread{
private WeatherTask weatherTask;
private WeatherTaskService weatherTaskService;
private SystemStorageProperties systemStorageProperties;
private WeatherDataMapper weatherDataMapper;
/**
* 初始化
*/
public void init(WeatherTask weatherTask,
WeatherTaskService weatherTaskService,
SystemStorageProperties systemStorageProperties,
WeatherDataMapper weatherDataMapper){
this.weatherTask = weatherTask;
this.weatherTaskService = weatherTaskService;
this.systemStorageProperties = systemStorageProperties;
this.weatherDataMapper = weatherDataMapper;
}
@Override
public void run() {
this.execute();
}
/**
* 执行任务
*/
public void execute() {
StopWatch stopWatch = new StopWatch();
stopWatch.start();
try{
//修改任务状态为执行中
this.weatherTaskService.updateTaskStatus(this.weatherTask.getId(), WeatherTaskStatusEnum.IN_OPERATION.getKey());
//如果此任务已存在历史日志，先清除
this.weatherTaskService.deleteTaskLog(this.weatherTask.getId());
//执行模拟
this.execSimulation();
//执行完成
this.execComplete(stopWatch,WeatherTaskStatusEnum.COMPLETED.getKey(), "");
}catch (Exception e){
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
//执行失败
this.execComplete(stopWatch,WeatherTaskStatusEnum.FAILURE.getKey(), taskErrorLog);
throw e;
}
}
/**
* 任务执行完成
* @param stopWatch
* @param taskStatus
* @param taskErrorLog
*/
private void execComplete(StopWatch stopWatch,Integer taskStatus,String taskErrorLog){
//添加任务耗时
stopWatch.stop();
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
double min = seconds/60D;
BigDecimal bgMin = new BigDecimal(min);
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
this.weatherTaskService.updateTaskStatusToCompleted(this.weatherTask.getId(),result.doubleValue(),taskStatus);
if (WeatherTaskStatusEnum.COMPLETED.getKey().equals(taskStatus)){
String taskCompletedLog = "任务执行完成，耗时："+result+"分钟";
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),taskCompletedLog));
}else if (WeatherTaskStatusEnum.FAILURE.getKey().equals(taskStatus)){
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),taskErrorLog));
}
}
/**
* 执行模拟
*/
private void execSimulation(){
try {
//输入文件
String inputFileName = this.weatherTask.getInputFile();
//预测文件
String outputFileName = "panguweather_output_"+this.weatherTask.getId()+".grib";
//定义名称后缀,执行grib_set命令时去除，否则命名会冲突
String gribFileSuffix = "_not_grib_set";
//临时目录grib_copy,grib_set,python转换的输入都在这个目录里，最后删除
String inputPath = systemStorageProperties.getPanguModelExecPath()+File.separator+this.weatherTask.getId();
if(!FileUtil.exist(inputPath)){
FileUtil.mkdir(inputPath);
}
//python转换的输出在这里，最后删除
String outputPath = inputPath+File.separator+"formatResult";
if(!FileUtil.exist(outputPath)){
FileUtil.mkdir(outputPath);
}
//预测气象数据
this.forecast(inputFileName,outputFileName);
//切割文件，6小时一个
this.splitGrib(outputFileName,gribFileSuffix,inputPath);
//适配flexpart
this.convertGrib(gribFileSuffix,inputPath,outputPath);
//保存文件数据入库
this.saveGribInfoToDB(inputPath,this.getPanguWeatherPath());
//删除预测文件
File inputFile = new File(systemStorageProperties.getPanguModelExecPath()+File.separator+inputFileName);
if (inputFile.exists()){
inputFile.delete();
}
// 删除输出的总文件
File outputFile = new File(systemStorageProperties.getPanguModelExecPath()+File.separator+outputFileName);
if (outputFile.exists()){
outputFile.delete();
}
//适配flexpart结束，删除目录
if(FileUtil.exist(inputPath)){
FileUtil.del(inputPath);
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/**
* 盘古预测
* @param inputFileAddr
* @param outputFileAddr
*/
private void forecast(String inputFileAddr,String outputFileAddr){
try {
//处理开始日志
String startLogFormat = "";
String startTimeFormat = LocalDateTimeUtil.format(this.weatherTask.getStartDate(),"yyyy-MM-dd HH:mm:ss");
String forecastTime = this.weatherTask.getLeadTime().toString();
String forecastModel = WeatherDataSourceEnum.PANGU.getValue();
if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){
startLogFormat = "任务开始，本次使用预测模型为：%s，预测开始时间为：%s，预测时长为：%s，前置数据采用CDS在线数据";
}else {
startLogFormat = "任务开始，本次使用预测模型为：%s，预测开始时间为：%s，预测时长为：%s，前置数据采用离线数据："+this.weatherTask.getInputFile();
}
String startLog = String.format(startLogFormat,forecastModel,startTimeFormat,forecastTime);
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),startLog));
//处理运行命令
List<String> command = new ArrayList<>();
String aiModelsPath = systemStorageProperties.getPanguEnvPath()+File.separator+"ai-models";
if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){
command.add(aiModelsPath);
command.add("--input");
command.add("cds");
command.add("--date");
command.add(LocalDateTimeUtil.format(this.weatherTask.getStartDate(),"yyyyMMdd"));
command.add("--time");
command.add(this.weatherTask.getStartTime().toString());
command.add("--lead-time");
command.add(this.weatherTask.getLeadTime().toString());
command.add("--path");
command.add(outputFileAddr);
command.add("--assets");
command.add("assets-panguweather");
command.add("panguweather");
}else if (this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.LOCATION_FILE.getKey())){
//离线文件还需处理
command.add(aiModelsPath);
command.add("--file");
command.add(inputFileAddr);
command.add("--lead-time");
command.add(this.weatherTask.getLeadTime().toString());
command.add("--path");
command.add(outputFileAddr);
command.add("--assets");
command.add("assets-panguweather");
command.add("panguweather");
}
String execLog = "执行任务命令，开始模拟，命令为："+command;
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),execLog));
ProcessBuilder processBuilder = new ProcessBuilder(command);
processBuilder.directory(new File(systemStorageProperties.getPanguModelExecPath()));
processBuilder.redirectErrorStream(true);
Process process = processBuilder.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream(), "UTF-8"));
//读取输出日志
String line;
while ((line = reader.readLine()) != null) {
if(StrUtil.isNotBlank(line)){
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),line));
}
}
//等待进程结束
process.waitFor();
}catch (Exception e){
throw new RuntimeException("盘古模型预测异常",e);
}
}
/**
* 分割grib文件
* @param outputFileAddr
* @param gribFileSuffix
* @param gribCopyTargetDir
*/
private void splitGrib(String outputFileAddr,String gribFileSuffix,String gribCopyTargetDir){
try {
String handleGribLog = "预测结束，开始处理grib文件,切割为6小时一个文件";
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog));
LocalDateTime startTime = this.weatherTask.getStartDate().atTime(this.weatherTask.getStartTime(), 0, 0);
//grib_copy命令
String gribCopyCommandPath = systemStorageProperties.getPanguEnvPath()+File.separator+"grib_copy";
//grib_set命令
String gribSetCommandPath = systemStorageProperties.getPanguEnvPath()+File.separator+"grib_set";
//公用gribProcessBuilder
ProcessBuilder gribProcessBuilder = new ProcessBuilder();
gribProcessBuilder.directory(new File(systemStorageProperties.getPanguModelExecPath()));
//把grib文件切割成每6小时一份
int step = 6;
int i=this.weatherTask.getStartTime();
while(i <= this.weatherTask.getLeadTime()){
String gribCopyFileAddr = gribCopyTargetDir+File.separator+"panguweather_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+gribFileSuffix+".grib";
//切割grib文件命令
List<String> gribCopyCommand = new ArrayList<>();
gribCopyCommand.add(gribCopyCommandPath);
gribCopyCommand.add("-w");
gribCopyCommand.add("step="+i);
gribCopyCommand.add(outputFileAddr);
gribCopyCommand.add(gribCopyFileAddr);
gribProcessBuilder.command(gribCopyCommand);
log.info("gribCopyCommand:{}", gribCopyCommand);
Process gribCopyProcess = gribProcessBuilder.start();
gribCopyProcess.waitFor();
//重新设置reftime信息
String gribSetFileAddr = gribCopyTargetDir+File.separator+"panguweather_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+".grib";
String date = LocalDateTimeUtil.format(startTime,"yyyyMMdd");
String time = LocalDateTimeUtil.format(startTime,"HHmm");
List<String> gribSetCommand = new ArrayList<>();
gribSetCommand.add(gribSetCommandPath);
gribSetCommand.add("-s");
gribSetCommand.add("dataDate="+date+",dataTime="+time+",endStep="+0);
gribSetCommand.add(gribCopyFileAddr);
gribSetCommand.add(gribSetFileAddr);
gribProcessBuilder.command(gribSetCommand);
log.info("gribSetCommand:{}", gribSetCommand);
Process gribSetProcess = gribProcessBuilder.start();
gribSetProcess.waitFor();
i+=step;
startTime = startTime.plusHours(step);
}
}catch (Exception e){
throw new RuntimeException("分割grib文件异常",e);
}
}
/**
* 转换grib
* @param gribFileSuffix
* @param inputPath
* @param outputPath
*/
private void convertGrib(String gribFileSuffix,String inputPath,String outputPath){
try {
// python3 pangu_reformat.py --ini_dir /export/xe_bk_data/weather/pangu_format_script --output_dir /export/xe_bk_data/weather/pangu2 --input_dir /export/xe_bk_data/weather/pangu2 --start_date 20251120 --end_date 20251120
//调用python脚本转换flexpart能识别的气象数据文件
//删除带有_not_grib_set名称的文件，此文件还未设置reftime属性
List<File> files = FileUtil.loopFiles(inputPath, file -> file.getName().contains(gribFileSuffix));
if(!files.isEmpty()){
for(File file:files){
file.delete();
}
}
//构建转换命令
String convertScriptPath = systemStorageProperties.getPanguFormatScriptPath();
String startDate = this.weatherTask.getStartDate().format(DateTimeFormatter.ofPattern("yyyyMMdd"));
String endDate = this.weatherTask.getStartDate().atTime(this.weatherTask.getStartTime(), 0, 0).plusHours(this.weatherTask.getLeadTime()).format(DateTimeFormatter.ofPattern("yyyyMMdd"));
List<String> convertCommand = new ArrayList<>();
convertCommand.add(systemStorageProperties.getFormatScriptPythonEnv()+File.separator+"python3");
convertCommand.add("pangu_reformat.py");
convertCommand.add("--ini_dir");
convertCommand.add(convertScriptPath);
convertCommand.add("--output_dir");
convertCommand.add(outputPath);
convertCommand.add("--input_dir");
convertCommand.add(inputPath);
convertCommand.add("--start_date");
convertCommand.add(startDate);
convertCommand.add("--end_date");
convertCommand.add(endDate);
String adapterGribLog = "文件切割结束，开始调用python脚本适配flexpart模型，执行命令为："+convertCommand;
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),adapterGribLog));
ProcessBuilder adapterGribProcessBuilder = new ProcessBuilder();
adapterGribProcessBuilder.directory(new File(convertScriptPath));
adapterGribProcessBuilder.command(convertCommand);
Process adapterGribProcess = adapterGribProcessBuilder.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(adapterGribProcess.getInputStream(), "UTF-8"));
//读取输出日志
String line;
while ((line = reader.readLine()) != null) {
if(StrUtil.isNotBlank(line)){
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),line));
}
}
boolean finished = adapterGribProcess.waitFor((5*3600),TimeUnit.SECONDS);
if (!finished) {
// 如果在指定时间内进程没有结束，则强制销毁它
adapterGribProcess.destroyForcibly(); // 强制终止进程
String destroyForciblyLog = "python脚本适配flexpart模型进程超时，终止进程";
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),destroyForciblyLog));
} else {
int exitCode = adapterGribProcess.exitValue();
String progressFinishedLog = "python脚本适配flexpart模型进程结束，exitCode："+exitCode;
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),progressFinishedLog));
}
}catch (Exception e){
throw new RuntimeException("适配flexpart，转换grib异常",e);
}
}
/**
* 保存生成的气象数据存储到数据库
* @param inputPath
* @param targetPath
*/
private void saveGribInfoToDB(String inputPath,String targetPath){
String handleGribLog = "格式转换结束，处理grib文件数据入库";
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog));
List<File> files = FileUtil.loopFiles(inputPath,file -> file.getName().startsWith("pangu_"));
List<File> targetFiles = new ArrayList<>();
if(!files.isEmpty()){
for(File file:files){
File targetFile = new File(targetPath+File.separator+file.getName());
FileUtil.move(file,targetFile,true);
if(targetFile.exists()){
targetFiles.add(targetFile);
}
}
for(File targetFile:targetFiles){
//获取文件数据开始日期
String reftime = NcUtil.getReftime(targetFile.getAbsolutePath());
if(StringUtils.isBlank(reftime)) {
throw new JeecgFileUploadException("解析气象文件起始时间数据异常，此文件可能损坏");
}
//计算文件大小M
BigDecimal divideVal = new BigDecimal("1024");
BigDecimal bg = new BigDecimal(targetFile.length());
BigDecimal fileSize = bg.divide(divideVal).divide(divideVal).setScale(2, RoundingMode.HALF_UP);
//处理文件数据开始时间
Instant instant = Instant.parse(reftime);
LocalDateTime utcDateTime = LocalDateTime.ofInstant(instant, ZoneId.of("UTC"));
//计算文件MD5值
String md5Val = "";
try (InputStream is = new FileInputStream(targetFile.getAbsolutePath())) {
md5Val = DigestUtils.md5Hex(is);
}catch (Exception e){
throw new RuntimeException(targetFile.getName()+"MD5值计算失败");
}
//校验文件是否存在，存在删除从新新增
LambdaQueryWrapper<WeatherData> queryWrapper = new LambdaQueryWrapper<>();
queryWrapper.eq(WeatherData::getFileName,targetFile.getName());
WeatherData queryResult = weatherDataMapper.selectOne(queryWrapper);
if(Objects.nonNull(queryResult)){
weatherDataMapper.deleteById(queryResult.getId());
}
//构建文件信息
WeatherData weatherData = new WeatherData();
weatherData.setFileName(targetFile.getName());
weatherData.setFileSize(fileSize.doubleValue());
weatherData.setFileExt(targetFile.getName().substring(targetFile.getName().lastIndexOf(".")+1));
weatherData.setDataStartTime(utcDateTime);
weatherData.setDataSource(weatherTask.getPredictionModel());
weatherData.setFilePath(targetFile.getAbsolutePath());
weatherData.setMd5Value(md5Val);
weatherData.setShareTotal(1);
weatherDataMapper.insert(weatherData);
}
}
}
/**
* 获取盘古数据存储路径
* @return
*/
private String getPanguWeatherPath(){
StringBuilder sb = new StringBuilder();
sb.append(this.systemStorageProperties.getRootPath());
sb.append(File.separator);
sb.append(this.systemStorageProperties.getPangu());
return sb.toString();
}
/**
* 获取Graphcast数据存储路径
* @return
*/
private String getGraphcastWeatherPath(){
StringBuilder sb = new StringBuilder();
sb.append(this.systemStorageProperties.getRootPath());
sb.append(File.separator);
sb.append(this.systemStorageProperties.getGraphcast());
return sb.toString();
}
}