fix:
1.按客户要求修改正演和反演逻辑 2.修改系统调用flexpart逻辑弃用docker，通过ssh调用宿主机安装的flexpart 3.通过docker-compose配置环境变量，系统自动读取flexpart所在主机
This commit is contained in:
panbaolin
parent
2fae10797a
commit
2a8c763b58
162
README.md
162
README.md
|
|
@ -1,164 +1,6 @@
|
|||
|
||||
JeecgBoot 低代码开发平台
|
||||
===============
|
||||
|
||||
当前最新版本： 3.8.0（发布日期：2025-05-16）
|
||||
|
||||
|
||||
[](https://github.com/zhangdaiscott/jeecg-boot/blob/master/LICENSE)
|
||||
[](http://jeecg.com/aboutusIndex)
|
||||
[](https://github.com/zhangdaiscott/jeecg-boot)
|
||||
[](https://github.com/zhangdaiscott/jeecg-boot)
|
||||
[](https://github.com/zhangdaiscott/jeecg-boot)
|
||||
|
||||
|
||||
|
||||
项目介绍
|
||||
-----------------------------------
|
||||
|
||||
<h3 align="center">Java Low Code Platform for Enterprise web applications</h3>
|
||||
|
||||
JeecgBoot 是一款基于代码生成器的`低代码开发平台`！前后端分离架构 SpringBoot2.x和3.x，SpringCloud，Ant Design Vue3，Mybatis-plus，Shiro，JWT，支持微服务。强大的代码生成器让前后端代码一键生成，实现低代码开发! JeecgBoot 引领新的低代码开发模式(OnlineCoding-> 代码生成器-> 手工MERGE)， 帮助解决Java项目70%的重复工作，让开发更多关注业务。既能快速提高效率，节省研发成本，同时又不失灵活性！
|
||||
|
||||
|
||||
#### 项目说明
|
||||
|
||||
| 项目名 | 说明 |
|
||||
|--------------------|------------------------|
|
||||
| `jeecg-boot` | 后端源码JAVA（SpringBoot微服务架构） |
|
||||
| `jeecgboot-vue3` | 前端源码VUE3（vue3+vite5+ts最新技术栈） |
|
||||
|
||||
|
||||
|
||||
技术文档
|
||||
-----------------------------------
|
||||
|
||||
- 官方网站： [http://www.jeecg.com](http://www.jeecg.com)
|
||||
- 新手指南： [快速入门](http://www.jeecg.com/doc/quickstart)
|
||||
- QQ交流群 ： ⑩716488839、⑨808791225、其他(满)
|
||||
- 在线演示 ： [在线演示](http://boot3.jeecg.com) | [APP演示](http://jeecg.com/appIndex)
|
||||
> 演示系统的登录账号密码，请点击 [获取账号密码](http://jeecg.com/doc/demo) 获取
|
||||
|
||||
|
||||
|
||||
启动项目
|
||||
-----------------------------------
|
||||
|
||||
- [IDEA启动前后端项目](https://help.jeecg.com/java/setup/idea/startup)
|
||||
- [Docker一键启动前后端](https://help.jeecg.com/java/docker/quick)
|
||||
|
||||
|
||||
微服务启动
|
||||
-----------------------------------
|
||||
- [单体快速切换微服务](https://help.jeecg.com/java/springcloud/switchcloud/monomer)
|
||||
- [Docker启动微服务后台](https://help.jeecg.com/java/docker/springcloud)
|
||||
|
||||
|
||||
|
||||
技术架构：
|
||||
-----------------------------------
|
||||
|
||||
#### 后端
|
||||
|
||||
- IDE建议： IDEA (必须安装lombok插件 )
|
||||
- 语言：Java 8+ (支持17)
|
||||
- 依赖管理：Maven
|
||||
- 基础框架：Spring Boot 2.7.18
|
||||
- 微服务框架： Spring Cloud Alibaba 2021.0.1.0
|
||||
- 持久层框架：MybatisPlus 3.5.3.2
|
||||
- 报表工具： JimuReport 1.9.4
|
||||
- 安全框架：Apache Shiro 1.12.0，Jwt 3.11.0
|
||||
- 微服务技术栈：Spring Cloud Alibaba、Nacos、Gateway、Sentinel、Skywalking
|
||||
- 数据库连接池：阿里巴巴Druid 1.1.24
|
||||
- 日志打印：logback
|
||||
- 缓存：Redis
|
||||
- 其他：autopoi, fastjson，poi，Swagger-ui，quartz, lombok（简化代码）等。
|
||||
- 默认数据库脚本：MySQL5.7+
|
||||
- [其他数据库，需要自己转](https://my.oschina.net/jeecg/blog/4905722)
|
||||
|
||||
|
||||
#### 前端
|
||||
|
||||
- 前端IDE建议：WebStorm、Vscode
|
||||
- 采用 Vue3.0+TypeScript+Vite+Ant-Design-Vue等新技术方案，包括二次封装组件、utils、hooks、动态菜单、权限校验、按钮级别权限控制等功能
|
||||
- 最新技术栈：Vue3.0 + TypeScript + Vite5 + ant-design-vue4 + pinia + echarts + unocss + vxe-table + qiankun + es6
|
||||
- 依赖管理：node、npm、pnpm
|
||||
|
||||
|
||||
|
||||
#### 支持库
|
||||
|
||||
| 数据库 | 支持 |
|
||||
| --- | --- |
|
||||
| MySQL | √ |
|
||||
| Oracle11g | √ |
|
||||
| Sqlserver2017 | √ |
|
||||
| PostgreSQL | √ |
|
||||
| MariaDB | √ |
|
||||
| 达梦 | √ |
|
||||
| 人大金仓 | √ |
|
||||
| TiDB | √ |
|
||||
|
||||
|
||||
|
||||
|
||||
## 微服务解决方案
|
||||
|
||||
|
||||
- 1、服务注册和发现 Nacos √
|
||||
- 2、统一配置中心 Nacos √
|
||||
- 3、路由网关 gateway(三种加载方式) √
|
||||
- 4、分布式 http feign √
|
||||
- 5、熔断降级限流 Sentinel √
|
||||
- 6、分布式文件 Minio、阿里OSS √
|
||||
- 7、统一权限控制 JWT + Shiro √
|
||||
- 8、服务监控 SpringBootAdmin√
|
||||
- 9、链路跟踪 Skywalking [参考文档](https://help.jeecg.com/java/springcloud/super/skywarking)
|
||||
- 10、消息中间件 RabbitMQ √
|
||||
- 11、分布式任务 xxl-job √
|
||||
- 12、分布式事务 Seata
|
||||
- 13、轻量分布式日志 Loki+grafana套件
|
||||
- 14、支持 docker-compose、k8s、jenkins
|
||||
- 15、CAS 单点登录 √
|
||||
- 16、路由限流 √
|
||||
|
||||
|
||||
|
||||
后台目录结构
|
||||
-----------------------------------
|
||||
```
|
||||
项目结构
|
||||
├─jeecg-boot-parent（父POM： 项目依赖、modules组织）
|
||||
│ ├─jeecg-boot-base-core（共通模块： 工具类、config、权限、查询过滤器、注解等）
|
||||
│ ├─jeecg-module-demo 示例代码
|
||||
│ ├─jeecg-module-system System系统管理目录
|
||||
│ │ ├─jeecg-system-biz System系统管理权限等功能
|
||||
│ │ ├─jeecg-system-start System单体启动项目(8080）
|
||||
│ │ ├─jeecg-system-api System系统管理模块对外api
|
||||
│ │ │ ├─jeecg-system-cloud-api System模块对外提供的微服务接口
|
||||
│ │ │ ├─jeecg-system-local-api System模块对外提供的单体接口
|
||||
│ ├─jeecg-server-cloud --微服务模块
|
||||
├─jeecg-cloud-gateway --微服务网关模块(9999)
|
||||
├─jeecg-cloud-nacos --Nacos服务模块(8848)
|
||||
├─jeecg-system-cloud-start --System微服务启动项目(7001)
|
||||
├─jeecg-demo-cloud-start --Demo微服务启动项目(7002)
|
||||
├─jeecg-visual
|
||||
├─jeecg-cloud-monitor --微服务监控模块 (9111)
|
||||
├─jeecg-cloud-xxljob --微服务xxljob定时任务服务端 (9080)
|
||||
├─jeecg-cloud-sentinel --sentinel服务端 (9000)
|
||||
├─jeecg-cloud-test -- 微服务测试示例（各种例子）
|
||||
├─jeecg-cloud-test-more -- 微服务测试示例（feign、熔断降级、xxljob、分布式锁）
|
||||
├─jeecg-cloud-test-rabbitmq -- 微服务测试示例（rabbitmq）
|
||||
├─jeecg-cloud-test-seata -- 微服务测试示例（seata分布式事务）
|
||||
├─jeecg-cloud-test-shardingsphere -- 微服务测试示例（分库分表）
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
#### 微服务架构图
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,23 @@
|
|||
package org.jeecg.common.constant.enums;
|
||||
|
||||
/**
|
||||
* 物种类型
|
||||
*/
|
||||
public enum FlexpartSpeciesType {
|
||||
|
||||
XE_131m(51),
|
||||
XE_133(52),
|
||||
XE_133m(53),
|
||||
XE_135(54),
|
||||
NOT_SPECIES(61);
|
||||
|
||||
private Integer value;
|
||||
|
||||
FlexpartSpeciesType(int value) {
|
||||
this.value = value;
|
||||
}
|
||||
|
||||
public Integer getValue() {
|
||||
return value;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
package org.jeecg.common.constant.enums;
|
||||
|
||||
public enum TransportReleaseDataSource {
|
||||
|
||||
/**
|
||||
* 自动填写
|
||||
*/
|
||||
AUTO_SELECT(1),
|
||||
/**
|
||||
* 手动输入
|
||||
*/
|
||||
MANUAL_ENTRY(2);
|
||||
|
||||
private Integer key;
|
||||
|
||||
TransportReleaseDataSource(Integer key) {
|
||||
this.key = key;
|
||||
}
|
||||
|
||||
public Integer getKey(){
|
||||
return this.key;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
package org.jeecg.common.constant.enums;
|
||||
|
||||
/**
|
||||
* 输运任务样品类型说明枚举
|
||||
*/
|
||||
public enum TransportSampleTypeEnum {
|
||||
|
||||
/**
|
||||
* 正向
|
||||
*/
|
||||
B,
|
||||
/**
|
||||
* 反向
|
||||
*/
|
||||
P;
|
||||
}
|
||||
|
|
@ -20,7 +20,11 @@ public class DataFusionProperties {
|
|||
/**
|
||||
* srs文件的上级目录有可能是flexpart.x.ecmwf.l1或flexpart.x.ncep.l1
|
||||
*/
|
||||
private String srmParentDir;
|
||||
private String idcSrmParentDir;
|
||||
/**
|
||||
* srs文件的上级目录有可能是flexpart.x.ecmwf.l1或flexpart.x.ncep.l1
|
||||
*/
|
||||
private String ndcSrmParentDir;
|
||||
/**
|
||||
* 浓度值过滤条件
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -0,0 +1,19 @@
|
|||
package org.jeecg.common.properties;
|
||||
|
||||
import lombok.Data;
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Data
|
||||
@Component
|
||||
@ConfigurationProperties(prefix = "server-prop")
|
||||
public class ServerProperties {
|
||||
|
||||
private String host;
|
||||
|
||||
private int port;
|
||||
|
||||
private String username;
|
||||
|
||||
private String password;
|
||||
}
|
||||
|
|
@ -60,10 +60,15 @@ public class SystemStorageProperties {
|
|||
private String formatScriptPythonEnv;
|
||||
|
||||
/**
|
||||
* ai-models 安装地址
|
||||
* pangu的ai-models 安装地址
|
||||
*/
|
||||
private String panguEnvPath;
|
||||
|
||||
/**
|
||||
* graphcast的ai-models 安装地址
|
||||
*/
|
||||
private String graphcastEnvPath;
|
||||
|
||||
/**
|
||||
* 盘古模型执行路径
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -9,11 +9,26 @@ import org.springframework.stereotype.Component;
|
|||
@ConfigurationProperties(prefix = "transport-simulation")
|
||||
public class TransportSimulationProperties {
|
||||
|
||||
/**
|
||||
* 颗粒物样品大气输运级别
|
||||
*/
|
||||
private Integer transportLevel_p;
|
||||
|
||||
/**
|
||||
* 惰性气体样品输运状态
|
||||
*/
|
||||
private Integer transportLevel_x;
|
||||
|
||||
/**
|
||||
* 模型输出路径
|
||||
*/
|
||||
private String outputPath;
|
||||
|
||||
/**
|
||||
* 正演各站点排放数据存储目录
|
||||
*/
|
||||
private String inputSiteHourPath;
|
||||
|
||||
/**
|
||||
* 参数配置文件路径（和fnl气象数据有关系）
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -5,7 +5,6 @@ import com.baomidou.mybatisplus.annotation.TableField;
|
|||
import com.baomidou.mybatisplus.annotation.TableId;
|
||||
import com.baomidou.mybatisplus.annotation.TableName;
|
||||
import com.fasterxml.jackson.annotation.JsonFormat;
|
||||
import jakarta.validation.Valid;
|
||||
import jakarta.validation.constraints.NotBlank;
|
||||
import jakarta.validation.constraints.NotNull;
|
||||
import jakarta.validation.constraints.Null;
|
||||
|
|
@ -13,6 +12,7 @@ import lombok.Data;
|
|||
import org.jeecg.common.validgroup.InsertGroup;
|
||||
import org.jeecg.common.validgroup.UpdateGroup;
|
||||
import org.springframework.format.annotation.DateTimeFormat;
|
||||
import java.time.LocalDateTime;
|
||||
import java.util.Date;
|
||||
import java.util.List;
|
||||
|
||||
|
|
@ -90,7 +90,7 @@ public class TransportTask{
|
|||
* 创建时间
|
||||
*/
|
||||
@TableField(value = "create_time")
|
||||
@JsonFormat(timezone = "GMT+8", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private Date createTime;
|
||||
|
||||
|
|
@ -104,7 +104,7 @@ public class TransportTask{
|
|||
* 更新时间
|
||||
*/
|
||||
@TableField(value = "update_time")
|
||||
@JsonFormat(timezone = "GMT+8", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private Date updateTime;
|
||||
|
||||
|
|
@ -112,15 +112,19 @@ public class TransportTask{
|
|||
* 模拟开始时间
|
||||
*/
|
||||
@NotNull(message = "模拟开始时间不能为空",groups = {InsertGroup.class, UpdateGroup.class})
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@TableField(value = "start_time")
|
||||
private Date startTime;
|
||||
private LocalDateTime startTime;
|
||||
|
||||
/**
|
||||
* 模拟结束时间
|
||||
*/
|
||||
@NotNull(message = "模拟结束时间不能为空",groups = {InsertGroup.class, UpdateGroup.class})
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@TableField(value = "end_time")
|
||||
private Date endTime;
|
||||
private LocalDateTime endTime;
|
||||
|
||||
/**
|
||||
* 释放下部高度
|
||||
|
|
@ -143,7 +147,27 @@ public class TransportTask{
|
|||
@TableField(value = "particle_count")
|
||||
private Integer particleCount;
|
||||
|
||||
@Valid
|
||||
/**
|
||||
* 正演释放数据来源 1自动填写,2手动输入
|
||||
*/
|
||||
@TableField(value = "release_data_source")
|
||||
private Integer releaseDataSource;
|
||||
|
||||
/**
|
||||
* 反演子表信息
|
||||
*/
|
||||
@TableField(exist = false)
|
||||
List<TransportTaskChild> childList;
|
||||
private List<TransportTaskBackwardChild> backwardChild;
|
||||
|
||||
/**
|
||||
* 正演子表信息
|
||||
*/
|
||||
@TableField(exist = false)
|
||||
private List<TransportTaskForwardChild> forwardChild;
|
||||
|
||||
/**
|
||||
* 正演物种信息
|
||||
*/
|
||||
@TableField(exist = false)
|
||||
private List<TransportTaskForwardSpecies> species;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -5,75 +5,87 @@ import com.baomidou.mybatisplus.annotation.TableField;
|
|||
import com.baomidou.mybatisplus.annotation.TableId;
|
||||
import com.baomidou.mybatisplus.annotation.TableName;
|
||||
import com.fasterxml.jackson.annotation.JsonFormat;
|
||||
import jakarta.validation.constraints.NotBlank;
|
||||
import jakarta.validation.constraints.NotNull;
|
||||
import jakarta.validation.constraints.Null;
|
||||
import lombok.Data;
|
||||
import org.jeecg.common.validgroup.InsertGroup;
|
||||
import org.jeecg.common.validgroup.UpdateGroup;
|
||||
import org.springframework.format.annotation.DateTimeFormat;
|
||||
import java.time.LocalDateTime;
|
||||
import java.util.Date;
|
||||
|
||||
/**
|
||||
* 输运模拟任务子表
|
||||
* 输运模拟反演任务配置子表
|
||||
*/
|
||||
@Data
|
||||
@TableName("stas_transport_task_child")
|
||||
public class TransportTaskChild{
|
||||
@TableName("stas_transport_task_backward_child")
|
||||
public class TransportTaskBackwardChild {
|
||||
|
||||
/**
|
||||
* ID
|
||||
*/
|
||||
@Null(message = "ID必须为空",groups = { InsertGroup.class})
|
||||
@TableId(type = IdType.AUTO)
|
||||
private Integer id;
|
||||
|
||||
/**
|
||||
* 任务ID
|
||||
*/
|
||||
@Null(message = "任务id必须为空",groups = {InsertGroup.class, UpdateGroup.class})
|
||||
@TableField(value = "task_id")
|
||||
private Integer taskId;
|
||||
|
||||
/**
|
||||
* 台站编目
|
||||
*/
|
||||
@NotBlank(message = "台站编码不能为空",groups = {InsertGroup.class, UpdateGroup.class})
|
||||
@TableField(value = "station_code")
|
||||
private String stationCode;
|
||||
|
||||
/**
|
||||
* 经度1
|
||||
* 经度
|
||||
*/
|
||||
@NotNull(message = "经度不能为空",groups = {InsertGroup.class, UpdateGroup.class})
|
||||
@TableField(value = "lon")
|
||||
private Double lon;
|
||||
|
||||
/**
|
||||
* 纬度1
|
||||
* 纬度
|
||||
*/
|
||||
@NotNull(message = "经度不能为空",groups = {InsertGroup.class, UpdateGroup.class})
|
||||
@TableField(value = "lat")
|
||||
private Double lat;
|
||||
|
||||
/**
|
||||
* 释放量
|
||||
*/
|
||||
@NotNull(message = "释放量不能为空",groups = {InsertGroup.class, UpdateGroup.class})
|
||||
@TableField(value = "release_amount")
|
||||
private String releaseAmount;
|
||||
|
||||
/**
|
||||
* 创建人
|
||||
*/
|
||||
@TableField(value = "create_by")
|
||||
private String createBy;
|
||||
|
||||
/**
|
||||
* 创建时间
|
||||
*/
|
||||
@TableField(value = "create_time")
|
||||
@JsonFormat(timezone = "GMT+8", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private Date createTime;
|
||||
|
||||
/**
|
||||
* 样品id
|
||||
*/
|
||||
@TableField(value = "sample_id")
|
||||
private Integer sampleId;
|
||||
|
||||
/**
|
||||
* 样品类型
|
||||
*/
|
||||
@TableField(value = "sample_type")
|
||||
private String sampleType;
|
||||
|
||||
/**
|
||||
* 开始测量时间
|
||||
*/
|
||||
@TableField(value = "acq_start_time")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private LocalDateTime acqStartTime;
|
||||
|
||||
/**
|
||||
* 结束测量时间
|
||||
*/
|
||||
@TableField(value = "acq_end_time")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private LocalDateTime acqEndTime;
|
||||
}
|
||||
|
|
@ -0,0 +1,64 @@
|
|||
package org.jeecg.modules.base.entity;
|
||||
|
||||
import com.baomidou.mybatisplus.annotation.IdType;
|
||||
import com.baomidou.mybatisplus.annotation.TableField;
|
||||
import com.baomidou.mybatisplus.annotation.TableId;
|
||||
import com.baomidou.mybatisplus.annotation.TableName;
|
||||
import com.fasterxml.jackson.annotation.JsonFormat;
|
||||
import lombok.Data;
|
||||
import org.springframework.format.annotation.DateTimeFormat;
|
||||
import java.util.Date;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* 输运模拟正演任务配置子表
|
||||
*/
|
||||
@Data
|
||||
@TableName("stas_transport_task_forward_child")
|
||||
public class TransportTaskForwardChild {
|
||||
|
||||
/**
|
||||
* ID
|
||||
*/
|
||||
@TableId(type = IdType.AUTO)
|
||||
private Integer id;
|
||||
|
||||
/**
|
||||
* 任务ID
|
||||
*/
|
||||
@TableField(value = "task_id")
|
||||
private Integer taskId;
|
||||
|
||||
/**
|
||||
* 台站编目
|
||||
*/
|
||||
@TableField(value = "station_code")
|
||||
private String stationCode;
|
||||
|
||||
/**
|
||||
* 经度
|
||||
*/
|
||||
@TableField(value = "lon")
|
||||
private Double lon;
|
||||
|
||||
/**
|
||||
* 纬度
|
||||
*/
|
||||
@TableField(value = "lat")
|
||||
private Double lat;
|
||||
|
||||
/**
|
||||
* 创建时间
|
||||
*/
|
||||
@TableField(value = "create_time")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private Date createTime;
|
||||
|
||||
/**
|
||||
* 正演子表信息
|
||||
*/
|
||||
@TableField(exist = false)
|
||||
private List<TransportTaskForwardRelease> forwardReleaseChild;
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
package org.jeecg.modules.base.entity;
|
||||
|
||||
import com.baomidou.mybatisplus.annotation.IdType;
|
||||
import com.baomidou.mybatisplus.annotation.TableField;
|
||||
import com.baomidou.mybatisplus.annotation.TableId;
|
||||
import com.baomidou.mybatisplus.annotation.TableName;
|
||||
import com.fasterxml.jackson.annotation.JsonFormat;
|
||||
import lombok.Data;
|
||||
import org.springframework.format.annotation.DateTimeFormat;
|
||||
import java.time.LocalDateTime;
|
||||
import java.util.Date;
|
||||
|
||||
/**
|
||||
* 输运模拟正演任务配置子表
|
||||
*/
|
||||
@Data
|
||||
@TableName("stas_transport_task_forward_release")
|
||||
public class TransportTaskForwardRelease {
|
||||
|
||||
/**
|
||||
* ID
|
||||
*/
|
||||
@TableId(type = IdType.AUTO)
|
||||
private Integer id;
|
||||
|
||||
/**
|
||||
* 任务ID
|
||||
*/
|
||||
@TableField(value = "task_id")
|
||||
private Integer taskId;
|
||||
|
||||
/**
|
||||
* 任务ID
|
||||
*/
|
||||
@TableField(value = "forward_child_id")
|
||||
private Integer forwardChildId;
|
||||
|
||||
/**
|
||||
* 开始时间
|
||||
*/
|
||||
@TableField(value = "start_time")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private LocalDateTime startTime;
|
||||
|
||||
/**
|
||||
* 结束时间
|
||||
*/
|
||||
@TableField(value = "end_time")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private LocalDateTime endTime;
|
||||
|
||||
/**
|
||||
* 释放量
|
||||
*/
|
||||
@TableField(value = "release_amount")
|
||||
private String releaseAmount;
|
||||
|
||||
/**
|
||||
* 创建时间
|
||||
*/
|
||||
@TableField(value = "create_time")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private Date createTime;
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
package org.jeecg.modules.base.entity;
|
||||
|
||||
import com.baomidou.mybatisplus.annotation.TableField;
|
||||
import com.baomidou.mybatisplus.annotation.TableName;
|
||||
import com.fasterxml.jackson.annotation.JsonFormat;
|
||||
import lombok.Data;
|
||||
import org.springframework.format.annotation.DateTimeFormat;
|
||||
import java.util.Date;
|
||||
|
||||
/**
|
||||
* 输运模拟正演任务配置子表
|
||||
*/
|
||||
@Data
|
||||
@TableName("stas_transport_task_forward_species")
|
||||
public class TransportTaskForwardSpecies {
|
||||
|
||||
/**
|
||||
* 任务ID
|
||||
*/
|
||||
@TableField(value = "task_id")
|
||||
private Integer taskId;
|
||||
|
||||
/**
|
||||
* 物种id
|
||||
*/
|
||||
@TableField(value = "species_id")
|
||||
private String speciesId;
|
||||
|
||||
/**
|
||||
* 创建时间
|
||||
*/
|
||||
@TableField(value = "create_time")
|
||||
@JsonFormat(timezone = "Asia/Shanghai", pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
@DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
|
||||
private Date createTime;
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
package org.jeecg.modules.base.mapper;
|
||||
|
||||
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
|
||||
import org.jeecg.modules.base.entity.TransportTaskBackwardChild;
|
||||
|
||||
public interface TransportTaskBackwardChildMapper extends BaseMapper<TransportTaskBackwardChild> {
|
||||
}
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
package org.jeecg.modules.base.mapper;
|
||||
|
||||
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
|
||||
import org.jeecg.modules.base.entity.TransportTaskChild;
|
||||
|
||||
public interface TransportTaskChildMapper extends BaseMapper<TransportTaskChild> {
|
||||
}
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
package org.jeecg.modules.base.mapper;
|
||||
|
||||
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
|
||||
import org.jeecg.modules.base.entity.TransportTaskForwardChild;
|
||||
|
||||
public interface TransportTaskForwardChildMapper extends BaseMapper<TransportTaskForwardChild> {
|
||||
}
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
package org.jeecg.modules.base.mapper;
|
||||
|
||||
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
|
||||
import org.jeecg.modules.base.entity.TransportTaskForwardRelease;
|
||||
|
||||
public interface TransportTaskForwardReleaseMapper extends BaseMapper<TransportTaskForwardRelease> {
|
||||
}
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
package org.jeecg.modules.base.mapper;
|
||||
|
||||
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
|
||||
import org.jeecg.modules.base.entity.TransportTaskForwardSpecies;
|
||||
|
||||
public interface TransportTaskForwardSpeciesMapper extends BaseMapper<TransportTaskForwardSpecies> {
|
||||
|
||||
}
|
||||
|
|
@ -1,6 +1,7 @@
|
|||
package org.jeecg.sample.mapper;
|
||||
|
||||
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
|
||||
import org.apache.ibatis.annotations.Param;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
|
@ -23,5 +24,6 @@ public interface IMSSampleAnalysesMapper extends BaseMapper {
|
|||
* 查询待输运的样品列表
|
||||
* @return
|
||||
*/
|
||||
List<Map<String, Object>> getSamplesTransportList();
|
||||
List<Map<String, Object>> getSamplesTransportList(@Param("transportLevel_p") Integer transportLevel_p,
|
||||
@Param("transportLevel_x") Integer transportLevel_x);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -40,9 +40,9 @@
|
|||
INNER JOIN CONFIGURATION.GARDS_STATIONS gst on gsd.STATION_ID = gst.STATION_ID
|
||||
WHERE CLOSE_STATUS = 0
|
||||
and (
|
||||
(gsd.SAMPLE_TYPE = 'P' AND ga.CATEGORY IN (4, 5))
|
||||
(gsd.SAMPLE_TYPE = 'P' AND ga.CATEGORY > #{transportLevel_p})
|
||||
or
|
||||
(gsd.SAMPLE_TYPE = 'B' AND ga.CATEGORY IN (3))
|
||||
(gsd.SAMPLE_TYPE = 'B' AND ga.CATEGORY > #{transportLevel_x})
|
||||
)
|
||||
ORDER BY gts.MODDATE desc
|
||||
</select>
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ package org.jeecg.sample.service.impl;
|
|||
import com.baomidou.dynamic.datasource.annotation.DS;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import org.jeecg.common.constant.enums.TransportTaskCloseStatusEnum;
|
||||
import org.jeecg.common.properties.TransportSimulationProperties;
|
||||
import org.jeecg.modules.base.entity.rnauto.GardsTransportStatus;
|
||||
import org.jeecg.modules.base.mapper.GardsTransportStatusMapper;
|
||||
import org.jeecg.sample.mapper.IMSSampleAnalysesMapper;
|
||||
|
|
@ -18,6 +19,7 @@ public class IMSSampleAnalysesServiceImpl implements IMSSampleAnalysesService {
|
|||
|
||||
private final IMSSampleAnalysesMapper sampleAnalysesMapper;
|
||||
private final GardsTransportStatusMapper transportStatusMapper;
|
||||
private final TransportSimulationProperties transportSimulationProperties;
|
||||
|
||||
/**
|
||||
* 统计气溶胶样品数量
|
||||
|
|
@ -46,7 +48,7 @@ public class IMSSampleAnalysesServiceImpl implements IMSSampleAnalysesService {
|
|||
@DS("ora")
|
||||
@Override
|
||||
public List<Map<String, Object>> getSamplesTransportList() {
|
||||
return sampleAnalysesMapper.getSamplesTransportList();
|
||||
return sampleAnalysesMapper.getSamplesTransportList(transportSimulationProperties.getTransportLevel_p(),transportSimulationProperties.getTransportLevel_x());
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -20,8 +20,8 @@ public class TransportResultDataController {
|
|||
|
||||
private final TransportResultDataService transportResultDataService;
|
||||
|
||||
@AutoLog(value = "分页查询输运任务数据")
|
||||
@Operation(summary = "分页查询输运任务数据")
|
||||
@AutoLog(value = "获取扩散数据")
|
||||
@Operation(summary = "获取扩散数据")
|
||||
@GetMapping("getDiffusionData")
|
||||
public Result<?> getDiffusionData(@Validated(value= QueryGroup.class) QueryDiffusionVO queryDiffusionVO) {
|
||||
return Result.OK(transportResultDataService.getDiffusionData(queryDiffusionVO));
|
||||
|
|
|
|||
|
|
@ -2,28 +2,25 @@ package org.jeecg.service.impl;
|
|||
|
||||
import cn.hutool.core.collection.CollUtil;
|
||||
import cn.hutool.core.date.DateUtil;
|
||||
import cn.hutool.core.date.LocalDateTimeUtil;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import cn.hutool.core.util.StrUtil;
|
||||
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.jeecg.common.constant.CommonConstant;
|
||||
import org.jeecg.common.constant.enums.TransportTaskModeEnum;
|
||||
import org.jeecg.common.constant.enums.TransportTimingAnalysisEnum;
|
||||
import org.jeecg.common.properties.TransportSimulationProperties;
|
||||
import org.jeecg.common.util.NcUtil;
|
||||
import org.jeecg.common.util.RedisUtil;
|
||||
import org.jeecg.modules.base.entity.TransportTask;
|
||||
import org.jeecg.modules.base.entity.TransportTaskChild;
|
||||
import org.jeecg.modules.base.entity.configuration.GardsNuclearReactors;
|
||||
import org.jeecg.modules.base.entity.TransportTaskBackwardChild;
|
||||
import org.jeecg.modules.base.entity.TransportTaskForwardChild;
|
||||
import org.jeecg.modules.base.entity.configuration.GardsStations;
|
||||
import org.jeecg.modules.base.mapper.TransportTaskChildMapper;
|
||||
import org.jeecg.modules.base.mapper.TransportTaskBackwardChildMapper;
|
||||
import org.jeecg.modules.base.mapper.TransportTaskForwardChildMapper;
|
||||
import org.jeecg.modules.base.mapper.TransportTaskMapper;
|
||||
import org.jeecg.service.StationDataService;
|
||||
import org.jeecg.service.TransportResultDataService;
|
||||
import org.jeecg.util.BilinearInterpolatorWithMath;
|
||||
import org.jeecg.vo.ConcModValVo;
|
||||
import org.jeecg.vo.ContributionAnalysisVO;
|
||||
import org.jeecg.vo.QueryDiffusionVO;
|
||||
import org.jeecg.vo.TaskStationsVO;
|
||||
|
|
@ -33,15 +30,10 @@ import ucar.ma2.DataType;
|
|||
import ucar.ma2.Index;
|
||||
import ucar.ma2.InvalidRangeException;
|
||||
import ucar.nc2.Attribute;
|
||||
import ucar.nc2.Dimension;
|
||||
import ucar.nc2.NetcdfFile;
|
||||
import ucar.nc2.Variable;
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.math.BigDecimal;
|
||||
import java.time.Instant;
|
||||
import java.time.LocalDateTime;
|
||||
import java.time.ZoneId;
|
||||
import java.util.*;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
|
|
@ -52,7 +44,8 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
|
||||
private final TransportTaskMapper transportTaskMapper;
|
||||
private final TransportSimulationProperties simulationProperties;
|
||||
private final TransportTaskChildMapper transportTaskChildMapper;
|
||||
private final TransportTaskBackwardChildMapper backwardChildMapper;
|
||||
private final TransportTaskForwardChildMapper forwardChildMapper;
|
||||
private final StationDataService stationDataService;
|
||||
private final RedisUtil redisUtil;
|
||||
private final static String FORWARD="forward";
|
||||
|
|
@ -123,18 +116,18 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
*/
|
||||
private List<Map<String,Object>> getBackwardData(TransportTask transportTask,QueryDiffusionVO queryDiffusionVO){
|
||||
//验证任务子信息
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,queryDiffusionVO.getTaskId());
|
||||
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskChild::getId);
|
||||
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
|
||||
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,queryDiffusionVO.getTaskId());
|
||||
queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
|
||||
List<TransportTaskBackwardChild> transportTaskChildren = backwardChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isEmpty(transportTaskChildren)){
|
||||
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
|
||||
}
|
||||
List<Map<String,Object>> result = new ArrayList<>();
|
||||
for(Integer pointNum : queryDiffusionVO.getPointspecs()){
|
||||
//获取nc文件路径
|
||||
TransportTaskChild transportTaskChild = transportTaskChildren.get(pointNum);
|
||||
TransportTaskBackwardChild transportTaskChild = transportTaskChildren.get(pointNum);
|
||||
String path = this.getBackForwardTaskNCPath(transportTask,transportTaskChild.getStationCode());
|
||||
try (NetcdfFile ncFile = NetcdfFile.open(path.toString())) {
|
||||
List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
|
||||
|
|
@ -175,28 +168,22 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
if(Objects.isNull(transportTask)){
|
||||
throw new RuntimeException("此任务不存在");
|
||||
}
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskChild::getId);
|
||||
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isEmpty(transportTaskChildren)){
|
||||
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
|
||||
}
|
||||
|
||||
List<String> stationCodes = this.getStationCodes(transportTask.getId(),transportTask.getTaskMode());
|
||||
//获取nc文件路径
|
||||
String path = "";
|
||||
if(TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())){
|
||||
path = this.getForwardTaskNCPath(transportTask);
|
||||
}else if(TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())){
|
||||
path = this.getBackForwardTaskNCPath(transportTask,transportTaskChildren.get(0).getStationCode());
|
||||
path = this.getBackForwardTaskNCPath(transportTask,stationCodes.get(0));
|
||||
}
|
||||
if(!FileUtil.exist(path)){
|
||||
throw new RuntimeException("此任务模拟结果不存在，请确认任务运行状态");
|
||||
}
|
||||
//本任务模拟的台站数据
|
||||
Map<Integer,Object> stationNumMap = new LinkedHashMap<>();
|
||||
for (int i = 0; i<transportTaskChildren.size();i++) {
|
||||
stationNumMap.put((i),transportTaskChildren.get(i).getStationCode());
|
||||
for (int i = 0; i<stationCodes.size();i++) {
|
||||
stationNumMap.put((i),stationCodes.get(i));
|
||||
}
|
||||
resultMap.put("stationNum",stationNumMap);
|
||||
//高度数据
|
||||
|
|
@ -231,6 +218,38 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
return resultMap;
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取本次模拟台站信息
|
||||
* @param taskId
|
||||
* @param taskMode
|
||||
* @return
|
||||
*/
|
||||
private List<String> getStationCodes(Integer taskId,Integer taskMode) {
|
||||
List<String> stationCodes = new ArrayList<>();
|
||||
if(TransportTaskModeEnum.FORWARD.getKey().equals(taskMode)){
|
||||
LambdaQueryWrapper<TransportTaskForwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskForwardChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskForwardChild::getId, TransportTaskForwardChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskForwardChild::getId);
|
||||
List<TransportTaskForwardChild> transportTaskChildren = forwardChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isEmpty(transportTaskChildren)){
|
||||
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
|
||||
}
|
||||
transportTaskChildren.forEach(taskChild -> stationCodes.add(taskChild.getStationCode()));
|
||||
}else if(TransportTaskModeEnum.BACK_FORWARD.getKey().equals(taskMode)){
|
||||
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
|
||||
List<TransportTaskBackwardChild> transportTaskChildren = backwardChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isEmpty(transportTaskChildren)){
|
||||
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
|
||||
}
|
||||
transportTaskChildren.forEach(taskChild -> stationCodes.add(taskChild.getStationCode()));
|
||||
}
|
||||
return stationCodes;
|
||||
}
|
||||
|
||||
/**
|
||||
* 查询符合度分析数据
|
||||
* @param taskId
|
||||
|
|
@ -241,99 +260,100 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
*/
|
||||
@Override
|
||||
public List<Map<String, Object>> getConformityAnalysis(Integer taskId,Integer stationId,String nuclideName,String facilityName) {
|
||||
TransportTask transportTask = transportTaskMapper.selectById(taskId);
|
||||
if(Objects.isNull(transportTask)){
|
||||
throw new RuntimeException("此任务不存在");
|
||||
}
|
||||
|
||||
List<Map<String, Object>> xeResults = stationDataService.getXeResults(stationId, transportTask.getStartTime(), transportTask.getEndTime(), nuclideName);
|
||||
//获取nc文件路径
|
||||
String path = this.getForwardTaskNCPath(transportTask);
|
||||
try (NetcdfFile ncFile = NetcdfFile.open(path.toString())) {
|
||||
int pointNum = -1;
|
||||
int maxPointNum = -1;
|
||||
if (StrUtil.isNotBlank(facilityName)) {
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskChild::getId);
|
||||
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
|
||||
for(int i=0;i<transportTaskChildren.size();i++){
|
||||
if (transportTaskChildren.get(i).getStationCode().equals(facilityName)){
|
||||
pointNum = i;
|
||||
maxPointNum = i;
|
||||
}
|
||||
}
|
||||
}else{
|
||||
Dimension pointspec = ncFile.findDimension("pointspec");
|
||||
maxPointNum = pointspec.getLength() -1;
|
||||
}
|
||||
|
||||
//找到最接近台站经纬度的值，作为对比点位
|
||||
List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
|
||||
List<Double> latData = NcUtil.getNCList(ncFile, "latitude");
|
||||
List<Double> timeData = NcUtil.getNCList(ncFile, "time");
|
||||
GardsStations station = this.stationDataService.getStationById(stationId);
|
||||
Map<String,Object> lonRecentValMap = this.getRecentValue(lonData, station.getLon());
|
||||
Map<String,Object> latRecentValMap = this.getRecentValue(latData, station.getLat());
|
||||
Integer lonBestIndex = Integer.parseInt(lonRecentValMap.get("bestIndex").toString());
|
||||
Integer latBestIndex = Integer.parseInt(latRecentValMap.get("bestIndex").toString());
|
||||
|
||||
Variable spec001Mr = ncFile.findVariable("spec001_mr");
|
||||
List<ConcModValVo> modValList = new ArrayList<>();
|
||||
for(int i=pointNum;i<=maxPointNum;i++){
|
||||
for(int k=0;k<timeData.size();k++){
|
||||
//nageclass=1, pointspec=1, time=30, height=6, latitude=710, longitude=1430
|
||||
int[] origin = {0, i,k,0, latBestIndex,lonBestIndex};
|
||||
int[] section = {1, 1,1,1,1,1};
|
||||
Array levelData = spec001Mr.read(origin,section);
|
||||
double[] pointData = (double[]) levelData.get1DJavaArray(DataType.DOUBLE);
|
||||
double pointDataVal = pointData[0];
|
||||
if(pointDataVal > 0){
|
||||
Instant instant = transportTask.getStartTime().toInstant();
|
||||
LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());
|
||||
localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue());
|
||||
long second = localDateTime.atZone(ZoneId.systemDefault()).toEpochSecond();
|
||||
ConcModValVo concModValVo = new ConcModValVo();
|
||||
concModValVo.setSecond(second);
|
||||
concModValVo.setPointDataVal(pointDataVal);
|
||||
modValList.add(concModValVo);
|
||||
}else {
|
||||
ConcModValVo concModValVo = new ConcModValVo();
|
||||
concModValVo.setSecond(0L);
|
||||
concModValVo.setPointDataVal(pointDataVal);
|
||||
modValList.add(concModValVo);
|
||||
}
|
||||
}
|
||||
}
|
||||
if(CollUtil.isNotEmpty(modValList)){
|
||||
xeResults.parallelStream().forEach(resultMap -> {
|
||||
Date acqStart = (Date)resultMap.get("acquisitionStart");
|
||||
Date acqEnd = (Date)resultMap.get("acquisitionStop");
|
||||
int acqStartSecond = (int)(acqStart.getTime()/1000);
|
||||
int acqStopSecond = (int)(acqEnd.getTime()/1000);
|
||||
int count = 0;
|
||||
Double sumVodValue = 0D;
|
||||
for (ConcModValVo modVal : modValList){
|
||||
if (modVal.getSecond() >= acqStartSecond && modVal.getSecond() <= acqStopSecond) {
|
||||
sumVodValue += modVal.getPointDataVal();
|
||||
count +=1;
|
||||
}
|
||||
}
|
||||
if (sumVodValue != 0){
|
||||
//纳克转换为毫克需除以1000000
|
||||
BigDecimal vodValue = new BigDecimal(sumVodValue);
|
||||
BigDecimal finalVodValue = vodValue.divide(new BigDecimal(count)).divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP);
|
||||
resultMap.put("modValue", finalVodValue);
|
||||
}else {
|
||||
resultMap.put("modValue", sumVodValue);
|
||||
}
|
||||
});
|
||||
}
|
||||
} catch (IOException | InvalidRangeException e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
return xeResults;
|
||||
// TransportTask transportTask = transportTaskMapper.selectById(taskId);
|
||||
// if(Objects.isNull(transportTask)){
|
||||
// throw new RuntimeException("此任务不存在");
|
||||
// }
|
||||
//
|
||||
// List<Map<String, Object>> xeResults = stationDataService.getXeResults(stationId, transportTask.getStartTime(), transportTask.getEndTime(), nuclideName);
|
||||
// //获取nc文件路径
|
||||
// String path = this.getForwardTaskNCPath(transportTask);
|
||||
// try (NetcdfFile ncFile = NetcdfFile.open(path.toString())) {
|
||||
// int pointNum = -1;
|
||||
// int maxPointNum = -1;
|
||||
// if (StrUtil.isNotBlank(facilityName)) {
|
||||
// LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
// queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
|
||||
// queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
|
||||
// queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
|
||||
// List<TransportTaskBackwardChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
|
||||
// for(int i=0;i<transportTaskChildren.size();i++){
|
||||
// if (transportTaskChildren.get(i).getStationCode().equals(facilityName)){
|
||||
// pointNum = i;
|
||||
// maxPointNum = i;
|
||||
// }
|
||||
// }
|
||||
// }else{
|
||||
// Dimension pointspec = ncFile.findDimension("pointspec");
|
||||
// maxPointNum = pointspec.getLength() -1;
|
||||
// }
|
||||
//
|
||||
// //找到最接近台站经纬度的值，作为对比点位
|
||||
// List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
|
||||
// List<Double> latData = NcUtil.getNCList(ncFile, "latitude");
|
||||
// List<Double> timeData = NcUtil.getNCList(ncFile, "time");
|
||||
// GardsStations station = this.stationDataService.getStationById(stationId);
|
||||
// Map<String,Object> lonRecentValMap = this.getRecentValue(lonData, station.getLon());
|
||||
// Map<String,Object> latRecentValMap = this.getRecentValue(latData, station.getLat());
|
||||
// Integer lonBestIndex = Integer.parseInt(lonRecentValMap.get("bestIndex").toString());
|
||||
// Integer latBestIndex = Integer.parseInt(latRecentValMap.get("bestIndex").toString());
|
||||
//
|
||||
// Variable spec001Mr = ncFile.findVariable("spec001_mr");
|
||||
// List<ConcModValVo> modValList = new ArrayList<>();
|
||||
// for(int i=pointNum;i<=maxPointNum;i++){
|
||||
// for(int k=0;k<timeData.size();k++){
|
||||
// //nageclass=1, pointspec=1, time=30, height=6, latitude=710, longitude=1430
|
||||
// int[] origin = {0, i,k,0, latBestIndex,lonBestIndex};
|
||||
// int[] section = {1, 1,1,1,1,1};
|
||||
// Array levelData = spec001Mr.read(origin,section);
|
||||
// double[] pointData = (double[]) levelData.get1DJavaArray(DataType.DOUBLE);
|
||||
// double pointDataVal = pointData[0];
|
||||
// if(pointDataVal > 0){
|
||||
// Instant instant = transportTask.getStartTime().toInstant();
|
||||
// LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());
|
||||
// localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue());
|
||||
// long second = localDateTime.atZone(ZoneId.systemDefault()).toEpochSecond();
|
||||
// ConcModValVo concModValVo = new ConcModValVo();
|
||||
// concModValVo.setSecond(second);
|
||||
// concModValVo.setPointDataVal(pointDataVal);
|
||||
// modValList.add(concModValVo);
|
||||
// }else {
|
||||
// ConcModValVo concModValVo = new ConcModValVo();
|
||||
// concModValVo.setSecond(0L);
|
||||
// concModValVo.setPointDataVal(pointDataVal);
|
||||
// modValList.add(concModValVo);
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// if(CollUtil.isNotEmpty(modValList)){
|
||||
// xeResults.parallelStream().forEach(resultMap -> {
|
||||
// Date acqStart = (Date)resultMap.get("acquisitionStart");
|
||||
// Date acqEnd = (Date)resultMap.get("acquisitionStop");
|
||||
// int acqStartSecond = (int)(acqStart.getTime()/1000);
|
||||
// int acqStopSecond = (int)(acqEnd.getTime()/1000);
|
||||
// int count = 0;
|
||||
// Double sumVodValue = 0D;
|
||||
// for (ConcModValVo modVal : modValList){
|
||||
// if (modVal.getSecond() >= acqStartSecond && modVal.getSecond() <= acqStopSecond) {
|
||||
// sumVodValue += modVal.getPointDataVal();
|
||||
// count +=1;
|
||||
// }
|
||||
// }
|
||||
// if (sumVodValue != 0){
|
||||
// //纳克转换为毫克需除以1000000
|
||||
// BigDecimal vodValue = new BigDecimal(sumVodValue);
|
||||
// BigDecimal finalVodValue = vodValue.divide(new BigDecimal(count)).divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP);
|
||||
// resultMap.put("modValue", finalVodValue);
|
||||
// }else {
|
||||
// resultMap.put("modValue", sumVodValue);
|
||||
// }
|
||||
// });
|
||||
// }
|
||||
// } catch (IOException | InvalidRangeException e) {
|
||||
// throw new RuntimeException(e);
|
||||
// }
|
||||
// return xeResults;
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -344,12 +364,12 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
*/
|
||||
@Override
|
||||
public List<String> getTaskFacility(Integer taskId) {
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskChild::getId);
|
||||
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
|
||||
return transportTaskChildren.stream().map(TransportTaskChild::getStationCode).collect(Collectors.toList());
|
||||
LambdaQueryWrapper<TransportTaskForwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskForwardChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskForwardChild::getId, TransportTaskForwardChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskForwardChild::getId);
|
||||
List<TransportTaskForwardChild> transportTaskChildren = forwardChildMapper.selectList(queryWrapper);
|
||||
return transportTaskChildren.stream().map(TransportTaskForwardChild::getStationCode).collect(Collectors.toList());
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -378,104 +398,104 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
*/
|
||||
@Override
|
||||
public void handleContributionAnalysis(Integer taskId) {
|
||||
//查询任务数据
|
||||
TransportTask transportTask = transportTaskMapper.selectById(taskId);
|
||||
//查询需要处理贡献分析数据的台站
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,taskId);
|
||||
List<TransportTaskChild> stationInfos = this.transportTaskChildMapper.selectList(queryWrapper);
|
||||
|
||||
//所以核设施数据
|
||||
List<GardsNuclearReactors> facilitys = (List<GardsNuclearReactors>) redisUtil.get(CommonConstant.ALL_NUCLEARFACILITY);
|
||||
|
||||
NetcdfFile ncFile = null;
|
||||
try {
|
||||
for(TransportTaskChild stationInfo :stationInfos) {
|
||||
//每个台站的结果数据
|
||||
ContributionAnalysisVO contributionAnalysisVO = new ContributionAnalysisVO();
|
||||
//存储台站每天的浓度值数据
|
||||
Map<String,Double> stationEveryDayConcDatas = new LinkedHashMap<>();
|
||||
//存储核设施每天的浓度值数据
|
||||
Map<String,Map<String,Double>> facilityEveryDayConcDatas = new HashMap<>();
|
||||
//饼图数据
|
||||
Map<String,Double> pipeChartData = new HashMap<>();
|
||||
//总浓度值
|
||||
Double totalConc = 0D;
|
||||
//获取nc文件路径
|
||||
String path = this.getBackForwardTaskNCPath(transportTask,stationInfo.getStationCode());
|
||||
ncFile = NetcdfFile.open(path.toString());
|
||||
List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
|
||||
List<Double> latData = NcUtil.getNCList(ncFile, "latitude");
|
||||
List<Double> timeData = NcUtil.getNCList(ncFile, "time");
|
||||
Variable spec001Mr = ncFile.findVariable("spec001_mr");
|
||||
for(int k=0;k<timeData.size();k++){
|
||||
System.out.println(stationInfo.getStationCode()+"循环:"+k+"，共"+timeData.size()+"次");
|
||||
//处理日期数据
|
||||
Instant instant = transportTask.getEndTime().toInstant();
|
||||
LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());
|
||||
localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue());
|
||||
String dayStr = LocalDateTimeUtil.format(localDateTime, "yyyy-MM-dd");
|
||||
if (!stationEveryDayConcDatas.containsKey(dayStr)) {
|
||||
stationEveryDayConcDatas.put(dayStr,0D);
|
||||
}
|
||||
//获取台站点位取整后±1.5度内的点坐标及数据，使用插值算法求，台站点位的值
|
||||
Double stationConc = this.getTargetSiteConc(lonData,latData,stationInfo.getLon(),stationInfo.getLat(),k,spec001Mr);
|
||||
//累加台站位置当前天的浓度数据
|
||||
stationEveryDayConcDatas.put(dayStr,stationEveryDayConcDatas.get(dayStr)+stationConc);
|
||||
|
||||
for (GardsNuclearReactors facility : facilitys){
|
||||
Double facilityConc = this.getTargetSiteConc(lonData,latData,facility.getLongitude(),facility.getLatitude(),k,spec001Mr);
|
||||
if (facilityConc>0){
|
||||
if (!facilityEveryDayConcDatas.containsKey(dayStr)) {
|
||||
Map<String,Double> facilityConcMap = new HashMap<>();
|
||||
facilityConcMap.put(facility.getUnitName(),facilityConc);
|
||||
facilityEveryDayConcDatas.put(dayStr,facilityConcMap);
|
||||
}else {
|
||||
Map<String,Double> facilityConcMap = facilityEveryDayConcDatas.get(dayStr);
|
||||
if (!facilityConcMap.containsKey(facility.getUnitName())) {
|
||||
facilityConcMap.put(facility.getUnitName(),facilityConc);
|
||||
}else {
|
||||
facilityConcMap.put(facility.getUnitName(),facilityConcMap.get(facility.getUnitName())+facilityConc);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
//计算本模拟时间内总浓度数据
|
||||
totalConc = stationEveryDayConcDatas.values().stream().mapToDouble(Double::doubleValue).sum();
|
||||
//处理饼图数据
|
||||
if(CollUtil.isNotEmpty(facilityEveryDayConcDatas)){
|
||||
for (Map<String,Double> facilityConcMap : facilityEveryDayConcDatas.values()) {
|
||||
Set<Map.Entry<String, Double>> entries = facilityConcMap.entrySet();
|
||||
for (Map.Entry<String, Double> entry : entries) {
|
||||
if (!pipeChartData.containsKey(entry.getKey())) {
|
||||
pipeChartData.put(entry.getKey(),entry.getValue());
|
||||
}else {
|
||||
pipeChartData.put(entry.getKey(),pipeChartData.get(entry.getKey())+entry.getValue());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
//处理返回值
|
||||
contributionAnalysisVO.setTotalConc(totalConc);
|
||||
contributionAnalysisVO.setStationEveryDayConcDatas(stationEveryDayConcDatas);
|
||||
contributionAnalysisVO.setFacilityEveryDayConcDatas(facilityEveryDayConcDatas);
|
||||
contributionAnalysisVO.setNuclideNames(new ArrayList<>(pipeChartData.keySet()));
|
||||
contributionAnalysisVO.setPipeChartData(pipeChartData);
|
||||
redisUtil.set(CommonConstant.TRANSPORT_CONTRIBUTION_ANALYSIS+transportTask.getId()+":"+stationInfo.getStationCode(), contributionAnalysisVO);
|
||||
ncFile.close();
|
||||
}
|
||||
}catch (IOException | InvalidRangeException e) {
|
||||
throw new RuntimeException(e);
|
||||
}finally {
|
||||
try {
|
||||
if(ncFile !=null){
|
||||
ncFile.close();
|
||||
}
|
||||
} catch (IOException e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
// //查询任务数据
|
||||
// TransportTask transportTask = transportTaskMapper.selectById(taskId);
|
||||
// //查询需要处理贡献分析数据的台站
|
||||
// LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
// queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
|
||||
// List<TransportTaskBackwardChild> stationInfos = this.transportTaskChildMapper.selectList(queryWrapper);
|
||||
//
|
||||
// //所以核设施数据
|
||||
// List<GardsNuclearReactors> facilitys = (List<GardsNuclearReactors>) redisUtil.get(CommonConstant.ALL_NUCLEARFACILITY);
|
||||
//
|
||||
// NetcdfFile ncFile = null;
|
||||
// try {
|
||||
// for(TransportTaskBackwardChild stationInfo :stationInfos) {
|
||||
// //每个台站的结果数据
|
||||
// ContributionAnalysisVO contributionAnalysisVO = new ContributionAnalysisVO();
|
||||
// //存储台站每天的浓度值数据
|
||||
// Map<String,Double> stationEveryDayConcDatas = new LinkedHashMap<>();
|
||||
// //存储核设施每天的浓度值数据
|
||||
// Map<String,Map<String,Double>> facilityEveryDayConcDatas = new HashMap<>();
|
||||
// //饼图数据
|
||||
// Map<String,Double> pipeChartData = new HashMap<>();
|
||||
// //总浓度值
|
||||
// Double totalConc = 0D;
|
||||
// //获取nc文件路径
|
||||
// String path = this.getBackForwardTaskNCPath(transportTask,stationInfo.getStationCode());
|
||||
// ncFile = NetcdfFile.open(path.toString());
|
||||
// List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
|
||||
// List<Double> latData = NcUtil.getNCList(ncFile, "latitude");
|
||||
// List<Double> timeData = NcUtil.getNCList(ncFile, "time");
|
||||
// Variable spec001Mr = ncFile.findVariable("spec001_mr");
|
||||
// for(int k=0;k<timeData.size();k++){
|
||||
// System.out.println(stationInfo.getStationCode()+"循环:"+k+"，共"+timeData.size()+"次");
|
||||
// //处理日期数据
|
||||
// Instant instant = transportTask.getEndTime().toInstant();
|
||||
// LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());
|
||||
// localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue());
|
||||
// String dayStr = LocalDateTimeUtil.format(localDateTime, "yyyy-MM-dd");
|
||||
// if (!stationEveryDayConcDatas.containsKey(dayStr)) {
|
||||
// stationEveryDayConcDatas.put(dayStr,0D);
|
||||
// }
|
||||
// //获取台站点位取整后±1.5度内的点坐标及数据，使用插值算法求，台站点位的值
|
||||
// Double stationConc = this.getTargetSiteConc(lonData,latData,stationInfo.getLon(),stationInfo.getLat(),k,spec001Mr);
|
||||
// //累加台站位置当前天的浓度数据
|
||||
// stationEveryDayConcDatas.put(dayStr,stationEveryDayConcDatas.get(dayStr)+stationConc);
|
||||
//
|
||||
// for (GardsNuclearReactors facility : facilitys){
|
||||
// Double facilityConc = this.getTargetSiteConc(lonData,latData,facility.getLongitude(),facility.getLatitude(),k,spec001Mr);
|
||||
// if (facilityConc>0){
|
||||
// if (!facilityEveryDayConcDatas.containsKey(dayStr)) {
|
||||
// Map<String,Double> facilityConcMap = new HashMap<>();
|
||||
// facilityConcMap.put(facility.getUnitName(),facilityConc);
|
||||
// facilityEveryDayConcDatas.put(dayStr,facilityConcMap);
|
||||
// }else {
|
||||
// Map<String,Double> facilityConcMap = facilityEveryDayConcDatas.get(dayStr);
|
||||
// if (!facilityConcMap.containsKey(facility.getUnitName())) {
|
||||
// facilityConcMap.put(facility.getUnitName(),facilityConc);
|
||||
// }else {
|
||||
// facilityConcMap.put(facility.getUnitName(),facilityConcMap.get(facility.getUnitName())+facilityConc);
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// //计算本模拟时间内总浓度数据
|
||||
// totalConc = stationEveryDayConcDatas.values().stream().mapToDouble(Double::doubleValue).sum();
|
||||
// //处理饼图数据
|
||||
// if(CollUtil.isNotEmpty(facilityEveryDayConcDatas)){
|
||||
// for (Map<String,Double> facilityConcMap : facilityEveryDayConcDatas.values()) {
|
||||
// Set<Map.Entry<String, Double>> entries = facilityConcMap.entrySet();
|
||||
// for (Map.Entry<String, Double> entry : entries) {
|
||||
// if (!pipeChartData.containsKey(entry.getKey())) {
|
||||
// pipeChartData.put(entry.getKey(),entry.getValue());
|
||||
// }else {
|
||||
// pipeChartData.put(entry.getKey(),pipeChartData.get(entry.getKey())+entry.getValue());
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// //处理返回值
|
||||
// contributionAnalysisVO.setTotalConc(totalConc);
|
||||
// contributionAnalysisVO.setStationEveryDayConcDatas(stationEveryDayConcDatas);
|
||||
// contributionAnalysisVO.setFacilityEveryDayConcDatas(facilityEveryDayConcDatas);
|
||||
// contributionAnalysisVO.setNuclideNames(new ArrayList<>(pipeChartData.keySet()));
|
||||
// contributionAnalysisVO.setPipeChartData(pipeChartData);
|
||||
// redisUtil.set(CommonConstant.TRANSPORT_CONTRIBUTION_ANALYSIS+transportTask.getId()+":"+stationInfo.getStationCode(), contributionAnalysisVO);
|
||||
// ncFile.close();
|
||||
// }
|
||||
// }catch (IOException | InvalidRangeException e) {
|
||||
// throw new RuntimeException(e);
|
||||
// }finally {
|
||||
// try {
|
||||
// if(ncFile !=null){
|
||||
// ncFile.close();
|
||||
// }
|
||||
// } catch (IOException e) {
|
||||
// throw new RuntimeException(e);
|
||||
// }
|
||||
// }
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -489,11 +509,11 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
if(Objects.isNull(transportTask)){
|
||||
throw new RuntimeException("此任务不存在");
|
||||
}
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskChild::getId);
|
||||
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
|
||||
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
|
||||
List<TransportTaskBackwardChild> transportTaskChildren = backwardChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isEmpty(transportTaskChildren)){
|
||||
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
|
||||
}
|
||||
|
|
@ -527,19 +547,19 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
if(Objects.isNull(transportTask)){
|
||||
throw new RuntimeException("此任务不存在");
|
||||
}
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskChild::getId,TransportTaskChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskChild::getId);
|
||||
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isEmpty(transportTaskChildren)){
|
||||
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
|
||||
}
|
||||
//获取nc文件路径
|
||||
String path = "";
|
||||
if(TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())){
|
||||
path = this.getForwardTaskNCPath(transportTask);
|
||||
}else if(TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())){
|
||||
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
|
||||
queryWrapper.select(TransportTaskBackwardChild::getId, TransportTaskBackwardChild::getStationCode);
|
||||
queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
|
||||
List<TransportTaskBackwardChild> transportTaskChildren = backwardChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isEmpty(transportTaskChildren)){
|
||||
throw new RuntimeException("此任务站点信息不存在，请确认任务配置信息");
|
||||
}
|
||||
path = this.getBackForwardTaskNCPath(transportTask,transportTaskChildren.get(0).getStationCode());
|
||||
}
|
||||
if(!FileUtil.exist(path)){
|
||||
|
|
@ -569,46 +589,46 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
*/
|
||||
@Override
|
||||
public Map<String,Double> getTimingAnalysis(Integer taskId,String stationCode,String facilityName,Integer timeNum) {
|
||||
TransportTask transportTask = transportTaskMapper.selectById(taskId);
|
||||
if(Objects.isNull(transportTask)){
|
||||
throw new RuntimeException("此任务不存在");
|
||||
}
|
||||
//包含从缓存中拿
|
||||
String key = CommonConstant.TRANSPORT_TIMING_ANALYSIS+transportTask.getId()+":"+stationCode+":"+facilityName;
|
||||
if(redisUtil.hasKey(key)){
|
||||
Map<String,Double> concValMap = (LinkedHashMap)redisUtil.get(key);
|
||||
//如果是3小时纬度，直接范围，因为最小是3小时
|
||||
if (TransportTimingAnalysisEnum.THREE_HOURS.getKey().equals(timeNum)) {
|
||||
concValMap.forEach((k,v)->{
|
||||
//纳克转换为毫克需除以1000000
|
||||
BigDecimal concValue = new BigDecimal(v);
|
||||
BigDecimal finalValue = concValue.divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP);
|
||||
concValMap.put(k,finalValue.doubleValue());
|
||||
});
|
||||
return concValMap;
|
||||
}else{
|
||||
Map<String,Double> resultMap = new LinkedHashMap<>();
|
||||
Instant endTimeInstant = transportTask.getEndTime().toInstant();
|
||||
LocalDateTime endTime = LocalDateTime.ofInstant(endTimeInstant, ZoneId.systemDefault());
|
||||
|
||||
Instant startTimeInstant = transportTask.getStartTime().toInstant();
|
||||
LocalDateTime startTime = LocalDateTime.ofInstant(startTimeInstant, ZoneId.systemDefault());
|
||||
|
||||
boolean flag = true;
|
||||
while(flag){
|
||||
String dayTimeStr = LocalDateTimeUtil.format(startTime, "yyyy-MM-dd HH:mm:ss");
|
||||
//纳克转换为毫克需除以1000000
|
||||
BigDecimal concValue = new BigDecimal(concValMap.get(dayTimeStr));
|
||||
BigDecimal finalValue = concValue.divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP);
|
||||
resultMap.put(dayTimeStr,finalValue.doubleValue());
|
||||
startTime = startTime.plusHours(timeNum);
|
||||
if(startTime.isEqual(endTime)){
|
||||
flag = false;
|
||||
}
|
||||
}
|
||||
return resultMap;
|
||||
}
|
||||
}
|
||||
// TransportTask transportTask = transportTaskMapper.selectById(taskId);
|
||||
// if(Objects.isNull(transportTask)){
|
||||
// throw new RuntimeException("此任务不存在");
|
||||
// }
|
||||
// //包含从缓存中拿
|
||||
// String key = CommonConstant.TRANSPORT_TIMING_ANALYSIS+transportTask.getId()+":"+stationCode+":"+facilityName;
|
||||
// if(redisUtil.hasKey(key)){
|
||||
// Map<String,Double> concValMap = (LinkedHashMap)redisUtil.get(key);
|
||||
// //如果是3小时纬度，直接范围，因为最小是3小时
|
||||
// if (TransportTimingAnalysisEnum.THREE_HOURS.getKey().equals(timeNum)) {
|
||||
// concValMap.forEach((k,v)->{
|
||||
// //纳克转换为毫克需除以1000000
|
||||
// BigDecimal concValue = new BigDecimal(v);
|
||||
// BigDecimal finalValue = concValue.divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP);
|
||||
// concValMap.put(k,finalValue.doubleValue());
|
||||
// });
|
||||
// return concValMap;
|
||||
// }else{
|
||||
// Map<String,Double> resultMap = new LinkedHashMap<>();
|
||||
// Instant endTimeInstant = transportTask.getEndTime().toInstant();
|
||||
// LocalDateTime endTime = LocalDateTime.ofInstant(endTimeInstant, ZoneId.systemDefault());
|
||||
//
|
||||
// Instant startTimeInstant = transportTask.getStartTime().toInstant();
|
||||
// LocalDateTime startTime = LocalDateTime.ofInstant(startTimeInstant, ZoneId.systemDefault());
|
||||
//
|
||||
// boolean flag = true;
|
||||
// while(flag){
|
||||
// String dayTimeStr = LocalDateTimeUtil.format(startTime, "yyyy-MM-dd HH:mm:ss");
|
||||
// //纳克转换为毫克需除以1000000
|
||||
// BigDecimal concValue = new BigDecimal(concValMap.get(dayTimeStr));
|
||||
// BigDecimal finalValue = concValue.divide(new BigDecimal(1000000)).setScale(5,BigDecimal.ROUND_HALF_UP);
|
||||
// resultMap.put(dayTimeStr,finalValue.doubleValue());
|
||||
// startTime = startTime.plusHours(timeNum);
|
||||
// if(startTime.isEqual(endTime)){
|
||||
// flag = false;
|
||||
// }
|
||||
// }
|
||||
// return resultMap;
|
||||
// }
|
||||
// }
|
||||
return Map.of();
|
||||
}
|
||||
|
||||
|
|
@ -619,62 +639,62 @@ public class TransportResultDataServiceImpl implements TransportResultDataServic
|
|||
*/
|
||||
@Override
|
||||
public void handleTimingAnalysis(Integer taskId) {
|
||||
TransportTask transportTask = this.transportTaskMapper.selectById(taskId);
|
||||
|
||||
//查询需要处理数据的台站
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,taskId);
|
||||
List<TransportTaskChild> stationInfos = this.transportTaskChildMapper.selectList(queryWrapper);
|
||||
|
||||
//所以核设施数据
|
||||
List<GardsNuclearReactors> facilitys = (List<GardsNuclearReactors>) redisUtil.get(CommonConstant.ALL_NUCLEARFACILITY);
|
||||
|
||||
NetcdfFile ncFile = null;
|
||||
try {
|
||||
for(TransportTaskChild stationInfo :stationInfos) {
|
||||
log.info("处理"+stationInfo.getStationCode()+"台站数据");
|
||||
Map<String,Map<String,Double>> everyFacilityConcDatas = new HashMap<>();
|
||||
//获取nc文件路径
|
||||
String path = this.getBackForwardTaskNCPath(transportTask,stationInfo.getStationCode());
|
||||
ncFile = NetcdfFile.open(path.toString());
|
||||
List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
|
||||
List<Double> latData = NcUtil.getNCList(ncFile, "latitude");
|
||||
List<Double> timeData = NcUtil.getNCList(ncFile, "time");
|
||||
Variable spec001Mr = ncFile.findVariable("spec001_mr");
|
||||
for (GardsNuclearReactors facility : facilitys){
|
||||
//存储台站每步的浓度值数据
|
||||
Map<String,Double> everyStepConcDatas = new LinkedHashMap<>();
|
||||
for(int k=0;k<timeData.size();k++){
|
||||
//处理日期数据
|
||||
Instant instant = transportTask.getEndTime().toInstant();
|
||||
LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());
|
||||
localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue());
|
||||
String dayTimeStr = LocalDateTimeUtil.format(localDateTime, "yyyy-MM-dd HH:mm:ss");
|
||||
//获取台站点位取整后±1.5度内的点坐标及数据，使用插值算法求，台站点位的值
|
||||
Double facilityConc = this.getTargetSiteConc(lonData,latData,facility.getLongitude(),facility.getLatitude(),k,spec001Mr);
|
||||
everyStepConcDatas.put(dayTimeStr,facilityConc);
|
||||
}
|
||||
everyFacilityConcDatas.put(facility.getUnitName(),everyStepConcDatas);
|
||||
}
|
||||
if(CollUtil.isNotEmpty(everyFacilityConcDatas)){
|
||||
everyFacilityConcDatas.forEach((facilityName,facilityConcData)->{
|
||||
String key = CommonConstant.TRANSPORT_TIMING_ANALYSIS+transportTask.getId()+":"+stationInfo.getStationCode()+":"+facilityName;
|
||||
redisUtil.set(key, facilityConcData);
|
||||
});
|
||||
}
|
||||
ncFile.close();
|
||||
}
|
||||
}catch (IOException | InvalidRangeException e) {
|
||||
throw new RuntimeException(e);
|
||||
}finally {
|
||||
try {
|
||||
if(ncFile !=null){
|
||||
ncFile.close();
|
||||
}
|
||||
} catch (IOException e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
// TransportTask transportTask = this.transportTaskMapper.selectById(taskId);
|
||||
//
|
||||
// //查询需要处理数据的台站
|
||||
// LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
// queryWrapper.eq(TransportTaskBackwardChild::getTaskId,taskId);
|
||||
// List<TransportTaskBackwardChild> stationInfos = this.transportTaskChildMapper.selectList(queryWrapper);
|
||||
//
|
||||
// //所以核设施数据
|
||||
// List<GardsNuclearReactors> facilitys = (List<GardsNuclearReactors>) redisUtil.get(CommonConstant.ALL_NUCLEARFACILITY);
|
||||
//
|
||||
// NetcdfFile ncFile = null;
|
||||
// try {
|
||||
// for(TransportTaskBackwardChild stationInfo :stationInfos) {
|
||||
// log.info("处理"+stationInfo.getStationCode()+"台站数据");
|
||||
// Map<String,Map<String,Double>> everyFacilityConcDatas = new HashMap<>();
|
||||
// //获取nc文件路径
|
||||
// String path = this.getBackForwardTaskNCPath(transportTask,stationInfo.getStationCode());
|
||||
// ncFile = NetcdfFile.open(path.toString());
|
||||
// List<Double> lonData = NcUtil.getNCList(ncFile, "longitude");
|
||||
// List<Double> latData = NcUtil.getNCList(ncFile, "latitude");
|
||||
// List<Double> timeData = NcUtil.getNCList(ncFile, "time");
|
||||
// Variable spec001Mr = ncFile.findVariable("spec001_mr");
|
||||
// for (GardsNuclearReactors facility : facilitys){
|
||||
// //存储台站每步的浓度值数据
|
||||
// Map<String,Double> everyStepConcDatas = new LinkedHashMap<>();
|
||||
// for(int k=0;k<timeData.size();k++){
|
||||
// //处理日期数据
|
||||
// Instant instant = transportTask.getEndTime().toInstant();
|
||||
// LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());
|
||||
// localDateTime = localDateTime.plusSeconds(timeData.get(k).intValue());
|
||||
// String dayTimeStr = LocalDateTimeUtil.format(localDateTime, "yyyy-MM-dd HH:mm:ss");
|
||||
// //获取台站点位取整后±1.5度内的点坐标及数据，使用插值算法求，台站点位的值
|
||||
// Double facilityConc = this.getTargetSiteConc(lonData,latData,facility.getLongitude(),facility.getLatitude(),k,spec001Mr);
|
||||
// everyStepConcDatas.put(dayTimeStr,facilityConc);
|
||||
// }
|
||||
// everyFacilityConcDatas.put(facility.getUnitName(),everyStepConcDatas);
|
||||
// }
|
||||
// if(CollUtil.isNotEmpty(everyFacilityConcDatas)){
|
||||
// everyFacilityConcDatas.forEach((facilityName,facilityConcData)->{
|
||||
// String key = CommonConstant.TRANSPORT_TIMING_ANALYSIS+transportTask.getId()+":"+stationInfo.getStationCode()+":"+facilityName;
|
||||
// redisUtil.set(key, facilityConcData);
|
||||
// });
|
||||
// }
|
||||
// ncFile.close();
|
||||
// }
|
||||
// }catch (IOException | InvalidRangeException e) {
|
||||
// throw new RuntimeException(e);
|
||||
// }finally {
|
||||
// try {
|
||||
// if(ncFile !=null){
|
||||
// ncFile.close();
|
||||
// }
|
||||
// } catch (IOException e) {
|
||||
// throw new RuntimeException(e);
|
||||
// }
|
||||
// }
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -7,22 +7,22 @@ import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
|
|||
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.jeecg.common.constant.CommonConstant;
|
||||
import org.jeecg.common.constant.enums.TransportReleaseDataSource;
|
||||
import org.jeecg.common.constant.enums.TransportTaskModeEnum;
|
||||
import org.jeecg.common.constant.enums.TransportTaskStatusEnum;
|
||||
import org.jeecg.common.constant.enums.TransportTaskTypeEnum;
|
||||
import org.jeecg.common.properties.DataFusionProperties;
|
||||
import org.jeecg.common.properties.ServerProperties;
|
||||
import org.jeecg.common.properties.SystemStorageProperties;
|
||||
import org.jeecg.common.properties.TransportSimulationProperties;
|
||||
import org.jeecg.common.system.query.PageRequest;
|
||||
import org.jeecg.common.util.RedisUtil;
|
||||
import org.jeecg.modules.base.entity.TransportTask;
|
||||
import org.jeecg.modules.base.entity.TransportTaskChild;
|
||||
import org.jeecg.modules.base.entity.TransportTaskLog;
|
||||
import org.jeecg.modules.base.mapper.TransportTaskChildMapper;
|
||||
import org.jeecg.modules.base.mapper.TransportTaskLogMapper;
|
||||
import org.jeecg.modules.base.mapper.TransportTaskMapper;
|
||||
import org.jeecg.modules.base.mapper.WeatherDataMapper;
|
||||
import org.jeecg.modules.base.entity.*;
|
||||
import org.jeecg.modules.base.mapper.*;
|
||||
import org.jeecg.service.TransportTaskService;
|
||||
import org.jeecg.task.flexparttask.TransportTaskExec;
|
||||
import org.jeecg.task.flexparttask.AbstractTaskExec;
|
||||
import org.jeecg.task.flexparttask.BackwardTaskExec;
|
||||
import org.jeecg.task.flexparttask.ForwardTaskExec;
|
||||
import org.springframework.stereotype.Service;
|
||||
import org.springframework.transaction.annotation.Transactional;
|
||||
import java.time.LocalDate;
|
||||
|
|
@ -37,12 +37,17 @@ import java.util.Objects;
|
|||
@Service
|
||||
public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,TransportTask> implements TransportTaskService {
|
||||
|
||||
private final TransportTaskChildMapper transportTaskChildMapper;
|
||||
private final TransportTaskBackwardChildMapper taskBackwardChildMapper;
|
||||
private final TransportTaskLogMapper transportTaskLogMapper;
|
||||
private final WeatherDataMapper weatherDataMapper;
|
||||
private final TransportSimulationProperties simulationProperties;
|
||||
private final SystemStorageProperties systemStorageProperties;
|
||||
private final RedisUtil redisUtil;
|
||||
private final DataFusionProperties dataFusionProperties;
|
||||
private final TransportTaskForwardReleaseMapper taskForwardReleaseMapper;
|
||||
private final TransportTaskForwardChildMapper taskForwardChildMapper;
|
||||
private final TransportTaskForwardSpeciesMapper taskForwardSpeciesMapper;
|
||||
private final ServerProperties serverProperties;
|
||||
|
||||
/**
|
||||
* 分页查询任务列表
|
||||
|
|
@ -92,11 +97,37 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
|
|||
transportTask.setTaskType(TransportTaskTypeEnum.MANUALLY.getKey());
|
||||
transportTask.setTimeConsuming(0D);
|
||||
this.baseMapper.insert(transportTask);
|
||||
if(CollUtil.isNotEmpty(transportTask.getChildList())){
|
||||
transportTask.getChildList().forEach(transportTaskChild -> {
|
||||
if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode()) &&
|
||||
CollUtil.isNotEmpty(transportTask.getBackwardChild())) {
|
||||
transportTask.getBackwardChild().forEach(transportTaskChild -> {
|
||||
transportTaskChild.setTaskId(transportTask.getId());
|
||||
});
|
||||
transportTaskChildMapper.insert(transportTask.getChildList());
|
||||
taskBackwardChildMapper.insert(transportTask.getBackwardChild());
|
||||
}else if (TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())) {
|
||||
if(CollUtil.isNotEmpty(transportTask.getForwardChild())){
|
||||
if(TransportReleaseDataSource.MANUAL_ENTRY.getKey().equals(transportTask.getReleaseDataSource())){
|
||||
for (TransportTaskForwardChild child : transportTask.getForwardChild()) {
|
||||
child.setTaskId(transportTask.getId());
|
||||
taskForwardChildMapper.insert(child);
|
||||
child.getForwardReleaseChild().forEach(forwardReleaseChild -> {
|
||||
forwardReleaseChild.setForwardChildId(child.getId());
|
||||
forwardReleaseChild.setTaskId(transportTask.getId());
|
||||
});
|
||||
taskForwardReleaseMapper.insert(child.getForwardReleaseChild());
|
||||
}
|
||||
}else {
|
||||
transportTask.getForwardChild().forEach(transportTaskChild -> {
|
||||
transportTaskChild.setTaskId(transportTask.getId());
|
||||
});
|
||||
taskForwardChildMapper.insert(transportTask.getForwardChild());
|
||||
}
|
||||
}
|
||||
if(CollUtil.isNotEmpty(transportTask.getSpecies())){
|
||||
transportTask.getSpecies().forEach(transportTaskSpecies -> {
|
||||
transportTaskSpecies.setTaskId(transportTask.getId());
|
||||
});
|
||||
taskForwardSpeciesMapper.insert(transportTask.getSpecies());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -112,11 +143,37 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
|
|||
if(Objects.isNull(transportTask)){
|
||||
throw new RuntimeException("此任务不存在");
|
||||
}
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,transportTask.getId());
|
||||
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isNotEmpty(transportTaskChildren)){
|
||||
transportTask.setChildList(transportTaskChildren);
|
||||
if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())) {
|
||||
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,transportTask.getId());
|
||||
List<TransportTaskBackwardChild> transportTaskChildren = taskBackwardChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isNotEmpty(transportTaskChildren)){
|
||||
transportTask.setBackwardChild(transportTaskChildren);
|
||||
}
|
||||
}else if (TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())) {
|
||||
LambdaQueryWrapper<TransportTaskForwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskForwardChild::getTaskId,transportTask.getId());
|
||||
List<TransportTaskForwardChild> taskChildren = taskForwardChildMapper.selectList(queryWrapper);
|
||||
if(CollUtil.isNotEmpty(taskChildren)){
|
||||
transportTask.setForwardChild(taskChildren);
|
||||
if(TransportReleaseDataSource.MANUAL_ENTRY.getKey().equals(transportTask.getReleaseDataSource())){
|
||||
taskChildren.forEach(taskChild -> {
|
||||
LambdaQueryWrapper<TransportTaskForwardRelease> releaseQueryWrapper = new LambdaQueryWrapper<>();
|
||||
releaseQueryWrapper.eq(TransportTaskForwardRelease::getForwardChildId,taskChild.getId());
|
||||
releaseQueryWrapper.orderByAsc(TransportTaskForwardRelease::getStartTime);
|
||||
List<TransportTaskForwardRelease> forwardReleases = taskForwardReleaseMapper.selectList(releaseQueryWrapper);
|
||||
if(CollUtil.isNotEmpty(forwardReleases)){
|
||||
taskChild.setForwardReleaseChild(forwardReleases);
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
LambdaQueryWrapper<TransportTaskForwardSpecies> speciesQueryWrapper = new LambdaQueryWrapper<>();
|
||||
speciesQueryWrapper.eq(TransportTaskForwardSpecies::getTaskId,transportTask.getId());
|
||||
List<TransportTaskForwardSpecies> species = taskForwardSpeciesMapper.selectList(speciesQueryWrapper);
|
||||
if(CollUtil.isNotEmpty(species)){
|
||||
transportTask.setSpecies(species);
|
||||
}
|
||||
}
|
||||
return transportTask;
|
||||
}
|
||||
|
|
@ -147,16 +204,55 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
|
|||
checkIdResult.setZ1(transportTask.getZ1());
|
||||
checkIdResult.setZ2(transportTask.getZ2());
|
||||
checkIdResult.setParticleCount(transportTask.getParticleCount());
|
||||
checkIdResult.setReleaseDataSource(transportTask.getReleaseDataSource());
|
||||
this.baseMapper.updateById(checkIdResult);
|
||||
//先删除再保存
|
||||
LambdaQueryWrapper<TransportTaskChild> delQueryWrapper = new LambdaQueryWrapper<>();
|
||||
delQueryWrapper.eq(TransportTaskChild::getTaskId,checkIdResult.getId());
|
||||
transportTaskChildMapper.delete(delQueryWrapper);
|
||||
if(CollUtil.isNotEmpty(transportTask.getChildList())){
|
||||
transportTask.getChildList().forEach(transportTaskChild -> {
|
||||
transportTaskChild.setTaskId(transportTask.getId());
|
||||
});
|
||||
transportTaskChildMapper.insert(transportTask.getChildList());
|
||||
if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())) {
|
||||
LambdaQueryWrapper<TransportTaskBackwardChild> delQueryWrapper = new LambdaQueryWrapper<>();
|
||||
delQueryWrapper.eq(TransportTaskBackwardChild::getTaskId,checkIdResult.getId());
|
||||
taskBackwardChildMapper.delete(delQueryWrapper);
|
||||
if(CollUtil.isNotEmpty(transportTask.getBackwardChild())){
|
||||
transportTask.getBackwardChild().forEach(transportTaskChild -> {
|
||||
transportTaskChild.setTaskId(transportTask.getId());
|
||||
});
|
||||
taskBackwardChildMapper.insert(transportTask.getBackwardChild());
|
||||
}
|
||||
}else if (TransportTaskModeEnum.FORWARD.getKey().equals(transportTask.getTaskMode())) {
|
||||
//先删除再新增
|
||||
LambdaQueryWrapper<TransportTaskForwardChild> delForwardChildQueryWrapper = new LambdaQueryWrapper<>();
|
||||
delForwardChildQueryWrapper.eq(TransportTaskForwardChild::getTaskId,checkIdResult.getId());
|
||||
taskForwardChildMapper.delete(delForwardChildQueryWrapper);
|
||||
LambdaQueryWrapper<TransportTaskForwardRelease> delForwardReleaseQueryWrapper = new LambdaQueryWrapper<>();
|
||||
delForwardReleaseQueryWrapper.eq(TransportTaskForwardRelease::getTaskId,checkIdResult.getId());
|
||||
taskForwardReleaseMapper.delete(delForwardReleaseQueryWrapper);
|
||||
LambdaQueryWrapper<TransportTaskForwardSpecies> delForwardSpeciesQueryWrapper = new LambdaQueryWrapper<>();
|
||||
delForwardSpeciesQueryWrapper.eq(TransportTaskForwardSpecies::getTaskId,checkIdResult.getId());
|
||||
taskForwardSpeciesMapper.delete(delForwardSpeciesQueryWrapper);
|
||||
//重新新增
|
||||
if(CollUtil.isNotEmpty(transportTask.getForwardChild())){
|
||||
if(TransportReleaseDataSource.MANUAL_ENTRY.getKey().equals(transportTask.getReleaseDataSource())){
|
||||
for (TransportTaskForwardChild child : transportTask.getForwardChild()) {
|
||||
child.setTaskId(transportTask.getId());
|
||||
taskForwardChildMapper.insert(child);
|
||||
child.getForwardReleaseChild().forEach(forwardReleaseChild -> {
|
||||
forwardReleaseChild.setForwardChildId(child.getId());
|
||||
forwardReleaseChild.setTaskId(transportTask.getId());
|
||||
});
|
||||
taskForwardReleaseMapper.insert(child.getForwardReleaseChild());
|
||||
}
|
||||
}else {
|
||||
transportTask.getForwardChild().forEach(transportTaskChild -> {
|
||||
transportTaskChild.setTaskId(transportTask.getId());
|
||||
});
|
||||
taskForwardChildMapper.insert(transportTask.getForwardChild());
|
||||
}
|
||||
}
|
||||
if(CollUtil.isNotEmpty(transportTask.getSpecies())){
|
||||
transportTask.getSpecies().forEach(transportTaskSpecies -> {
|
||||
transportTaskSpecies.setTaskId(transportTask.getId());
|
||||
});
|
||||
taskForwardSpeciesMapper.insert(transportTask.getSpecies());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -168,13 +264,15 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
|
|||
@Transactional(rollbackFor = RuntimeException.class)
|
||||
@Override
|
||||
public void delete(Integer id) {
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,id);
|
||||
transportTaskChildMapper.delete(queryWrapper);
|
||||
this.baseMapper.deleteById(id);
|
||||
//删除存储的贡献分析数据和时序分析数据
|
||||
redisUtil.del(CommonConstant.TRANSPORT_CONTRIBUTION_ANALYSIS+id);
|
||||
redisUtil.del(CommonConstant.TRANSPORT_TIMING_ANALYSIS+id);
|
||||
TransportTask transportTask = this.baseMapper.selectById(id);
|
||||
if(Objects.nonNull(transportTask)){
|
||||
if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(transportTask.getTaskMode())) {
|
||||
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,id);
|
||||
taskBackwardChildMapper.delete(queryWrapper);
|
||||
}
|
||||
this.baseMapper.deleteById(id);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -190,19 +288,51 @@ public class TransportTaskServiceImpl extends ServiceImpl<TransportTaskMapper,Tr
|
|||
if(Objects.isNull(checkIdResult)){
|
||||
throw new RuntimeException("此任务不存在");
|
||||
}
|
||||
LambdaQueryWrapper<TransportTaskChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskChild::getTaskId,id);
|
||||
queryWrapper.orderByAsc(TransportTaskChild::getId);
|
||||
List<TransportTaskChild> transportTaskChildren = transportTaskChildMapper.selectList(queryWrapper);
|
||||
if(Objects.isNull(transportTaskChildren)){
|
||||
throw new RuntimeException("此任务对应的flexpart模型配置信息不存在，请确认");
|
||||
if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(checkIdResult.getTaskMode())){
|
||||
LambdaQueryWrapper<TransportTaskBackwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskBackwardChild::getTaskId,id);
|
||||
queryWrapper.orderByAsc(TransportTaskBackwardChild::getId);
|
||||
List<TransportTaskBackwardChild> transportTaskChildren = taskBackwardChildMapper.selectList(queryWrapper);
|
||||
if(Objects.isNull(transportTaskChildren)){
|
||||
throw new RuntimeException("此任务对应的flexpart反演配置信息不存在，请确认");
|
||||
}
|
||||
AbstractTaskExec taskExec = new BackwardTaskExec();
|
||||
taskExec.init(weatherDataMapper,this,
|
||||
checkIdResult,transportTaskChildren,
|
||||
simulationProperties,systemStorageProperties,
|
||||
dataFusionProperties,serverProperties);
|
||||
taskExec.setName("大气输运反演任务执行线程");
|
||||
taskExec.start();
|
||||
}else if (TransportTaskModeEnum.FORWARD.getKey().equals(checkIdResult.getTaskMode())){
|
||||
//查询物种
|
||||
LambdaQueryWrapper<TransportTaskForwardSpecies> speciesQueryWrapper = new LambdaQueryWrapper<>();
|
||||
speciesQueryWrapper.eq(TransportTaskForwardSpecies::getTaskId,id);
|
||||
List<TransportTaskForwardSpecies> species = this.taskForwardSpeciesMapper.selectList(speciesQueryWrapper);
|
||||
if (CollUtil.isNotEmpty(species)) {
|
||||
checkIdResult.setSpecies(species);
|
||||
}
|
||||
//查询台站信息
|
||||
LambdaQueryWrapper<TransportTaskForwardChild> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(TransportTaskForwardChild::getTaskId,id);
|
||||
List<TransportTaskForwardChild> stationInfo = taskForwardChildMapper.selectList(queryWrapper);
|
||||
if (CollUtil.isNotEmpty(stationInfo)) {
|
||||
stationInfo.forEach(station -> {
|
||||
LambdaQueryWrapper<TransportTaskForwardRelease> releaseQueryWrapper = new LambdaQueryWrapper<>();
|
||||
releaseQueryWrapper.eq(TransportTaskForwardRelease::getForwardChildId,station.getId());
|
||||
releaseQueryWrapper.orderByAsc(TransportTaskForwardRelease::getStartTime);
|
||||
List<TransportTaskForwardRelease> releaseList = taskForwardReleaseMapper.selectList(releaseQueryWrapper);
|
||||
if (CollUtil.isNotEmpty(releaseList)) {
|
||||
station.setForwardReleaseChild(releaseList);
|
||||
}
|
||||
});
|
||||
checkIdResult.setForwardChild(stationInfo);
|
||||
}
|
||||
AbstractTaskExec taskExec = new ForwardTaskExec();
|
||||
taskExec.init(weatherDataMapper,this,checkIdResult,
|
||||
simulationProperties,systemStorageProperties,serverProperties);
|
||||
taskExec.setName("大气输运反演任务执行线程");
|
||||
taskExec.start();
|
||||
}
|
||||
TransportTaskExec taskExec = new TransportTaskExec();
|
||||
taskExec.init(weatherDataMapper,this,
|
||||
checkIdResult,transportTaskChildren,
|
||||
simulationProperties,systemStorageProperties);
|
||||
taskExec.setName("输运模拟执行线程");
|
||||
taskExec.start();
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -0,0 +1,219 @@
|
|||
package org.jeecg.task.flexparttask;
|
||||
|
||||
import cn.hutool.core.util.StrUtil;
|
||||
import com.jcraft.jsch.ChannelExec;
|
||||
import com.jcraft.jsch.JSch;
|
||||
import com.jcraft.jsch.JSchException;
|
||||
import com.jcraft.jsch.Session;
|
||||
import lombok.Setter;
|
||||
import org.apache.logging.log4j.util.Strings;
|
||||
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
|
||||
import org.jeecg.common.properties.DataFusionProperties;
|
||||
import org.jeecg.common.properties.ServerProperties;
|
||||
import org.jeecg.common.properties.SystemStorageProperties;
|
||||
import org.jeecg.common.properties.TransportSimulationProperties;
|
||||
import org.jeecg.modules.base.entity.TransportTask;
|
||||
import org.jeecg.modules.base.entity.TransportTaskBackwardChild;
|
||||
import org.jeecg.modules.base.mapper.WeatherDataMapper;
|
||||
import org.jeecg.service.TransportTaskService;
|
||||
|
||||
import java.io.*;
|
||||
import java.util.List;
|
||||
import java.util.Properties;
|
||||
|
||||
public abstract class AbstractTaskExec extends Thread{
|
||||
|
||||
protected WeatherDataMapper weatherDataMapper;
|
||||
protected TransportTaskService transportTaskService;
|
||||
protected TransportTask transportTask;
|
||||
protected List<TransportTaskBackwardChild> transportTaskChildren;
|
||||
protected TransportSimulationProperties simulationProperties;
|
||||
protected SystemStorageProperties systemStorageProperties;
|
||||
protected DataFusionProperties dataFusionProperties;
|
||||
protected ServerProperties serverProperties;
|
||||
|
||||
/**
|
||||
* 初始化
|
||||
*/
|
||||
public void init(WeatherDataMapper weatherDataMapper,
|
||||
TransportTaskService transportTaskService,
|
||||
TransportTask transportTask,
|
||||
TransportSimulationProperties simulationProperties,
|
||||
SystemStorageProperties systemStorageProperties,
|
||||
ServerProperties serverProperties){
|
||||
this.weatherDataMapper = weatherDataMapper;
|
||||
this.transportTaskService = transportTaskService;
|
||||
this.transportTask = transportTask;
|
||||
this.simulationProperties = simulationProperties;
|
||||
this.systemStorageProperties = systemStorageProperties;
|
||||
this.serverProperties = serverProperties;
|
||||
}
|
||||
|
||||
/**
|
||||
* 初始化
|
||||
*/
|
||||
public void init(WeatherDataMapper weatherDataMapper,
|
||||
TransportTaskService transportTaskService,
|
||||
TransportTask transportTask,
|
||||
List<TransportTaskBackwardChild> transportTaskChildren,
|
||||
TransportSimulationProperties simulationProperties,
|
||||
SystemStorageProperties systemStorageProperties,
|
||||
DataFusionProperties dataFusionProperties,
|
||||
ServerProperties serverProperties){
|
||||
this.weatherDataMapper = weatherDataMapper;
|
||||
this.transportTaskService = transportTaskService;
|
||||
this.transportTask = transportTask;
|
||||
this.simulationProperties = simulationProperties;
|
||||
this.systemStorageProperties = systemStorageProperties;
|
||||
this.transportTaskChildren = transportTaskChildren;
|
||||
this.dataFusionProperties = dataFusionProperties;
|
||||
this.serverProperties = serverProperties;
|
||||
}
|
||||
|
||||
public abstract void execute();
|
||||
|
||||
protected abstract void checkMetData();
|
||||
|
||||
protected abstract void execSimulation();
|
||||
|
||||
/**
|
||||
* 获取气象数据路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
protected String getMetDataPath(Integer dataSource){
|
||||
StringBuilder path = new StringBuilder();
|
||||
path.append(systemStorageProperties.getRootPath());
|
||||
path.append(File.separator);
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getPangu());
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getGraphcast());
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getCra40());
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getNcep());
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getFnl());
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getT1h());
|
||||
}
|
||||
return path.toString();
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取参数配置文件路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
protected String getParamConfigPath(Integer dataSource){
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
return simulationProperties.getPanguParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
return simulationProperties.getGraphcastParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
return simulationProperties.getCra40ParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
return simulationProperties.getNcepParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
return simulationProperties.getFnlParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
return simulationProperties.getT1hParamConfigPath();
|
||||
}
|
||||
return Strings.EMPTY;
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取台站参数配置文件路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
protected String getStationsConfigPath(Integer dataSource){
|
||||
StringBuilder path = new StringBuilder();
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
path.append(simulationProperties.getPanguStationsConfigPath());
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
path.append(simulationProperties.getCra40StationsConfigPath());
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
path.append(simulationProperties.getCra40StationsConfigPath());
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
path.append(simulationProperties.getNcepStationsConfigPath());
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
path.append(simulationProperties.getFnlStationsConfigPath());
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
path.append(simulationProperties.getT1hStationsConfigPath());
|
||||
}
|
||||
return path.toString();
|
||||
}
|
||||
|
||||
|
||||
protected class JSchRemoteRunner{
|
||||
|
||||
private JSch jsch = new JSch();
|
||||
|
||||
private Session session = null;
|
||||
|
||||
private ChannelExec channel = null;
|
||||
|
||||
@Setter
|
||||
private String command;
|
||||
|
||||
/**
|
||||
* 登录
|
||||
* @param host
|
||||
* @param port
|
||||
* @param username
|
||||
* @param password
|
||||
* @return
|
||||
* @throws JSchException
|
||||
*/
|
||||
protected Session login(String host,int port,String username,String password) throws JSchException {
|
||||
Properties config = new Properties();
|
||||
config.put("StrictHostKeyChecking","no");
|
||||
|
||||
session = jsch.getSession(username, host, port);
|
||||
session.setPassword(password);
|
||||
session.setConfig(config);
|
||||
session.connect();
|
||||
return session;
|
||||
}
|
||||
|
||||
/**
|
||||
* 持续读取日志
|
||||
*/
|
||||
protected void execCommand() throws JSchException, IOException {
|
||||
try{
|
||||
//打开一个执行通道
|
||||
channel = (ChannelExec) session.openChannel("exec");
|
||||
String fullCommand = command + " 2>&1";
|
||||
channel.setCommand(fullCommand);
|
||||
// 获取脚本的标准输出流，包含错误输出流
|
||||
InputStream in = channel.getInputStream();
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
|
||||
// 连接通道
|
||||
channel.connect();
|
||||
|
||||
String line;
|
||||
while ((line = reader.readLine()) != null) {
|
||||
if(StrUtil.isNotBlank(line)){
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(transportTask.getId(),line));
|
||||
}
|
||||
}
|
||||
// // 等待脚本执行完成
|
||||
// while (channel.isConnected()) {
|
||||
// Thread.sleep(1000);
|
||||
// }
|
||||
}catch(JSchException |IOException e){
|
||||
throw new RuntimeException(e);
|
||||
}finally {
|
||||
// 关闭资源
|
||||
if (channel != null) {
|
||||
channel.disconnect();
|
||||
}
|
||||
if (session != null) {
|
||||
session.disconnect();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,187 @@
|
|||
package org.jeecg.task.flexparttask;
|
||||
|
||||
import cn.hutool.core.collection.CollUtil;
|
||||
import cn.hutool.core.date.LocalDateTimeUtil;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
|
||||
import org.apache.commons.lang3.time.StopWatch;
|
||||
import org.apache.logging.log4j.util.Strings;
|
||||
import org.jeecg.common.constant.enums.FlexpartSpeciesType;
|
||||
import org.jeecg.common.constant.enums.TransportTaskStatusEnum;
|
||||
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
|
||||
import org.jeecg.modules.base.entity.*;
|
||||
import java.io.*;
|
||||
import java.math.BigDecimal;
|
||||
import java.math.RoundingMode;
|
||||
import java.text.DecimalFormat;
|
||||
import java.time.LocalDateTime;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.time.temporal.ChronoUnit;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
public class BackwardTaskExec extends AbstractTaskExec{
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
this.execute();
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行任务
|
||||
*/
|
||||
public void execute() {
|
||||
StopWatch stopWatch = new StopWatch();
|
||||
stopWatch.start();
|
||||
try{
|
||||
//修改任务状态为执行中
|
||||
super.transportTaskService.updateTaskStatus(super.transportTask.getId(), TransportTaskStatusEnum.IN_OPERATION.getValue());
|
||||
//如果此任务已存在历史日志，先清除
|
||||
super.transportTaskService.deleteTaskLog(super.transportTask.getId());
|
||||
//检查气象数据
|
||||
this.checkMetData();
|
||||
//执行模拟
|
||||
this.execSimulation();
|
||||
//生成SRS文件
|
||||
// this.generateSRSFile();
|
||||
}catch (Exception e){
|
||||
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),taskErrorLog));
|
||||
super.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
|
||||
throw e;
|
||||
}finally {
|
||||
//添加任务耗时
|
||||
stopWatch.stop();
|
||||
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
|
||||
double min = seconds/60D;
|
||||
BigDecimal bgMin = new BigDecimal(min);
|
||||
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
|
||||
super.transportTaskService.updateTaskStatusToCompleted(super.transportTask.getId(),result.doubleValue());
|
||||
String taskCompletedLog = "任务执行完成，耗时："+min+"分钟";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),taskCompletedLog));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查气象数据
|
||||
*/
|
||||
protected void checkMetData(){
|
||||
String msg = "检查气象数据";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),msg));
|
||||
|
||||
LocalDateTime startTime = super.transportTask.getStartTime();
|
||||
LocalDateTime endTime = super.transportTask.getEndTime();
|
||||
LambdaQueryWrapper<WeatherData> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(WeatherData::getDataSource,super.transportTask.getUseMetType());
|
||||
queryWrapper.between(WeatherData::getDataStartTime,startTime,endTime);
|
||||
List<WeatherData> dataList = super.weatherDataMapper.selectList(queryWrapper);
|
||||
long hours = ChronoUnit.HOURS.between(startTime, endTime)/6;
|
||||
//一天至少4个气象数据，至少是00,06,12,18
|
||||
if(CollUtil.isEmpty(dataList) || dataList.size() < hours){
|
||||
String exceptionMsg = "此任务时间范围(%s - %s)参数所涉及的气象数据不完整，请核对气象数据";
|
||||
String formatMsg = String.format(exceptionMsg, LocalDateTimeUtil.format(startTime, "yyyy-MM-dd hh:mm:ss"), LocalDateTimeUtil.format(endTime, "yyyy-MM-dd hh:mm:ss"));
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),formatMsg));
|
||||
throw new RuntimeException(formatMsg);
|
||||
}
|
||||
String finalMsg = "检查气象数据完毕";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),finalMsg));
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行模拟
|
||||
*/
|
||||
protected void execSimulation(){
|
||||
// Process process = null;
|
||||
try {
|
||||
String paramMsg = "生成flexpart所需参数文件：param.config,stations.config";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),paramMsg));
|
||||
//处理参数配置文件
|
||||
String paramConfigPath = super.getParamConfigPath(super.transportTask.getUseMetType());
|
||||
String metDataPath = super.getMetDataPath(super.transportTask.getUseMetType());
|
||||
if(!FileUtil.exist(paramConfigPath)){
|
||||
FileUtil.touch(paramConfigPath);
|
||||
}
|
||||
StringBuilder paramContent = new StringBuilder();
|
||||
paramContent.append(super.transportTask.getStartTime().format(DateTimeFormatter.ofPattern("yyyyMMdd"))).append("\n");
|
||||
paramContent.append(super.transportTask.getEndTime().format(DateTimeFormatter.ofPattern("yyyyMMdd"))).append("\n");
|
||||
paramContent.append(super.transportTask.getStartTime().format(DateTimeFormatter.ofPattern("HHmmss"))).append("\n");
|
||||
paramContent.append(super.transportTask.getEndTime().format(DateTimeFormatter.ofPattern("HHmmss"))).append("\n");
|
||||
paramContent.append(super.transportTask.getParticleCount()).append("\n");
|
||||
paramContent.append(super.transportTask.getZ1()).append("\n");
|
||||
paramContent.append(super.transportTask.getZ2()).append("\n");
|
||||
paramContent.append(metDataPath).append("\n");
|
||||
paramContent.append(super.simulationProperties.getOutputPath()+File.separator+super.transportTask.getTaskName()).append("\n");
|
||||
paramContent.append(FlexpartSpeciesType.NOT_SPECIES.getValue()).append("\n");//反演固定61，不显示具体核素，只显示Xe
|
||||
FileUtil.writeString(paramContent.toString(),paramConfigPath,"UTF-8");
|
||||
//处理台站数据文件
|
||||
List<String> stationConfigInfo = new ArrayList<>();
|
||||
String stationsConfigPath = super.getStationsConfigPath(super.transportTask.getUseMetType())+".backward";
|
||||
if(!FileUtil.exist(stationsConfigPath)){
|
||||
FileUtil.touch(stationsConfigPath);
|
||||
}
|
||||
super.transportTaskChildren.forEach(taskChild -> {
|
||||
String format = "%s,%f,%f,%s,%s,%s,%s,%s";
|
||||
BigDecimal srcReleaseAmount = new BigDecimal(taskChild.getReleaseAmount());
|
||||
BigDecimal constant = new BigDecimal("1000");
|
||||
DecimalFormat scientificFormat = new DecimalFormat("0.00E0");
|
||||
double value = srcReleaseAmount.divide(constant).doubleValue();
|
||||
String releaseAmount = scientificFormat.format(value);
|
||||
String row = String.format(format,taskChild.getStationCode(),taskChild.getLat(),taskChild.getLon(),releaseAmount,
|
||||
taskChild.getAcqStartTime().format(DateTimeFormatter.ofPattern("yyyyMMdd")),
|
||||
taskChild.getAcqEndTime().format(DateTimeFormatter.ofPattern("yyyyMMdd")),
|
||||
taskChild.getAcqStartTime().format(DateTimeFormatter.ofPattern("HHmmss")),
|
||||
taskChild.getAcqEndTime().format(DateTimeFormatter.ofPattern("HHmmss")));
|
||||
stationConfigInfo.add(row);
|
||||
});
|
||||
//最后一行需要换行，否则启动flexpart报错
|
||||
stationConfigInfo.add("\n");
|
||||
FileUtil.writeLines(stationConfigInfo,stationsConfigPath,"UTF-8");
|
||||
//获取反演脚本路径
|
||||
String scriptPath = this.getBackForwardScriptPath(super.transportTask.getUseMetType());
|
||||
String execScriptMsg = "执行任务脚本，开始模拟，路径为："+scriptPath;
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),execScriptMsg));
|
||||
//ssh连接宿主机调用flexpart
|
||||
JSchRemoteRunner jschRemoteRunner = new JSchRemoteRunner();
|
||||
jschRemoteRunner.setCommand(scriptPath);
|
||||
jschRemoteRunner.login(super.serverProperties.getHost(),super.serverProperties.getPort(),
|
||||
super.serverProperties.getUsername(),super.serverProperties.getPassword());
|
||||
jschRemoteRunner.execCommand();
|
||||
} catch (Exception e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取反演脚本路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
private String getBackForwardScriptPath(Integer dataSource){
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
return simulationProperties.getPanguBackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
return simulationProperties.getGraphcastBackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
return simulationProperties.getCra40BackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
return simulationProperties.getNcepBackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
return simulationProperties.getFnlBackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
return simulationProperties.getT1hBackwardScriptPath();
|
||||
}
|
||||
return Strings.EMPTY;
|
||||
}
|
||||
|
||||
/**
|
||||
* 生成SRS文件
|
||||
*/
|
||||
private void generateSRSFile(){
|
||||
for (TransportTaskBackwardChild transportTaskChild : this.transportTaskChildren){
|
||||
BuildNcToSrsFile ncToSrsFile = new BuildNcToSrsFile();
|
||||
ncToSrsFile.init(dataFusionProperties,simulationProperties,transportTask,transportTaskChild,transportTaskChild.getReleaseAmount());
|
||||
ncToSrsFile.execute();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,272 @@
|
|||
package org.jeecg.task.flexparttask;
|
||||
|
||||
import cn.hutool.core.date.DateUtil;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.logging.log4j.util.Strings;
|
||||
import org.jeecg.common.constant.enums.TransportSampleTypeEnum;
|
||||
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
|
||||
import org.jeecg.common.properties.DataFusionProperties;
|
||||
import org.jeecg.common.properties.TransportSimulationProperties;
|
||||
import org.jeecg.modules.base.entity.TransportTask;
|
||||
import org.jeecg.modules.base.entity.TransportTaskBackwardChild;
|
||||
import ucar.ma2.Array;
|
||||
import ucar.ma2.DataType;
|
||||
import ucar.ma2.InvalidRangeException;
|
||||
import ucar.nc2.Attribute;
|
||||
import ucar.nc2.NetcdfFile;
|
||||
import ucar.nc2.Variable;
|
||||
|
||||
import java.io.File;
|
||||
import java.io.FileOutputStream;
|
||||
import java.io.IOException;
|
||||
import java.math.BigDecimal;
|
||||
import java.math.RoundingMode;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.Paths;
|
||||
import java.text.DecimalFormat;
|
||||
import java.time.LocalDateTime;
|
||||
import java.time.ZoneId;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.zip.GZIPOutputStream;
|
||||
|
||||
/**
|
||||
* 根据NC文件构建SRS文件
|
||||
*/
|
||||
@Slf4j
|
||||
public class BuildNcToSrsFile {
|
||||
|
||||
private final static String BACK_FORWARD="backward";
|
||||
private DataFusionProperties dataFusionProperties;
|
||||
private TransportSimulationProperties simulationProperties;
|
||||
private TransportTask transportTask;
|
||||
private TransportTaskBackwardChild transportTaskChild;
|
||||
private final int radius=6371*1000;
|
||||
private final double pi=3.14;
|
||||
private double factorMul;
|
||||
|
||||
public void init(DataFusionProperties dataFusionProperties,
|
||||
TransportSimulationProperties simulationProperties,
|
||||
TransportTask transportTask,
|
||||
TransportTaskBackwardChild transportTaskChild,
|
||||
String releaseAmount) {
|
||||
this.dataFusionProperties = dataFusionProperties;
|
||||
this.simulationProperties = simulationProperties;
|
||||
this.transportTask = transportTask;
|
||||
this.transportTaskChild = transportTaskChild;
|
||||
this.factorMul = Double.parseDouble(releaseAmount);
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行转换任务
|
||||
*/
|
||||
public void execute(){
|
||||
String ncFilePath = this.getBackForwardTaskNCPath();
|
||||
if(!FileUtil.exist(ncFilePath)){
|
||||
throw new RuntimeException("此任务模拟结果不存在，请确认任务运行状态");
|
||||
}
|
||||
try (NetcdfFile ncFile = NetcdfFile.open(ncFilePath.toString())) {
|
||||
String header = this.buildHeader(ncFile);
|
||||
List<String> body = buildBody(ncFile);
|
||||
body.add(0,header);
|
||||
File file = new File(this.getSrsStoragePath());
|
||||
if(!file.exists()){
|
||||
if(!file.getParentFile().exists()){
|
||||
file.getParentFile().mkdirs();
|
||||
}
|
||||
file.createNewFile();
|
||||
}
|
||||
Path outputPath = Paths.get(file.getParent(), file.getName());
|
||||
this.compressContentToFile(body,outputPath);
|
||||
log.info("SRS文件生成完毕，路径为：{}", file.getAbsolutePath());
|
||||
} catch (IOException | InvalidRangeException e) {
|
||||
String errLog = "SRS文件生成失败，原因："+e.getMessage();
|
||||
log.error(errLog,e);
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
|
||||
private String buildHeader(NetcdfFile ncFile){
|
||||
//处理台站经纬度
|
||||
BigDecimal lonBigDecimal = new BigDecimal(this.transportTaskChild.getLon());
|
||||
BigDecimal latBigDecimal = new BigDecimal(this.transportTaskChild.getLat());
|
||||
Double lon = lonBigDecimal.setScale(2, RoundingMode.HALF_UP).doubleValue();
|
||||
Double lat = latBigDecimal.setScale(2, RoundingMode.HALF_UP).doubleValue();
|
||||
//处理测量开始和结束时间参数
|
||||
LocalDateTime acqStartTime = transportTaskChild.getAcqStartTime();
|
||||
LocalDateTime acqEndTime = transportTaskChild.getAcqEndTime();
|
||||
String acqStartTimeStr = acqStartTime.format(DateTimeFormatter.ofPattern("yyyyMMdd"));
|
||||
String acqEndTimeStr = acqEndTime.format(DateTimeFormatter.ofPattern("yyyyMMdd"));
|
||||
//处理模拟总时长
|
||||
LocalDateTime taskStartTime = transportTask.getStartTime();
|
||||
LocalDateTime taskEndTime = transportTask.getEndTime();
|
||||
long startHour = taskStartTime.atZone(ZoneId.of("Asia/Shanghai")).getHour();
|
||||
long endHour = taskEndTime.atZone(ZoneId.of("Asia/Shanghai")).getHour();
|
||||
long totalHour = endHour - startHour;
|
||||
//处理输出频率[小时] 和 平均时间[小时]参数
|
||||
Attribute loutstep = ncFile.findGlobalAttribute("loutstep");
|
||||
Attribute loutaver = ncFile.findGlobalAttribute("loutaver");
|
||||
int outputFreHour = Math.abs(loutstep.getNumericValue().intValue())/3600;
|
||||
int averageTime = Math.abs(loutaver.getNumericValue().intValue())/3600;
|
||||
//获取网格精度
|
||||
Attribute dxoutVar = ncFile.findGlobalAttribute("dxout");
|
||||
Attribute dyoutVar = ncFile.findGlobalAttribute("dyout");
|
||||
String header = String.format(" %s %s %s %s %s %s %s %s %s %s %s %s %s",
|
||||
lon,lat,acqStartTimeStr,acqStartTime.getHour(),acqEndTimeStr,acqEndTime.getHour(),
|
||||
this.transportTaskChild.getReleaseAmount(),totalHour,outputFreHour,averageTime,
|
||||
dxoutVar.getNumericValue().doubleValue(),dyoutVar.getNumericValue().doubleValue(),
|
||||
this.transportTaskChild.getStationCode());
|
||||
return header;
|
||||
}
|
||||
|
||||
/**
|
||||
* 计算body数据
|
||||
* @param ncFile
|
||||
* @return
|
||||
* @throws InvalidRangeException
|
||||
* @throws IOException
|
||||
*/
|
||||
private List<String> buildBody(NetcdfFile ncFile) throws InvalidRangeException, IOException {
|
||||
//获取时间序号
|
||||
Variable time = ncFile.findVariable("time");
|
||||
int timeLen = time.getShape()[0];
|
||||
//获取经纬度网格数量
|
||||
Variable longitude = ncFile.findVariable("longitude");
|
||||
Variable latitude = ncFile.findVariable("latitude");
|
||||
int lonLen = longitude.getShape()[0];
|
||||
int latLen = latitude.getShape()[0];
|
||||
Array latArray = latitude.read();
|
||||
Array lonArray = longitude.read();
|
||||
double[] lats = (double[]) latArray.get1DJavaArray(DataType.DOUBLE);
|
||||
double[] lons = (double[]) lonArray.get1DJavaArray(DataType.DOUBLE);
|
||||
//获取网格精度
|
||||
Attribute dxoutVar = ncFile.findGlobalAttribute("dxout");
|
||||
Attribute dyoutVar = ncFile.findGlobalAttribute("dyout");
|
||||
double dxout = dxoutVar.getNumericValue().doubleValue();
|
||||
double dyout = dyoutVar.getNumericValue().doubleValue();
|
||||
//处理输出频率[小时]
|
||||
Attribute loutstep = ncFile.findGlobalAttribute("loutstep");
|
||||
double outputFreHour = loutstep.getNumericValue().doubleValue();
|
||||
|
||||
List<String> body = new ArrayList<>();
|
||||
Variable spec001Mr = ncFile.findVariable("spec001_mr");
|
||||
for (int t=0;t<timeLen;t++){
|
||||
int[] origin = {0, 0,t,0, 0, 0};
|
||||
int[] section = {1, 1,1,1,latLen,lonLen};
|
||||
Array data = spec001Mr.read(origin,section);
|
||||
double[] pointData = (double[]) data.get1DJavaArray(DataType.DOUBLE);
|
||||
for (int y=0;y<latLen;y++){
|
||||
for (int x=0;x<lonLen;x++){
|
||||
int index = y * lonLen + x;
|
||||
double pointVal = pointData[index];
|
||||
if (pointVal>0){
|
||||
double currentLat = lats[y];
|
||||
double currentLon = lons[x];
|
||||
double dx_1d=cal_dx_1d(currentLat);
|
||||
double dy_1d = radius * pi / 180;
|
||||
double area=dyout * dxout * dy_1d * dx_1d;
|
||||
double volume=area*100;//height = 100 米（固定的高度层厚度）
|
||||
double factor = pointVal / (outputFreHour * 3600) / volume * factorMul;
|
||||
DecimalFormat scientificFormat = new DecimalFormat("0.0000000E0");
|
||||
String result = scientificFormat.format(factor);
|
||||
String line = String.format(" %-7s %8s %-4s %s",currentLat,currentLon,t+1,result);
|
||||
body.add(line);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取反演NC结果文件路径
|
||||
* @return
|
||||
*/
|
||||
private String getBackForwardTaskNCPath(){
|
||||
//拼接nc文件路径
|
||||
StringBuilder path = new StringBuilder();
|
||||
path.append(simulationProperties.getOutputPath());
|
||||
path.append(File.separator);
|
||||
path.append(this.transportTask.getTaskName());
|
||||
path.append(File.separator);
|
||||
path.append(BACK_FORWARD);
|
||||
path.append(File.separator);
|
||||
path.append(this.transportTaskChild.getStationCode());
|
||||
path.append(File.separator);
|
||||
path.append("grid_time_").append(DateUtil.format(transportTask.getEndTime(), "yyyyMMddHHmmss")).append(".nc");
|
||||
return path.toString();
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取SRS文件存储路径
|
||||
* @return
|
||||
*/
|
||||
private String getSrsStoragePath(){
|
||||
StringBuilder path = new StringBuilder();
|
||||
path.append(dataFusionProperties.getNdcSRSPath());
|
||||
path.append(File.separator);
|
||||
path.append(transportTaskChild.getAcqEndTime().format(DateTimeFormatter.ofPattern("yyyyMMdd")));
|
||||
path.append(File.separator);
|
||||
if(TransportSampleTypeEnum.B.name().equals(transportTaskChild.getSampleType())){
|
||||
path.append("flexpart.x."+this.getMetType()+".l1");
|
||||
}else if(TransportSampleTypeEnum.P.name().equals(transportTaskChild.getSampleType())){
|
||||
path.append("flexpart."+this.getMetType()+".l1");
|
||||
}
|
||||
path.append(File.separator);
|
||||
path.append(transportTaskChild.getStationCode()+".fp.");
|
||||
path.append(transportTaskChild.getAcqEndTime().format(DateTimeFormatter.ofPattern("yyyyMMddHH")));
|
||||
path.append(".f9.srm.gz");
|
||||
return path.toString();
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取气象类型
|
||||
* @return
|
||||
*/
|
||||
private String getMetType(){
|
||||
if (WeatherDataSourceEnum.PANGU.getKey().equals(transportTask.getUseMetType())){
|
||||
return WeatherDataSourceEnum.PANGU.name().toLowerCase();
|
||||
}else if (WeatherDataSourceEnum.GRAPHCAST.getKey().equals(transportTask.getUseMetType())){
|
||||
return WeatherDataSourceEnum.GRAPHCAST.name().toLowerCase();
|
||||
}else if (WeatherDataSourceEnum.CRA40.getKey().equals(transportTask.getUseMetType())){
|
||||
return WeatherDataSourceEnum.CRA40.name().toLowerCase();
|
||||
}else if (WeatherDataSourceEnum.NCEP.getKey().equals(transportTask.getUseMetType())){
|
||||
return WeatherDataSourceEnum.NCEP.name().toLowerCase();
|
||||
}else if (WeatherDataSourceEnum.FNL.getKey().equals(transportTask.getUseMetType())){
|
||||
return WeatherDataSourceEnum.FNL.name().toLowerCase();
|
||||
}else if (WeatherDataSourceEnum.T1H.getKey().equals(transportTask.getUseMetType())){
|
||||
return WeatherDataSourceEnum.T1H.name().toLowerCase();
|
||||
}
|
||||
return Strings.EMPTY;
|
||||
}
|
||||
|
||||
/**
|
||||
* 计算经度方向的实际距离（米）
|
||||
* 考虑地球曲率效应，经度1°的距离随纬度变化
|
||||
* @param lat_input
|
||||
* @return
|
||||
*/
|
||||
private Double cal_dx_1d(Double lat_input){
|
||||
return radius*2*pi*Math.cos(lat_input/180*pi)/360;
|
||||
}
|
||||
|
||||
private void compressContentToFile(List<String> contents, Path outputFile) throws IOException {
|
||||
try (FileOutputStream fileOut = new FileOutputStream(outputFile.toFile());
|
||||
GZIPOutputStream gzipOut = new GZIPOutputStream(fileOut)) {
|
||||
|
||||
for (int i = 0; i < contents.size(); i++) {
|
||||
String str = contents.get(i);
|
||||
|
||||
byte[] bytes = str.getBytes(StandardCharsets.UTF_8);
|
||||
gzipOut.write(bytes);
|
||||
|
||||
if (i < contents.size() - 1) {
|
||||
gzipOut.write('\n');
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,222 @@
|
|||
package org.jeecg.task.flexparttask;
|
||||
|
||||
import cn.hutool.core.collection.CollUtil;
|
||||
import cn.hutool.core.date.LocalDateTimeUtil;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
|
||||
import org.apache.commons.lang3.time.StopWatch;
|
||||
import org.apache.logging.log4j.util.Strings;
|
||||
import org.jeecg.common.constant.enums.TransportTaskStatusEnum;
|
||||
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
|
||||
import org.jeecg.modules.base.entity.*;
|
||||
import java.io.File;
|
||||
import java.math.BigDecimal;
|
||||
import java.math.RoundingMode;
|
||||
import java.text.DecimalFormat;
|
||||
import java.time.LocalDateTime;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.time.temporal.ChronoUnit;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
public class ForwardTaskExec extends AbstractTaskExec{
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
this.execute();
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行任务
|
||||
*/
|
||||
public void execute() {
|
||||
StopWatch stopWatch = new StopWatch();
|
||||
stopWatch.start();
|
||||
try{
|
||||
//修改任务状态为执行中
|
||||
super.transportTaskService.updateTaskStatus(super.transportTask.getId(), TransportTaskStatusEnum.IN_OPERATION.getValue());
|
||||
//如果此任务已存在历史日志，先清除
|
||||
super.transportTaskService.deleteTaskLog(super.transportTask.getId());
|
||||
//检查气象数据
|
||||
this.checkMetData();
|
||||
//执行模拟
|
||||
this.execSimulation();
|
||||
}catch (Exception e){
|
||||
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),taskErrorLog));
|
||||
this.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
|
||||
throw e;
|
||||
}finally {
|
||||
//添加任务耗时
|
||||
stopWatch.stop();
|
||||
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
|
||||
double min = seconds/60D;
|
||||
BigDecimal bgMin = new BigDecimal(min);
|
||||
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
|
||||
super.transportTaskService.updateTaskStatusToCompleted(super.transportTask.getId(),result.doubleValue());
|
||||
String taskCompletedLog = "任务执行完成，耗时："+min+"分钟";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),taskCompletedLog));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查气象数据
|
||||
*/
|
||||
protected void checkMetData(){
|
||||
String msg = "检查气象数据";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),msg));
|
||||
|
||||
LocalDateTime startTime = super.transportTask.getStartTime();
|
||||
LocalDateTime endTime = super.transportTask.getEndTime();
|
||||
LambdaQueryWrapper<WeatherData> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(WeatherData::getDataSource,super.transportTask.getUseMetType());
|
||||
queryWrapper.between(WeatherData::getDataStartTime,startTime,endTime);
|
||||
List<WeatherData> dataList = super.weatherDataMapper.selectList(queryWrapper);
|
||||
long hours = ChronoUnit.HOURS.between(startTime, endTime)/6;
|
||||
//一天至少4个气象数据，至少是00,06,12,18
|
||||
if(CollUtil.isEmpty(dataList) || dataList.size() < hours){
|
||||
String exceptionMsg = "此任务时间范围(%s - %s)参数所涉及的气象数据不完整，请核对气象数据";
|
||||
String formatMsg = String.format(exceptionMsg, LocalDateTimeUtil.format(startTime, "yyyy-MM-dd hh:mm:ss"), LocalDateTimeUtil.format(endTime, "yyyy-MM-dd hh:mm:ss"));
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),formatMsg));
|
||||
throw new RuntimeException(formatMsg);
|
||||
}
|
||||
String finalMsg = "检查气象数据完毕";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),finalMsg));
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行模拟
|
||||
*/
|
||||
protected void execSimulation(){
|
||||
Process process = null;
|
||||
try {
|
||||
String paramMsg = "生成flexpart所需配置文件：param.config,stations.config,input_site_hour";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),paramMsg));
|
||||
//处理参数配置文件-param.config
|
||||
String paramConfigPath = super.getParamConfigPath(super.transportTask.getUseMetType());
|
||||
String metDataPath = super.getMetDataPath(super.transportTask.getUseMetType());
|
||||
if(!FileUtil.exist(paramConfigPath)){
|
||||
FileUtil.touch(paramConfigPath);
|
||||
}
|
||||
StringBuilder paramContent = new StringBuilder();
|
||||
paramContent.append(super.transportTask.getStartTime().format(DateTimeFormatter.ofPattern("yyyyMMdd"))).append("\n");
|
||||
paramContent.append(super.transportTask.getEndTime().format(DateTimeFormatter.ofPattern("yyyyMMdd"))).append("\n");
|
||||
paramContent.append(super.transportTask.getStartTime().format(DateTimeFormatter.ofPattern("HHmmss"))).append("\n");
|
||||
paramContent.append(super.transportTask.getEndTime().format(DateTimeFormatter.ofPattern("HHmmss"))).append("\n");
|
||||
paramContent.append(super.transportTask.getParticleCount()).append("\n");
|
||||
paramContent.append(super.transportTask.getZ1()).append("\n");
|
||||
paramContent.append(super.transportTask.getZ2()).append("\n");
|
||||
paramContent.append(metDataPath).append("\n");
|
||||
paramContent.append(super.simulationProperties.getOutputPath()+File.separator+super.transportTask.getTaskName()).append("\n");
|
||||
|
||||
//配置物种信息
|
||||
List<TransportTaskForwardSpecies> species = super.transportTask.getSpecies();
|
||||
if(CollUtil.isEmpty(species)){
|
||||
String speciesMissingLog = super.transportTask.getTaskName()+"任务物种信息不存在，任务停止";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),speciesMissingLog));
|
||||
super.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
|
||||
return;
|
||||
}
|
||||
for(int i=0;i<species.size();i++){
|
||||
paramContent.append(species.get(i).getSpeciesId());
|
||||
if (i != species.size()-1){
|
||||
paramContent.append(",");
|
||||
}else {
|
||||
paramContent.append("\n");
|
||||
}
|
||||
}
|
||||
FileUtil.writeString(paramContent.toString(),paramConfigPath,"UTF-8");
|
||||
|
||||
//处理台站数据文件-stations.config
|
||||
if (CollUtil.isEmpty(super.transportTask.getForwardChild())){
|
||||
String stationsMissingLog = super.transportTask.getTaskName()+"任务站点信息不存在，任务停止";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),stationsMissingLog));
|
||||
super.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
|
||||
return;
|
||||
}
|
||||
List<String> stationConfigInfo = new ArrayList<>();
|
||||
String stationsConfigPath = super.getStationsConfigPath(super.transportTask.getUseMetType());
|
||||
if(!FileUtil.exist(stationsConfigPath)){
|
||||
FileUtil.touch(stationsConfigPath);
|
||||
}
|
||||
super.transportTask.getForwardChild().forEach(station ->{
|
||||
String format = "%s,%f,%f,%s";
|
||||
//最后0是以前版本的总释放量，改版后无用，给个默认值
|
||||
String row = String.format(format,station.getStationCode(),station.getLat(),station.getLon(),"0");
|
||||
stationConfigInfo.add(row);
|
||||
});
|
||||
//最后一行需要换行，否则启动flexpart报错
|
||||
stationConfigInfo.add("\n");
|
||||
FileUtil.writeLines(stationConfigInfo,stationsConfigPath,"UTF-8");
|
||||
|
||||
//配置各站点排放数据
|
||||
//如果input_site_hour目录不存在先创建
|
||||
if(!FileUtil.exist(super.simulationProperties.getInputSiteHourPath())){
|
||||
FileUtil.mkdir(super.simulationProperties.getInputSiteHourPath());
|
||||
}else {
|
||||
FileUtil.clean(super.simulationProperties.getInputSiteHourPath());
|
||||
}
|
||||
Map<String,List<TransportTaskForwardRelease>> releaseInfoMap = new HashMap<>();
|
||||
for (TransportTaskForwardChild station : super.transportTask.getForwardChild()){
|
||||
if (CollUtil.isEmpty(station.getForwardReleaseChild())){
|
||||
String releaseMissingLog = station.getStationCode()+"站点释放信息缺失，任务停止";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),releaseMissingLog));
|
||||
super.transportTaskService.updateTaskStatus(super.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
|
||||
return;
|
||||
}
|
||||
releaseInfoMap.put(station.getStationCode(),station.getForwardReleaseChild());
|
||||
}
|
||||
releaseInfoMap .forEach((stationCode,releaseInfoList)->{
|
||||
String stationReleaseFile = super.simulationProperties.getInputSiteHourPath() + File.separator + stationCode + ".txt";
|
||||
FileUtil.touch(stationReleaseFile);
|
||||
List<String> stationReleaseInfo = new ArrayList<>();
|
||||
releaseInfoList.forEach(releaseInfo ->{
|
||||
String format = "%s,%s,%s";
|
||||
String startTime = releaseInfo.getStartTime().format(DateTimeFormatter.ofPattern("yyyyMMddHH"));
|
||||
String endTime = releaseInfo.getEndTime().format(DateTimeFormatter.ofPattern("yyyyMMddHH"));
|
||||
BigDecimal releaseAmount = new BigDecimal(releaseInfo.getReleaseAmount());
|
||||
BigDecimal constant = new BigDecimal("1000");
|
||||
DecimalFormat scientificFormat = new DecimalFormat("0.00E0");
|
||||
double value = releaseAmount.divide(constant).doubleValue();
|
||||
String releaseAmountFormat = scientificFormat.format(value);
|
||||
String row = String.format(format, startTime, endTime, releaseAmountFormat);
|
||||
stationReleaseInfo.add(row);
|
||||
});
|
||||
FileUtil.writeLines(stationReleaseInfo,stationReleaseFile,"UTF-8");
|
||||
});
|
||||
//获取正演脚本路径
|
||||
String scriptPath = this.getForwardScriptPath(super.transportTask.getUseMetType());
|
||||
String execScriptMsg = "执行任务脚本，开始模拟，路径为："+scriptPath;
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(super.transportTask.getId(),execScriptMsg));
|
||||
//ssh连接宿主机调用flexpart
|
||||
JSchRemoteRunner jschRemoteRunner = new JSchRemoteRunner();
|
||||
jschRemoteRunner.setCommand(scriptPath);
|
||||
jschRemoteRunner.login(super.serverProperties.getHost(),super.serverProperties.getPort(),
|
||||
super.serverProperties.getUsername(),super.serverProperties.getPassword());
|
||||
jschRemoteRunner.execCommand();
|
||||
} catch (Exception e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
/**
|
||||
* 获取正演脚本路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
private String getForwardScriptPath(Integer dataSource){
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
return simulationProperties.getPanguForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
return simulationProperties.getGraphcastForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
return simulationProperties.getCra40ForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
return simulationProperties.getNcepForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
return simulationProperties.getFnlForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
return simulationProperties.getT1hForwardScriptPath();
|
||||
}
|
||||
return Strings.EMPTY;
|
||||
}
|
||||
}
|
||||
|
|
@ -1,307 +0,0 @@
|
|||
package org.jeecg.task.flexparttask;
|
||||
|
||||
import cn.hutool.core.date.DateUtil;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import cn.hutool.core.util.StrUtil;
|
||||
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
|
||||
import org.apache.commons.lang3.time.StopWatch;
|
||||
import org.apache.logging.log4j.util.Strings;
|
||||
import org.jeecg.common.constant.enums.TransportTaskModeEnum;
|
||||
import org.jeecg.common.constant.enums.TransportTaskStatusEnum;
|
||||
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
|
||||
import org.jeecg.common.properties.SystemStorageProperties;
|
||||
import org.jeecg.common.properties.TransportSimulationProperties;
|
||||
import org.jeecg.modules.base.entity.*;
|
||||
import org.jeecg.modules.base.mapper.WeatherDataMapper;
|
||||
import org.jeecg.service.TransportTaskService;
|
||||
import java.io.*;
|
||||
import java.math.BigDecimal;
|
||||
import java.math.RoundingMode;
|
||||
import java.time.LocalDateTime;
|
||||
import java.time.ZoneId;
|
||||
import java.time.temporal.ChronoUnit;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
public class TransportTaskExec extends Thread{
|
||||
|
||||
private WeatherDataMapper weatherDataMapper;
|
||||
private TransportTaskService transportTaskService;
|
||||
private TransportTask transportTask;
|
||||
private List<TransportTaskChild> transportTaskChildren;
|
||||
private TransportSimulationProperties simulationProperties;
|
||||
private SystemStorageProperties systemStorageProperties;
|
||||
|
||||
/**
|
||||
* 初始化
|
||||
*/
|
||||
public void init(WeatherDataMapper weatherDataMapper,
|
||||
TransportTaskService transportTaskService,
|
||||
TransportTask transportTask,
|
||||
List<TransportTaskChild> transportTaskChildren,
|
||||
TransportSimulationProperties simulationProperties,
|
||||
SystemStorageProperties systemStorageProperties){
|
||||
this.weatherDataMapper = weatherDataMapper;
|
||||
this.transportTaskService = transportTaskService;
|
||||
this.transportTask = transportTask;
|
||||
this.transportTaskChildren = transportTaskChildren;
|
||||
this.simulationProperties = simulationProperties;
|
||||
this.systemStorageProperties = systemStorageProperties;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
this.execute();
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行任务
|
||||
*/
|
||||
public void execute() {
|
||||
StopWatch stopWatch = new StopWatch();
|
||||
stopWatch.start();
|
||||
try{
|
||||
//修改任务状态为执行中
|
||||
this.transportTaskService.updateTaskStatus(this.transportTask.getId(), TransportTaskStatusEnum.IN_OPERATION.getValue());
|
||||
//如果此任务已存在历史日志，先清除
|
||||
this.transportTaskService.deleteTaskLog(this.transportTask.getId());
|
||||
//检查气象数据
|
||||
this.checkMetData();
|
||||
//执行模拟
|
||||
this.execSimulation();
|
||||
}catch (Exception e){
|
||||
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),taskErrorLog));
|
||||
this.transportTaskService.updateTaskStatus(this.transportTask.getId(),TransportTaskStatusEnum.FAILURE.getValue());
|
||||
throw e;
|
||||
}finally {
|
||||
//添加任务耗时
|
||||
stopWatch.stop();
|
||||
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
|
||||
double min = seconds/60D;
|
||||
BigDecimal bgMin = new BigDecimal(min);
|
||||
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
|
||||
this.transportTaskService.updateTaskStatusToCompleted(this.transportTask.getId(),result.doubleValue());
|
||||
String taskCompletedLog = "任务执行完成，耗时："+min+"分钟";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),taskCompletedLog));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查气象数据
|
||||
*/
|
||||
private void checkMetData(){
|
||||
String msg = "检查气象数据";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),msg));
|
||||
|
||||
LocalDateTime startTime = this.transportTask.getStartTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();
|
||||
LocalDateTime endTime = this.transportTask.getEndTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();
|
||||
startTime = startTime.minusDays(3);
|
||||
endTime = endTime.plusDays(3);
|
||||
LambdaQueryWrapper<WeatherData> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(WeatherData::getDataSource,this.transportTask.getUseMetType());
|
||||
queryWrapper.between(WeatherData::getDataStartTime,startTime,endTime);
|
||||
List<WeatherData> dataList = weatherDataMapper.selectList(queryWrapper);
|
||||
long days = ChronoUnit.DAYS.between(startTime, endTime);
|
||||
//一天至少4个气象数据，至少是00,06,12,18
|
||||
// if(CollUtil.isEmpty(dataList) || dataList.size() < (days*4)){
|
||||
// String exceptionMsg = "此任务时间范围(%s - %s)参数所涉及的气象数据不完整，请核对气象数据";
|
||||
// String formatMsg = String.format(exceptionMsg, LocalDateTimeUtil.format(startTime, "yyyy-MM-dd hh:mm:ss"), LocalDateTimeUtil.format(endTime, "yyyy-MM-dd hh:mm:ss"));
|
||||
// ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),formatMsg));
|
||||
// throw new RuntimeException(formatMsg);
|
||||
// }
|
||||
String finalMsg = "检查气象数据完毕";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),finalMsg));
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行模拟
|
||||
*/
|
||||
private void execSimulation(){
|
||||
Process process = null;
|
||||
try {
|
||||
String paramMsg = "生成flexpart所需参数文件：param.config,stations.config";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),paramMsg));
|
||||
//处理参数配置文件
|
||||
String paramConfigPath = this.getParamConfigPath(this.transportTask.getUseMetType());
|
||||
String metDataPath = this.getMetDataPath(this.transportTask.getUseMetType());
|
||||
if(!FileUtil.exist(paramConfigPath)){
|
||||
FileUtil.touch(paramConfigPath);
|
||||
}
|
||||
StringBuilder paramContent = new StringBuilder();
|
||||
paramContent.append(DateUtil.format(this.transportTask.getStartTime(),"yyyyMMdd")).append("\n");
|
||||
paramContent.append(DateUtil.format(this.transportTask.getEndTime(),"yyyyMMdd")).append("\n");
|
||||
paramContent.append(DateUtil.format(this.transportTask.getStartTime(),"HHmmss")).append("\n");
|
||||
paramContent.append(DateUtil.format(this.transportTask.getEndTime(),"HHmmss")).append("\n");
|
||||
paramContent.append(this.transportTask.getParticleCount()).append("\n");
|
||||
paramContent.append(this.transportTask.getZ1()).append("\n");
|
||||
paramContent.append(this.transportTask.getZ2()).append("\n");
|
||||
paramContent.append(metDataPath).append("\n");
|
||||
paramContent.append(this.simulationProperties.getOutputPath()+File.separator+this.transportTask.getTaskName()).append("\n");
|
||||
if(TransportTaskModeEnum.FORWARD.getKey().equals(this.transportTask.getTaskMode())){
|
||||
paramContent.append(54).append("\n");//物种先固定写54对应XE135
|
||||
} else if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(this.transportTask.getTaskMode())) {
|
||||
paramContent.append(64).append("\n");//物种先固定对应XE135
|
||||
}
|
||||
FileUtil.writeString(paramContent.toString(),paramConfigPath,"UTF-8");
|
||||
//处理台站数据文件
|
||||
List<String> stationConfigInfo = new ArrayList<>();
|
||||
String stationsConfigPath = this.getStationsConfigPath(this.transportTask.getUseMetType());
|
||||
if(!FileUtil.exist(stationsConfigPath)){
|
||||
FileUtil.touch(stationsConfigPath);
|
||||
}
|
||||
this.transportTaskChildren.forEach(taskChild -> {
|
||||
String format = "%s,%f,%f,%s";
|
||||
String row = String.format(format,taskChild.getStationCode(),taskChild.getLat(),taskChild.getLon(),taskChild.getReleaseAmount());
|
||||
stationConfigInfo.add(row);
|
||||
});
|
||||
//最后一行需要换行，否则启动flexpart报错
|
||||
stationConfigInfo.add("\n");
|
||||
FileUtil.writeLines(stationConfigInfo,stationsConfigPath,"UTF-8");
|
||||
//获取脚本路径
|
||||
String scriptPath = null;
|
||||
if(TransportTaskModeEnum.FORWARD.getKey().equals(this.transportTask.getTaskMode())){
|
||||
scriptPath = this.getForwardScriptPath(this.transportTask.getUseMetType());
|
||||
} else if (TransportTaskModeEnum.BACK_FORWARD.getKey().equals(this.transportTask.getTaskMode())) {
|
||||
scriptPath = this.getBackForwardScriptPath(this.transportTask.getUseMetType());
|
||||
}
|
||||
String execScriptMsg = "执行任务脚本，开始模拟，路径为："+scriptPath;
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),execScriptMsg));
|
||||
ProcessBuilder processBuilder = new ProcessBuilder(scriptPath);
|
||||
processBuilder.directory(new File(new File(scriptPath).getParent()));
|
||||
processBuilder.redirectErrorStream(true);
|
||||
process = processBuilder.start();
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream(), "UTF-8"));
|
||||
//读取输出日志
|
||||
String line;
|
||||
while ((line = reader.readLine()) != null) {
|
||||
if(StrUtil.isNotBlank(line)){
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.transportTask.getId(),line));
|
||||
}
|
||||
}
|
||||
//等待进程结束
|
||||
process.waitFor();
|
||||
} catch (Exception e) {
|
||||
throw new RuntimeException(e);
|
||||
}finally {
|
||||
if(Objects.nonNull(process)){
|
||||
process.destroy();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取气象数据路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
private String getMetDataPath(Integer dataSource){
|
||||
StringBuilder path = new StringBuilder();
|
||||
path.append(systemStorageProperties.getRootPath());
|
||||
path.append(File.separator);
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getPangu());
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getGraphcast());
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getCra40());
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getNcep());
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getFnl());
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
path.append(systemStorageProperties.getT1h());
|
||||
}
|
||||
return path.toString();
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取参数配置文件路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
private String getParamConfigPath(Integer dataSource){
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
return simulationProperties.getPanguParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
return simulationProperties.getGraphcastParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
return simulationProperties.getCra40ParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
return simulationProperties.getNcepParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
return simulationProperties.getFnlParamConfigPath();
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
return simulationProperties.getT1hParamConfigPath();
|
||||
}
|
||||
return Strings.EMPTY;
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取台站参数配置文件路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
private String getStationsConfigPath(Integer dataSource){
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
return simulationProperties.getPanguStationsConfigPath();
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
return simulationProperties.getCra40StationsConfigPath();
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
return simulationProperties.getCra40StationsConfigPath();
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
return simulationProperties.getNcepStationsConfigPath();
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
return simulationProperties.getFnlStationsConfigPath();
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
return simulationProperties.getT1hStationsConfigPath();
|
||||
}
|
||||
return Strings.EMPTY;
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取正演脚本路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
private String getForwardScriptPath(Integer dataSource){
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
return simulationProperties.getPanguForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
return simulationProperties.getGraphcastForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
return simulationProperties.getCra40ForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
return simulationProperties.getNcepForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
return simulationProperties.getFnlForwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
return simulationProperties.getT1hForwardScriptPath();
|
||||
}
|
||||
return Strings.EMPTY;
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取正演脚本路径
|
||||
* @param dataSource
|
||||
* @return
|
||||
*/
|
||||
private String getBackForwardScriptPath(Integer dataSource){
|
||||
if(WeatherDataSourceEnum.PANGU.getKey().equals(dataSource)){
|
||||
return simulationProperties.getPanguBackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.GRAPHCAST.getKey().equals(dataSource)){
|
||||
return simulationProperties.getGraphcastBackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.CRA40.getKey().equals(dataSource)){
|
||||
return simulationProperties.getCra40BackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.NCEP.getKey().equals(dataSource)){
|
||||
return simulationProperties.getNcepBackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.FNL.getKey().equals(dataSource)){
|
||||
return simulationProperties.getFnlBackwardScriptPath();
|
||||
}else if(WeatherDataSourceEnum.T1H.getKey().equals(dataSource)){
|
||||
return simulationProperties.getT1hBackwardScriptPath();
|
||||
}
|
||||
return Strings.EMPTY;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,460 @@
|
|||
package org.jeecg.task.nuclearfacilitiestask;
|
||||
|
||||
import cn.hutool.core.collection.CollUtil;
|
||||
import cn.hutool.core.date.DateUtil;
|
||||
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.lang3.time.StopWatch;
|
||||
import org.apache.logging.log4j.util.Strings;
|
||||
import org.jeecg.common.constant.enums.AssociatedWaveformTaskEnum;
|
||||
import org.jeecg.common.constant.enums.WaveformEventResultTypeEnum;
|
||||
import org.jeecg.common.properties.DataFusionProperties;
|
||||
import org.jeecg.config.datasource.DataSourceSwitcher;
|
||||
import org.jeecg.modules.base.entity.copsdb.Origin;
|
||||
import org.jeecg.modules.base.entity.rnauto.GardsWaveformEvent;
|
||||
import org.jeecg.modules.base.entity.rnauto.GardsWaveformEventResult;
|
||||
import org.jeecg.modules.base.mapper.GardsWaveformEventMapper;
|
||||
import org.jeecg.modules.base.mapper.GardsWaveformEventResultMapper;
|
||||
import org.jeecg.modules.base.mapper.OriginMapper;
|
||||
import org.jeecg.vo.SRSRecord;
|
||||
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
|
||||
import org.springframework.transaction.TransactionDefinition;
|
||||
import org.springframework.transaction.TransactionStatus;
|
||||
import java.io.BufferedReader;
|
||||
import java.io.File;
|
||||
import java.io.FileInputStream;
|
||||
import java.io.InputStreamReader;
|
||||
import java.math.BigDecimal;
|
||||
import java.math.RoundingMode;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.LocalDateTime;
|
||||
import java.time.ZoneId;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.stream.Collectors;
|
||||
import java.util.zip.GZIPInputStream;
|
||||
|
||||
@Slf4j
|
||||
public class AssociatedNuclearFacilitiesTaskExec extends Thread{
|
||||
|
||||
private DataSourceTransactionManager transactionManager;
|
||||
private TransactionDefinition transactionDefinition;
|
||||
private DataFusionProperties dataFusionProperties;
|
||||
private GardsWaveformEventMapper waveformEventMapper;
|
||||
private GardsWaveformEventResultMapper waveformEventResultMapper;
|
||||
private OriginMapper originMapper;
|
||||
private Integer sampleId;
|
||||
private Date acqEndTime;
|
||||
private String stationCode;
|
||||
private File idcSrsFile = null;
|
||||
private File ndcSrsFile = null;
|
||||
private List<String> idcSrsContents;
|
||||
private List<String> ndcSrsContents;
|
||||
|
||||
/**
|
||||
* 初始化
|
||||
*/
|
||||
public void init(
|
||||
DataSourceTransactionManager transactionManager,
|
||||
TransactionDefinition transactionDefinition,
|
||||
DataFusionProperties dataFusionProperties,
|
||||
GardsWaveformEventMapper waveformEventMapper,
|
||||
OriginMapper originMapper,
|
||||
GardsWaveformEventResultMapper waveformEventResultMapper,
|
||||
Integer sampleId,
|
||||
Date acqEndTime,
|
||||
String stationCode){
|
||||
this.transactionManager = transactionManager;
|
||||
this.transactionDefinition = transactionDefinition;
|
||||
this.dataFusionProperties = dataFusionProperties;
|
||||
this.waveformEventMapper = waveformEventMapper;
|
||||
this.originMapper = originMapper;
|
||||
this.waveformEventResultMapper = waveformEventResultMapper;
|
||||
this.sampleId = sampleId;
|
||||
this.acqEndTime = acqEndTime;
|
||||
this.stationCode = stationCode;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
this.execute();
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行任务
|
||||
*/
|
||||
public void execute() {
|
||||
StopWatch stopWatch = new StopWatch();
|
||||
stopWatch.start();
|
||||
try{
|
||||
//清除历史关联结果
|
||||
this.cleanHistoryResult(this.sampleId);
|
||||
//修改任务状态为执行中
|
||||
this.updateTaskStatus(this.sampleId, AssociatedWaveformTaskEnum.IN_PROCESS.getValue());
|
||||
//检查srs文件是否存在
|
||||
this.checkSRSExist();
|
||||
//解析SRS文件数据
|
||||
this.parseSrsFile();
|
||||
//关联波形数据
|
||||
this.associationWaveform();
|
||||
this.execComplete(stopWatch,AssociatedWaveformTaskEnum.COMPLETE.getValue(),null);
|
||||
}catch (Exception e){
|
||||
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
|
||||
this.execComplete(stopWatch,AssociatedWaveformTaskEnum.UNSUCCESSFUL.getValue(),taskErrorLog);
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查IDC和NDC SRS文件是否存在
|
||||
*/
|
||||
private void checkSRSExist(){
|
||||
String acqEndDate = DateUtil.format(this.acqEndTime,"yyyyMMdd");
|
||||
this.checkIdcSrsExist(acqEndDate);
|
||||
this.checkNdcSrsExist(acqEndDate);
|
||||
}
|
||||
|
||||
/**
|
||||
* 解析SRS文件
|
||||
*/
|
||||
private void parseSrsFile(){
|
||||
if(Objects.nonNull(this.idcSrsFile)){
|
||||
this.idcSrsContents = this.readSrmFromGzStreaming(this.idcSrsFile);
|
||||
}
|
||||
if(Objects.nonNull(this.ndcSrsFile)){
|
||||
this.ndcSrsContents = this.readSrmFromGzStreaming(this.ndcSrsFile);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 关联波形数据
|
||||
*/
|
||||
private void associationWaveform(){
|
||||
Set<Integer> idcEvents = this.associationSrs(this.idcSrsContents);
|
||||
Set<Integer> ndcEvents = this.associationSrs(this.ndcSrsContents);
|
||||
DataSourceSwitcher.switchToOracle();
|
||||
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
|
||||
try {
|
||||
//保存关联结果
|
||||
if (CollUtil.isNotEmpty(idcEvents)){
|
||||
this.saveWaveformEventResult(idcEvents,WaveformEventResultTypeEnum.IDC.getKey());
|
||||
}
|
||||
if (CollUtil.isNotEmpty(ndcEvents)){
|
||||
this.saveWaveformEventResult(ndcEvents,WaveformEventResultTypeEnum.NDC.getKey());
|
||||
}
|
||||
//修改关联结果统计
|
||||
GardsWaveformEvent waveformEvent = this.waveformEventMapper.selectById(sampleId);
|
||||
waveformEvent.setIdcEvents(idcEvents.size());
|
||||
waveformEvent.setNdcEvents(ndcEvents.size());
|
||||
waveformEvent.setDescription(Strings.EMPTY);
|
||||
waveformEvent.setModdate(new Date());
|
||||
waveformEvent.setStatus(AssociatedWaveformTaskEnum.COMPLETE.getValue());
|
||||
this.waveformEventMapper.updateById(waveformEvent);
|
||||
this.transactionManager.commit(transactionStatus);
|
||||
}finally {
|
||||
DataSourceSwitcher.clearDataSource();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 关联idc srs文件
|
||||
*/
|
||||
private Set<Integer> associationSrs(List<String> srsContents){
|
||||
// 台站经度 台站纬度 开始测量日期 小时 结束测量时间 小时 系数 总共模拟时长 小时数 网格大小 台站编码
|
||||
// 139.08 36.30 20241203 12 20241203 18 0.1300000E+16 336 1 1 0.50 0.50 "JPX38"
|
||||
String[] firstLine = srsContents.get(0).split("\\s+");
|
||||
Integer totalHour = Integer.parseInt(firstLine[8]);
|
||||
Integer hourlyCoefficient = Integer.parseInt(firstLine[9]);
|
||||
Double gridLonResolution = Double.parseDouble(firstLine[11]);
|
||||
Double gridLatResolution = Double.parseDouble(firstLine[12]);
|
||||
//根据测量结束时间，生成时间范围，用于查询范围内的波形事件
|
||||
LocalDateTime endTime = LocalDateTime.ofInstant(this.acqEndTime.toInstant(), ZoneId.of("Asia/Shanghai"));
|
||||
endTime = endTime.withMinute(0).withSecond(0);
|
||||
LocalDateTime startTime = endTime.minusHours(totalHour).minusHours(dataFusionProperties.getTraceabilityTime()*24);
|
||||
System.out.println(startTime.format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")));
|
||||
System.out.println(endTime.format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")));
|
||||
//查询范围内波形事件
|
||||
List<Origin> origins = this.selectOriginByTime(startTime,endTime);
|
||||
//处理srs文件中的坐标记录格式化成对象
|
||||
List<SRSRecord> srsRecords = buildSRSRecord(srsContents, hourlyCoefficient,endTime);
|
||||
//怎么对srsRecords进行按经纬度分组，分组后，对每组经纬度只求一个网格边界，然后再根据时间做origins查询，统计关联数
|
||||
Map<Map.Entry<Double, Double>, List<SRSRecord>> srsRecordsGroup = srsRecords.stream()
|
||||
.collect(Collectors.groupingBy(record ->
|
||||
new AbstractMap.SimpleImmutableEntry<>(record.getLon(), record.getLat())
|
||||
));
|
||||
return this.queryWaveform(origins, srsRecordsGroup, gridLonResolution,gridLatResolution);
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查IDC SRS文件是否存在
|
||||
*/
|
||||
private void checkIdcSrsExist(String acqEndDate){
|
||||
//构建srs文件路径
|
||||
StringBuilder srsFilePath = new StringBuilder();
|
||||
srsFilePath.append(dataFusionProperties.getIdcSRSPath());
|
||||
srsFilePath.append(File.separator);
|
||||
srsFilePath.append(acqEndDate);
|
||||
srsFilePath.append(File.separator);
|
||||
srsFilePath.append(dataFusionProperties.getIdcSrmParentDir());
|
||||
|
||||
//构建srm.gz文件名称
|
||||
StringBuilder srmFileName = new StringBuilder();
|
||||
srmFileName.append(stationCode);
|
||||
srmFileName.append(".fp.");
|
||||
srmFileName.append(DateUtil.format(this.acqEndTime,"yyyyMMddHH"));
|
||||
srmFileName.append(".f9.srm.gz");
|
||||
|
||||
String fullFilePath = srsFilePath + File.separator + srmFileName;
|
||||
File idcSrsFile = new File(fullFilePath);
|
||||
if (idcSrsFile.exists()){
|
||||
this.idcSrsFile = idcSrsFile;
|
||||
}else {
|
||||
this.updateTaskNotSuccessful("IDC SRS文件不存在");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查NDC SRS文件是否存在
|
||||
*/
|
||||
private void checkNdcSrsExist(String acqEndDate){
|
||||
//构建srs文件路径
|
||||
StringBuilder srsFilePath = new StringBuilder();
|
||||
srsFilePath.append(dataFusionProperties.getNdcSRSPath());
|
||||
srsFilePath.append(File.separator);
|
||||
srsFilePath.append(acqEndDate);
|
||||
srsFilePath.append(File.separator);
|
||||
srsFilePath.append(dataFusionProperties.getNdcSrmParentDir());
|
||||
|
||||
//构建srm.gz文件名称
|
||||
StringBuilder srmFileName = new StringBuilder();
|
||||
srmFileName.append(stationCode);
|
||||
srmFileName.append(".fp.");
|
||||
srmFileName.append(DateUtil.format(this.acqEndTime,"yyyyMMddHH"));
|
||||
srmFileName.append(".f9.srm.gz");
|
||||
|
||||
String fullFilePath = srsFilePath + File.separator + srmFileName;
|
||||
File ndcSrsFile = new File(fullFilePath);
|
||||
if (ndcSrsFile.exists()){
|
||||
this.ndcSrsFile = idcSrsFile;
|
||||
}else {
|
||||
this.updateTaskNotSuccessful("NDC SRS文件不存在");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 处理srs文件中的坐标记录格式化成对象
|
||||
* @param contents
|
||||
* @param hourlyCoefficient
|
||||
* @param acqEndTime 测量结束时间
|
||||
* @return
|
||||
*/
|
||||
private List<SRSRecord> buildSRSRecord(List<String> contents,Integer hourlyCoefficient,LocalDateTime acqEndTime){
|
||||
List<SRSRecord> list = contents.parallelStream().skip(1).map(content->{
|
||||
String[] line = content.split("\\s+");
|
||||
BigDecimal val1 = new BigDecimal(line[4]);
|
||||
BigDecimal val2 = new BigDecimal(dataFusionProperties.getFilterConditions());
|
||||
//过滤掉比较小的值
|
||||
if(val1.compareTo(val2) > 0){
|
||||
SRSRecord srsRecord = new SRSRecord();
|
||||
srsRecord.setLat(Double.parseDouble(line[1]));
|
||||
srsRecord.setLon(Double.parseDouble(line[2]));
|
||||
srsRecord.setHour(Integer.parseInt(line[3])*hourlyCoefficient);
|
||||
srsRecord.setConc(line[4]);
|
||||
LocalDateTime endTime = acqEndTime.minusHours(srsRecord.getHour());
|
||||
LocalDateTime startTime = acqEndTime.minusHours(srsRecord.getHour()+dataFusionProperties.getTraceabilityTime()*24);
|
||||
long startSecond = startTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
|
||||
long endSecond = endTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
|
||||
srsRecord.setStartSecond(startSecond);
|
||||
srsRecord.setEndSecond(endSecond);
|
||||
return srsRecord;
|
||||
}
|
||||
return null;
|
||||
}).filter(Objects::nonNull)
|
||||
.toList();
|
||||
return list;
|
||||
}
|
||||
|
||||
/**
|
||||
* 根据经纬度及时间查询范围内的波形数据，并进行计数，最后的计数就是关联的波形结果
|
||||
* @param srsRecordsGroup
|
||||
*/
|
||||
private Set<Integer> queryWaveform(List<Origin> origins,Map<Map.Entry<Double, Double>, List<SRSRecord>> srsRecordsGroup,Double gridLonResolution,Double gridLatResolution){
|
||||
Set<Integer> result = ConcurrentHashMap.newKeySet();
|
||||
if (CollUtil.isNotEmpty(srsRecordsGroup) && CollUtil.isNotEmpty(origins)){
|
||||
srsRecordsGroup.forEach((key,srsRecords)->{
|
||||
double leftLon = Math.floor(key.getKey() / gridLonResolution) * gridLonResolution;
|
||||
double bottomLat = Math.floor(key.getValue() / gridLatResolution) * gridLatResolution;
|
||||
double rightLon = leftLon + gridLonResolution;
|
||||
double topLat = bottomLat + gridLatResolution;
|
||||
//统计srs中每条记录内关联的波形数据
|
||||
srsRecords.parallelStream().forEach(record->{
|
||||
Set<Integer> orids = origins.stream()
|
||||
.filter(origin ->origin.getLon() >= leftLon && origin.getLon() < rightLon &&
|
||||
origin.getLat() >= bottomLat && origin.getLat() < topLat &&
|
||||
origin.getTime() >= record.getStartSecond() && origin.getTime() <= record.getEndSecond())
|
||||
.map(Origin::getOrid).collect(Collectors.toSet());
|
||||
if (CollUtil.isNotEmpty(orids)){
|
||||
result.addAll(orids);
|
||||
}
|
||||
});
|
||||
});
|
||||
return result;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* 修改任务状态
|
||||
* @param sampleId
|
||||
* @param status
|
||||
*/
|
||||
private void updateTaskStatus(Integer sampleId, Integer status) {
|
||||
DataSourceSwitcher.switchToOracle();
|
||||
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
|
||||
try {
|
||||
GardsWaveformEvent waveformEvent = this.waveformEventMapper.selectById(sampleId);
|
||||
waveformEvent.setStatus(status);
|
||||
waveformEvent.setModdate(new Date());
|
||||
this.waveformEventMapper.updateById(waveformEvent);
|
||||
this.transactionManager.commit(transactionStatus);
|
||||
}catch (Exception e){
|
||||
this.transactionManager.rollback(transactionStatus);
|
||||
log.error(e.getMessage(),e);
|
||||
}finally {
|
||||
DataSourceSwitcher.clearDataSource();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 修改任务关联未成功
|
||||
* @param description
|
||||
*/
|
||||
private void updateTaskNotSuccessful(String description) {
|
||||
DataSourceSwitcher.switchToOracle();
|
||||
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
|
||||
try {
|
||||
GardsWaveformEvent waveformEvent = this.waveformEventMapper.selectById(this.sampleId);
|
||||
waveformEvent.setStatus(AssociatedWaveformTaskEnum.UNSUCCESSFUL.getValue());
|
||||
waveformEvent.setDescription(waveformEvent.getDescription()+"\n"+description);
|
||||
waveformEvent.setModdate(new Date());
|
||||
this.waveformEventMapper.updateById(waveformEvent);
|
||||
this.transactionManager.commit(transactionStatus);
|
||||
}catch (Exception e){
|
||||
this.transactionManager.rollback(transactionStatus);
|
||||
log.error(e.getMessage(),e);
|
||||
}finally {
|
||||
DataSourceSwitcher.clearDataSource();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 清除历史关联结果
|
||||
* @param sampleId
|
||||
*/
|
||||
private void cleanHistoryResult(Integer sampleId) {
|
||||
DataSourceSwitcher.switchToOracle();
|
||||
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
|
||||
try {
|
||||
LambdaQueryWrapper<GardsWaveformEventResult> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(GardsWaveformEventResult::getSampleId, sampleId);
|
||||
this.waveformEventResultMapper.delete(queryWrapper);
|
||||
this.transactionManager.commit(transactionStatus);
|
||||
}catch (Exception e){
|
||||
this.transactionManager.rollback(transactionStatus);
|
||||
log.error(e.getMessage(),e);
|
||||
}finally {
|
||||
DataSourceSwitcher.clearDataSource();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取时间范围内的波形数据
|
||||
* @param startTime
|
||||
* @param endTime
|
||||
* @return
|
||||
*/
|
||||
private List<Origin> selectOriginByTime(LocalDateTime startTime,LocalDateTime endTime){
|
||||
DataSourceSwitcher.switchToOracle();
|
||||
try {
|
||||
long startSecond = startTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
|
||||
long endSecond = endTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
|
||||
return originMapper.selectOriginByTime(startSecond,endSecond);
|
||||
}finally {
|
||||
DataSourceSwitcher.clearDataSource();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 保存关联结果
|
||||
* @param orids
|
||||
*/
|
||||
private void saveWaveformEventResult(Set<Integer> orids,Integer type){
|
||||
DataSourceSwitcher.switchToOracle();
|
||||
final TransactionStatus transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
|
||||
try {
|
||||
if(CollUtil.isNotEmpty(orids)){
|
||||
List<GardsWaveformEventResult> waveformEventResults = new ArrayList<>();
|
||||
orids.forEach(orid->{
|
||||
GardsWaveformEventResult waveformEventResult = new GardsWaveformEventResult();
|
||||
waveformEventResult.setSampleId(this.sampleId);
|
||||
waveformEventResult.setOrid(orid);
|
||||
waveformEventResult.setType(type);
|
||||
waveformEventResult.setModdate(new Date());
|
||||
waveformEventResults.add(waveformEventResult);
|
||||
});
|
||||
this.waveformEventResultMapper.insert(waveformEventResults);
|
||||
this.transactionManager.commit(transactionStatus);
|
||||
}
|
||||
}finally {
|
||||
DataSourceSwitcher.clearDataSource();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 读取srm文件
|
||||
* @param gzFile
|
||||
* @return
|
||||
*/
|
||||
public List<String> readSrmFromGzStreaming(File gzFile){
|
||||
try {
|
||||
List<String> contents = new ArrayList<>();
|
||||
try (FileInputStream fis = new FileInputStream(gzFile);
|
||||
GZIPInputStream gis = new GZIPInputStream(fis);
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8))) {
|
||||
|
||||
String line;
|
||||
while ((line = reader.readLine()) != null) {
|
||||
contents.add(line);
|
||||
}
|
||||
}
|
||||
return contents;
|
||||
}catch (Exception e){
|
||||
log.error("SRS文件解析异常，路径为：{},原因为：{}",gzFile.getAbsolutePath(),e.getMessage());
|
||||
return List.of();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 任务执行完成
|
||||
* @param stopWatch
|
||||
* @param taskStatus
|
||||
* @param taskErrorLog
|
||||
*/
|
||||
private void execComplete(StopWatch stopWatch,Integer taskStatus,String taskErrorLog){
|
||||
//添加任务耗时
|
||||
stopWatch.stop();
|
||||
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
|
||||
double min = seconds/60D;
|
||||
BigDecimal bgMin = new BigDecimal(min);
|
||||
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
|
||||
if (AssociatedWaveformTaskEnum.COMPLETE.getValue().equals(taskStatus)){
|
||||
log.info("任务执行完成，耗时{}分钟",result);
|
||||
|
||||
}else if (AssociatedWaveformTaskEnum.UNSUCCESSFUL.getValue().equals(taskStatus)){
|
||||
log.error(taskErrorLog);
|
||||
}
|
||||
this.updateTaskStatus(this.sampleId,taskStatus);
|
||||
}
|
||||
}
|
||||
|
|
@ -164,7 +164,8 @@ public class AssociatedWaveformTaskExec extends Thread{
|
|||
String[] firstLine = srsContents.get(0).split("\\s+");
|
||||
Integer totalHour = Integer.parseInt(firstLine[8]);
|
||||
Integer hourlyCoefficient = Integer.parseInt(firstLine[9]);
|
||||
Double gridSize = Double.parseDouble(firstLine[11]);
|
||||
Double gridLonResolution = Double.parseDouble(firstLine[11]);
|
||||
Double gridLatResolution = Double.parseDouble(firstLine[12]);
|
||||
//根据测量结束时间，生成时间范围，用于查询范围内的波形事件
|
||||
LocalDateTime endTime = LocalDateTime.ofInstant(this.acqEndTime.toInstant(), ZoneId.of("Asia/Shanghai"));
|
||||
endTime = endTime.withMinute(0).withSecond(0);
|
||||
|
|
@ -180,7 +181,7 @@ public class AssociatedWaveformTaskExec extends Thread{
|
|||
.collect(Collectors.groupingBy(record ->
|
||||
new AbstractMap.SimpleImmutableEntry<>(record.getLon(), record.getLat())
|
||||
));
|
||||
return this.queryWaveform(origins, srsRecordsGroup, gridSize);
|
||||
return this.queryWaveform(origins, srsRecordsGroup, gridLonResolution,gridLatResolution);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -193,7 +194,7 @@ public class AssociatedWaveformTaskExec extends Thread{
|
|||
srsFilePath.append(File.separator);
|
||||
srsFilePath.append(acqEndDate);
|
||||
srsFilePath.append(File.separator);
|
||||
srsFilePath.append(dataFusionProperties.getSrmParentDir());
|
||||
srsFilePath.append(dataFusionProperties.getIdcSrmParentDir());
|
||||
|
||||
//构建srm.gz文件名称
|
||||
StringBuilder srmFileName = new StringBuilder();
|
||||
|
|
@ -221,7 +222,7 @@ public class AssociatedWaveformTaskExec extends Thread{
|
|||
srsFilePath.append(File.separator);
|
||||
srsFilePath.append(acqEndDate);
|
||||
srsFilePath.append(File.separator);
|
||||
srsFilePath.append(dataFusionProperties.getSrmParentDir());
|
||||
srsFilePath.append(dataFusionProperties.getNdcSrmParentDir());
|
||||
|
||||
//构建srm.gz文件名称
|
||||
StringBuilder srmFileName = new StringBuilder();
|
||||
|
|
@ -276,14 +277,14 @@ public class AssociatedWaveformTaskExec extends Thread{
|
|||
* 根据经纬度及时间查询范围内的波形数据，并进行计数，最后的计数就是关联的波形结果
|
||||
* @param srsRecordsGroup
|
||||
*/
|
||||
private Set<Integer> queryWaveform(List<Origin> origins,Map<Map.Entry<Double, Double>, List<SRSRecord>> srsRecordsGroup,Double gridSize){
|
||||
private Set<Integer> queryWaveform(List<Origin> origins,Map<Map.Entry<Double, Double>, List<SRSRecord>> srsRecordsGroup,Double gridLonResolution,Double gridLatResolution){
|
||||
Set<Integer> result = ConcurrentHashMap.newKeySet();
|
||||
if (CollUtil.isNotEmpty(srsRecordsGroup) && CollUtil.isNotEmpty(origins)){
|
||||
srsRecordsGroup.forEach((key,srsRecords)->{
|
||||
double leftLon = Math.floor(key.getKey() / gridSize) * gridSize;
|
||||
double bottomLat = Math.floor(key.getValue() / gridSize) * gridSize;
|
||||
double rightLon = leftLon + gridSize;
|
||||
double topLat = bottomLat + gridSize;
|
||||
double leftLon = Math.floor(key.getKey() / gridLonResolution) * gridLonResolution;
|
||||
double bottomLat = Math.floor(key.getValue() / gridLatResolution) * gridLatResolution;
|
||||
double rightLon = leftLon + gridLonResolution;
|
||||
double topLat = bottomLat + gridLatResolution;
|
||||
//统计srs中每条记录内关联的波形数据
|
||||
srsRecords.parallelStream().forEach(record->{
|
||||
Set<Integer> orids = origins.stream()
|
||||
|
|
@ -453,17 +454,4 @@ public class AssociatedWaveformTaskExec extends Thread{
|
|||
}
|
||||
this.updateTaskStatus(this.sampleId,taskStatus);
|
||||
}
|
||||
|
||||
public static void main(String[] args) {
|
||||
LocalDateTime endTime = LocalDateTime.of(2024,12, 3,18, 0, 0);
|
||||
LocalDateTime startTime = endTime.minusDays(74);
|
||||
|
||||
long startSecond = startTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
|
||||
long endSecond = endTime.atZone(ZoneId.of("Asia/Shanghai")).toEpochSecond();
|
||||
|
||||
System.out.println(startSecond);
|
||||
System.out.println(endSecond);
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
|||
|
|
@ -321,7 +321,8 @@ public class WeatherDataServiceImpl extends ServiceImpl<WeatherDataMapper, Weath
|
|||
queryWrapper.between((Objects.nonNull(startTime) && Objects.nonNull(endTime)),WeatherData::getDataStartTime,startTime,endTime);
|
||||
queryWrapper.eq(StringUtils.isNotBlank(fileExt),WeatherData::getFileExt, fileExt);
|
||||
queryWrapper.like(StringUtils.isNotBlank(fileName),WeatherData::getFileName, fileName);
|
||||
queryWrapper.select(WeatherData::getId,WeatherData::getFileName,WeatherData::getFileSize,WeatherData::getDataSource,WeatherData::getFileExt,WeatherData::getDataStartTime,WeatherData::getFilePath);
|
||||
queryWrapper.select(WeatherData::getId,WeatherData::getFileName,WeatherData::getFileSize,WeatherData::getDataSource,
|
||||
WeatherData::getFileExt,WeatherData::getDataStartTime,WeatherData::getFilePath,WeatherData::getTimeBatch);
|
||||
IPage<WeatherData> iPage = new Page<>(pageRequest.getPageNum(),pageRequest.getPageSize());
|
||||
return this.baseMapper.selectPage(iPage, queryWrapper);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
|
|||
import lombok.RequiredArgsConstructor;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.apache.logging.log4j.util.Strings;
|
||||
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
|
||||
import org.jeecg.common.constant.enums.WeatherFileSuffixEnum;
|
||||
import org.jeecg.common.constant.enums.WeatherForecastDatasourceEnum;
|
||||
import org.jeecg.common.constant.enums.WeatherTaskStatusEnum;
|
||||
|
|
@ -20,7 +21,8 @@ import org.jeecg.modules.base.mapper.WeatherDataMapper;
|
|||
import org.jeecg.modules.base.mapper.WeatherTaskLogMapper;
|
||||
import org.jeecg.modules.base.mapper.WeatherTaskMapper;
|
||||
import org.jeecg.service.WeatherTaskService;
|
||||
import org.jeecg.task.WeatherTaskExec;
|
||||
import org.jeecg.task.GraphcastWeatherTaskExec;
|
||||
import org.jeecg.task.PanGuWeatherTaskExec;
|
||||
import org.springframework.stereotype.Service;
|
||||
import org.springframework.transaction.annotation.Transactional;
|
||||
import org.springframework.web.multipart.MultipartFile;
|
||||
|
|
@ -225,10 +227,17 @@ public class WeatherTaskServiceImpl extends ServiceImpl<WeatherTaskMapper, Weath
|
|||
if(Objects.isNull(task)){
|
||||
throw new RuntimeException("此任务不存在");
|
||||
}
|
||||
WeatherTaskExec exec = new WeatherTaskExec();
|
||||
exec.init(task,this,systemStorageProperties,weatherDataMapper);
|
||||
exec.setName("天气预测任务线程");
|
||||
exec.start();
|
||||
if (WeatherDataSourceEnum.PANGU.getKey().equals(task.getPredictionModel())){
|
||||
PanGuWeatherTaskExec exec = new PanGuWeatherTaskExec();
|
||||
exec.init(task,this,systemStorageProperties,weatherDataMapper);
|
||||
exec.setName("天气预测任务线程");
|
||||
exec.start();
|
||||
} else if (WeatherDataSourceEnum.GRAPHCAST.getKey().equals(task.getPredictionModel())) {
|
||||
GraphcastWeatherTaskExec exec = new GraphcastWeatherTaskExec();
|
||||
exec.init(task,this,systemStorageProperties,weatherDataMapper);
|
||||
exec.setName("天气预测任务线程");
|
||||
exec.start();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -1,7 +1,5 @@
|
|||
package org.jeecg.task;
|
||||
|
||||
import cn.hutool.core.collection.CollUtil;
|
||||
import cn.hutool.core.date.DateUtil;
|
||||
import cn.hutool.core.date.LocalDateTimeUtil;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import cn.hutool.core.util.StrUtil;
|
||||
|
|
@ -17,16 +15,13 @@ import org.jeecg.common.properties.SystemStorageProperties;
|
|||
import org.jeecg.common.util.NcUtil;
|
||||
import org.jeecg.modules.base.entity.WeatherData;
|
||||
import org.jeecg.modules.base.entity.WeatherTask;
|
||||
import org.jeecg.modules.base.entity.WeatherTaskLog;
|
||||
import org.jeecg.modules.base.mapper.WeatherDataMapper;
|
||||
import org.jeecg.service.WeatherDataService;
|
||||
import org.jeecg.service.WeatherTaskService;
|
||||
|
||||
import java.io.*;
|
||||
import java.math.BigDecimal;
|
||||
import java.math.RoundingMode;
|
||||
import java.time.Instant;
|
||||
import java.time.LocalDate;
|
||||
import java.time.LocalDateTime;
|
||||
import java.time.ZoneId;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
|
|
@ -38,7 +33,7 @@ import java.util.concurrent.TimeUnit;
|
|||
/**
|
||||
* 气象预测任务执行线程
|
||||
*/
|
||||
public class WeatherTaskExec extends Thread{
|
||||
public class GraphcastWeatherTaskExec extends Thread{
|
||||
|
||||
private WeatherTask weatherTask;
|
||||
private WeatherTaskService weatherTaskService;
|
||||
|
|
@ -116,11 +111,11 @@ public class WeatherTaskExec extends Thread{
|
|||
//输入文件
|
||||
String inputFileName = this.weatherTask.getInputFile();
|
||||
//预测文件
|
||||
String outputFileName = "panguweather_output_"+this.weatherTask.getId()+".grib";
|
||||
String outputFileName = "graphcast_output_"+this.weatherTask.getId()+".grib";
|
||||
//定义名称后缀,执行grib_set命令时去除，否则命名会冲突
|
||||
String gribFileSuffix = "_not_grib_set";
|
||||
//临时目录grib_copy,grib_set,python转换都在这个目录里，最后删除
|
||||
String tempDir = systemStorageProperties.getPanguModelExecPath()+File.separator+this.weatherTask.getId();
|
||||
String tempDir = systemStorageProperties.getGraphcastModelExecPath()+File.separator+this.weatherTask.getId();
|
||||
if(!FileUtil.exist(tempDir)){
|
||||
FileUtil.mkdir(tempDir);
|
||||
}
|
||||
|
|
@ -129,23 +124,23 @@ public class WeatherTaskExec extends Thread{
|
|||
//切割文件，6小时一个
|
||||
this.splitGrib(outputFileName,gribFileSuffix,tempDir);
|
||||
//适配flexpart
|
||||
this.convertGrib(gribFileSuffix,tempDir);
|
||||
// this.convertGrib(gribFileSuffix,tempDir);
|
||||
//保存文件数据入库
|
||||
this.saveGribInfoToDB(tempDir,this.getPanguWeatherPath());
|
||||
this.saveGribInfoToDB(tempDir,this.getPanguWeatherPath(),gribFileSuffix);
|
||||
//删除预测文件
|
||||
File inputFile = new File(systemStorageProperties.getPanguModelExecPath()+File.separator+inputFileName);
|
||||
if (inputFile.exists()){
|
||||
inputFile.delete();
|
||||
}
|
||||
//删除输出的总文件
|
||||
File outputFile = new File(systemStorageProperties.getPanguModelExecPath()+File.separator+outputFileName);
|
||||
if (outputFile.exists()){
|
||||
outputFile.delete();
|
||||
}
|
||||
//适配flexpart结束，删除目录
|
||||
if(FileUtil.exist(tempDir)){
|
||||
FileUtil.del(tempDir);
|
||||
}
|
||||
// File inputFile = new File(systemStorageProperties.getGraphcastModelExecPath()+File.separator+inputFileName);
|
||||
// if (inputFile.exists()){
|
||||
// inputFile.delete();
|
||||
// }
|
||||
// //删除输出的总文件
|
||||
// File outputFile = new File(systemStorageProperties.getGraphcastModelExecPath()+File.separator+outputFileName);
|
||||
// if (outputFile.exists()){
|
||||
// outputFile.delete();
|
||||
// }
|
||||
// //适配flexpart结束，删除目录
|
||||
// if(FileUtil.exist(tempDir)){
|
||||
// FileUtil.del(tempDir);
|
||||
// }
|
||||
} catch (Exception e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
|
|
@ -159,15 +154,10 @@ public class WeatherTaskExec extends Thread{
|
|||
private void forecast(String inputFileAddr,String outputFileAddr){
|
||||
try {
|
||||
//处理开始日志
|
||||
String forecastModel = "";
|
||||
String startLogFormat = "";
|
||||
String startTimeFormat = LocalDateTimeUtil.format(this.weatherTask.getStartDate(),"yyyy-MM-dd HH:mm:ss");
|
||||
String forecastTime = this.weatherTask.getLeadTime().toString();
|
||||
if (WeatherDataSourceEnum.GRAPHCAST.getKey().equals(this.weatherTask.getPredictionModel())){
|
||||
forecastModel = WeatherDataSourceEnum.GRAPHCAST.getValue();
|
||||
}else if(WeatherDataSourceEnum.PANGU.getKey().equals(this.weatherTask.getPredictionModel())){
|
||||
forecastModel = WeatherDataSourceEnum.PANGU.getValue();
|
||||
}
|
||||
String forecastModel = WeatherDataSourceEnum.GRAPHCAST.getValue();
|
||||
if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){
|
||||
startLogFormat = "任务开始，本次使用预测模型为：%s，预测开始时间为：%s，预测时长为：%s，前置数据采用CDS在线数据";
|
||||
}else {
|
||||
|
|
@ -177,7 +167,7 @@ public class WeatherTaskExec extends Thread{
|
|||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),startLog));
|
||||
//处理运行命令
|
||||
List<String> command = new ArrayList<>();
|
||||
String aiModelsPath = systemStorageProperties.getPanguEnvPath()+File.separator+"ai-models";
|
||||
String aiModelsPath = systemStorageProperties.getGraphcastEnvPath()+File.separator+"ai-models";
|
||||
|
||||
if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){
|
||||
command.add(aiModelsPath);
|
||||
|
|
@ -192,8 +182,8 @@ public class WeatherTaskExec extends Thread{
|
|||
command.add("--path");
|
||||
command.add(outputFileAddr);
|
||||
command.add("--assets");
|
||||
command.add("assets-panguweather");
|
||||
command.add("panguweather");
|
||||
command.add("assets-graphcast");
|
||||
command.add("--class od graphcast");
|
||||
}else if (this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.LOCATION_FILE.getKey())){
|
||||
//离线文件还需处理
|
||||
command.add(aiModelsPath);
|
||||
|
|
@ -204,13 +194,13 @@ public class WeatherTaskExec extends Thread{
|
|||
command.add("--path");
|
||||
command.add(outputFileAddr);
|
||||
command.add("--assets");
|
||||
command.add("assets-panguweather");
|
||||
command.add("panguweather");
|
||||
command.add("assets-graphcast");
|
||||
command.add("--class od graphcast");
|
||||
}
|
||||
String execLog = "执行任务命令，开始模拟，命令为："+command;
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),execLog));
|
||||
ProcessBuilder processBuilder = new ProcessBuilder(command);
|
||||
processBuilder.directory(new File(systemStorageProperties.getPanguModelExecPath()));
|
||||
processBuilder.directory(new File(systemStorageProperties.getGraphcastModelExecPath()));
|
||||
processBuilder.redirectErrorStream(true);
|
||||
Process process = processBuilder.start();
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream(), "UTF-8"));
|
||||
|
|
@ -240,17 +230,17 @@ public class WeatherTaskExec extends Thread{
|
|||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog));
|
||||
LocalDateTime startTime = this.weatherTask.getStartDate().atTime(this.weatherTask.getStartTime(), 0, 0);
|
||||
//grib_copy命令
|
||||
String gribCopyCommandPath = systemStorageProperties.getPanguEnvPath()+File.separator+"grib_copy";
|
||||
String gribCopyCommandPath = systemStorageProperties.getGraphcastEnvPath()+File.separator+"grib_copy";
|
||||
//grib_set命令
|
||||
String gribSetCommandPath = systemStorageProperties.getPanguEnvPath()+File.separator+"grib_set";
|
||||
String gribSetCommandPath = systemStorageProperties.getGraphcastEnvPath()+File.separator+"grib_set";
|
||||
//公用gribProcessBuilder
|
||||
ProcessBuilder gribProcessBuilder = new ProcessBuilder();
|
||||
gribProcessBuilder.directory(new File(systemStorageProperties.getPanguModelExecPath()));
|
||||
gribProcessBuilder.directory(new File(systemStorageProperties.getGraphcastModelExecPath()));
|
||||
//把grib文件切割成每6小时一份
|
||||
int step = 6;
|
||||
int i=this.weatherTask.getStartTime();
|
||||
while(i <= this.weatherTask.getLeadTime()){
|
||||
String gribCopyFileAddr = gribCopyTargetDir+File.separator+"panguweather_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+gribFileSuffix+".grib";
|
||||
String gribCopyFileAddr = gribCopyTargetDir+File.separator+"graphcast_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+gribFileSuffix+".grib";
|
||||
//切割grib文件命令
|
||||
List<String> gribCopyCommand = new ArrayList<>();
|
||||
gribCopyCommand.add(gribCopyCommandPath);
|
||||
|
|
@ -264,7 +254,7 @@ public class WeatherTaskExec extends Thread{
|
|||
gribCopyProcess.waitFor();
|
||||
|
||||
//重新设置reftime信息
|
||||
String gribSetFileAddr = gribCopyTargetDir+File.separator+"panguweather_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+".grib";
|
||||
String gribSetFileAddr = gribCopyTargetDir+File.separator+"graphcast_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+".grib";
|
||||
String date = LocalDateTimeUtil.format(startTime,"yyyyMMdd");
|
||||
String time = LocalDateTimeUtil.format(startTime,"HHmm");
|
||||
List<String> gribSetCommand = new ArrayList<>();
|
||||
|
|
@ -338,11 +328,11 @@ public class WeatherTaskExec extends Thread{
|
|||
* @param inputPath
|
||||
* @param targetPath
|
||||
*/
|
||||
private void saveGribInfoToDB(String inputPath,String targetPath){
|
||||
private void saveGribInfoToDB(String inputPath,String targetPath,String gribFileSuffix){
|
||||
String handleGribLog = "格式转换结束，处理grib文件数据入库";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog));
|
||||
|
||||
List<File> files = FileUtil.loopFiles(inputPath,file -> file.getName().startsWith("pangu_"));
|
||||
List<File> files = FileUtil.loopFiles(inputPath,file -> file.getName().startsWith("graphcast_") && !file.getName().contains(gribFileSuffix));
|
||||
List<File> targetFiles = new ArrayList<>();
|
||||
if(!files.isEmpty()){
|
||||
for(File file:files){
|
||||
|
|
@ -0,0 +1,438 @@
|
|||
package org.jeecg.task;
|
||||
|
||||
import cn.hutool.core.date.LocalDateTimeUtil;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import cn.hutool.core.util.StrUtil;
|
||||
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.codec.digest.DigestUtils;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.apache.commons.lang3.time.StopWatch;
|
||||
import org.jeecg.common.constant.enums.WeatherDataSourceEnum;
|
||||
import org.jeecg.common.constant.enums.WeatherForecastDatasourceEnum;
|
||||
import org.jeecg.common.constant.enums.WeatherTaskStatusEnum;
|
||||
import org.jeecg.common.exception.JeecgFileUploadException;
|
||||
import org.jeecg.common.properties.SystemStorageProperties;
|
||||
import org.jeecg.common.util.NcUtil;
|
||||
import org.jeecg.modules.base.entity.WeatherData;
|
||||
import org.jeecg.modules.base.entity.WeatherTask;
|
||||
import org.jeecg.modules.base.mapper.WeatherDataMapper;
|
||||
import org.jeecg.service.WeatherTaskService;
|
||||
|
||||
import java.io.*;
|
||||
import java.math.BigDecimal;
|
||||
import java.math.RoundingMode;
|
||||
import java.time.Instant;
|
||||
import java.time.LocalDateTime;
|
||||
import java.time.ZoneId;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
/**
|
||||
* 气象预测任务执行线程
|
||||
*/
|
||||
@Slf4j
|
||||
public class PanGuWeatherTaskExec extends Thread{
|
||||
|
||||
private WeatherTask weatherTask;
|
||||
private WeatherTaskService weatherTaskService;
|
||||
private SystemStorageProperties systemStorageProperties;
|
||||
private WeatherDataMapper weatherDataMapper;
|
||||
|
||||
/**
|
||||
* 初始化
|
||||
*/
|
||||
public void init(WeatherTask weatherTask,
|
||||
WeatherTaskService weatherTaskService,
|
||||
SystemStorageProperties systemStorageProperties,
|
||||
WeatherDataMapper weatherDataMapper){
|
||||
this.weatherTask = weatherTask;
|
||||
this.weatherTaskService = weatherTaskService;
|
||||
this.systemStorageProperties = systemStorageProperties;
|
||||
this.weatherDataMapper = weatherDataMapper;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
this.execute();
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行任务
|
||||
*/
|
||||
public void execute() {
|
||||
StopWatch stopWatch = new StopWatch();
|
||||
stopWatch.start();
|
||||
try{
|
||||
//修改任务状态为执行中
|
||||
this.weatherTaskService.updateTaskStatus(this.weatherTask.getId(), WeatherTaskStatusEnum.IN_OPERATION.getKey());
|
||||
//如果此任务已存在历史日志，先清除
|
||||
this.weatherTaskService.deleteTaskLog(this.weatherTask.getId());
|
||||
//执行模拟
|
||||
this.execSimulation();
|
||||
//执行完成
|
||||
this.execComplete(stopWatch,WeatherTaskStatusEnum.COMPLETED.getKey(), "");
|
||||
}catch (Exception e){
|
||||
String taskErrorLog = "任务执行失败，原因："+e.getMessage();
|
||||
//执行失败
|
||||
this.execComplete(stopWatch,WeatherTaskStatusEnum.FAILURE.getKey(), taskErrorLog);
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 任务执行完成
|
||||
* @param stopWatch
|
||||
* @param taskStatus
|
||||
* @param taskErrorLog
|
||||
*/
|
||||
private void execComplete(StopWatch stopWatch,Integer taskStatus,String taskErrorLog){
|
||||
//添加任务耗时
|
||||
stopWatch.stop();
|
||||
long seconds = stopWatch.getTime(TimeUnit.SECONDS);
|
||||
double min = seconds/60D;
|
||||
BigDecimal bgMin = new BigDecimal(min);
|
||||
BigDecimal result = bgMin.setScale(2, RoundingMode.HALF_UP);
|
||||
this.weatherTaskService.updateTaskStatusToCompleted(this.weatherTask.getId(),result.doubleValue(),taskStatus);
|
||||
if (WeatherTaskStatusEnum.COMPLETED.getKey().equals(taskStatus)){
|
||||
String taskCompletedLog = "任务执行完成，耗时："+result+"分钟";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),taskCompletedLog));
|
||||
}else if (WeatherTaskStatusEnum.FAILURE.getKey().equals(taskStatus)){
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),taskErrorLog));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 执行模拟
|
||||
*/
|
||||
private void execSimulation(){
|
||||
try {
|
||||
//输入文件
|
||||
String inputFileName = this.weatherTask.getInputFile();
|
||||
//预测文件
|
||||
String outputFileName = "panguweather_output_"+this.weatherTask.getId()+".grib";
|
||||
//定义名称后缀,执行grib_set命令时去除，否则命名会冲突
|
||||
String gribFileSuffix = "_not_grib_set";
|
||||
|
||||
//临时目录grib_copy,grib_set,python转换的输入都在这个目录里，最后删除
|
||||
String inputPath = systemStorageProperties.getPanguModelExecPath()+File.separator+this.weatherTask.getId();
|
||||
if(!FileUtil.exist(inputPath)){
|
||||
FileUtil.mkdir(inputPath);
|
||||
}
|
||||
//python转换的输出在这里，最后删除
|
||||
String outputPath = inputPath+File.separator+"formatResult";
|
||||
if(!FileUtil.exist(outputPath)){
|
||||
FileUtil.mkdir(outputPath);
|
||||
}
|
||||
//预测气象数据
|
||||
this.forecast(inputFileName,outputFileName);
|
||||
//切割文件，6小时一个
|
||||
this.splitGrib(outputFileName,gribFileSuffix,inputPath);
|
||||
//适配flexpart
|
||||
this.convertGrib(gribFileSuffix,inputPath,outputPath);
|
||||
//保存文件数据入库
|
||||
this.saveGribInfoToDB(inputPath,this.getPanguWeatherPath());
|
||||
//删除预测文件
|
||||
File inputFile = new File(systemStorageProperties.getPanguModelExecPath()+File.separator+inputFileName);
|
||||
if (inputFile.exists()){
|
||||
inputFile.delete();
|
||||
}
|
||||
// 删除输出的总文件
|
||||
File outputFile = new File(systemStorageProperties.getPanguModelExecPath()+File.separator+outputFileName);
|
||||
if (outputFile.exists()){
|
||||
outputFile.delete();
|
||||
}
|
||||
//适配flexpart结束，删除目录
|
||||
if(FileUtil.exist(inputPath)){
|
||||
FileUtil.del(inputPath);
|
||||
}
|
||||
} catch (Exception e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 盘古预测
|
||||
* @param inputFileAddr
|
||||
* @param outputFileAddr
|
||||
*/
|
||||
private void forecast(String inputFileAddr,String outputFileAddr){
|
||||
try {
|
||||
//处理开始日志
|
||||
String startLogFormat = "";
|
||||
String startTimeFormat = LocalDateTimeUtil.format(this.weatherTask.getStartDate(),"yyyy-MM-dd HH:mm:ss");
|
||||
String forecastTime = this.weatherTask.getLeadTime().toString();
|
||||
String forecastModel = WeatherDataSourceEnum.PANGU.getValue();
|
||||
if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){
|
||||
startLogFormat = "任务开始，本次使用预测模型为：%s，预测开始时间为：%s，预测时长为：%s，前置数据采用CDS在线数据";
|
||||
}else {
|
||||
startLogFormat = "任务开始，本次使用预测模型为：%s，预测开始时间为：%s，预测时长为：%s，前置数据采用离线数据："+this.weatherTask.getInputFile();
|
||||
}
|
||||
String startLog = String.format(startLogFormat,forecastModel,startTimeFormat,forecastTime);
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),startLog));
|
||||
//处理运行命令
|
||||
List<String> command = new ArrayList<>();
|
||||
String aiModelsPath = systemStorageProperties.getPanguEnvPath()+File.separator+"ai-models";
|
||||
|
||||
if(this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.CDS.getKey())){
|
||||
command.add(aiModelsPath);
|
||||
command.add("--input");
|
||||
command.add("cds");
|
||||
command.add("--date");
|
||||
command.add(LocalDateTimeUtil.format(this.weatherTask.getStartDate(),"yyyyMMdd"));
|
||||
command.add("--time");
|
||||
command.add(this.weatherTask.getStartTime().toString());
|
||||
command.add("--lead-time");
|
||||
command.add(this.weatherTask.getLeadTime().toString());
|
||||
command.add("--path");
|
||||
command.add(outputFileAddr);
|
||||
command.add("--assets");
|
||||
command.add("assets-panguweather");
|
||||
command.add("panguweather");
|
||||
}else if (this.weatherTask.getDataSources().equals(WeatherForecastDatasourceEnum.LOCATION_FILE.getKey())){
|
||||
//离线文件还需处理
|
||||
command.add(aiModelsPath);
|
||||
command.add("--file");
|
||||
command.add(inputFileAddr);
|
||||
command.add("--lead-time");
|
||||
command.add(this.weatherTask.getLeadTime().toString());
|
||||
command.add("--path");
|
||||
command.add(outputFileAddr);
|
||||
command.add("--assets");
|
||||
command.add("assets-panguweather");
|
||||
command.add("panguweather");
|
||||
}
|
||||
String execLog = "执行任务命令，开始模拟，命令为："+command;
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),execLog));
|
||||
ProcessBuilder processBuilder = new ProcessBuilder(command);
|
||||
processBuilder.directory(new File(systemStorageProperties.getPanguModelExecPath()));
|
||||
processBuilder.redirectErrorStream(true);
|
||||
Process process = processBuilder.start();
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream(), "UTF-8"));
|
||||
//读取输出日志
|
||||
String line;
|
||||
while ((line = reader.readLine()) != null) {
|
||||
if(StrUtil.isNotBlank(line)){
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),line));
|
||||
}
|
||||
}
|
||||
//等待进程结束
|
||||
process.waitFor();
|
||||
}catch (Exception e){
|
||||
throw new RuntimeException("盘古模型预测异常",e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 分割grib文件
|
||||
* @param outputFileAddr
|
||||
* @param gribFileSuffix
|
||||
* @param gribCopyTargetDir
|
||||
*/
|
||||
private void splitGrib(String outputFileAddr,String gribFileSuffix,String gribCopyTargetDir){
|
||||
try {
|
||||
String handleGribLog = "预测结束，开始处理grib文件,切割为6小时一个文件";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog));
|
||||
LocalDateTime startTime = this.weatherTask.getStartDate().atTime(this.weatherTask.getStartTime(), 0, 0);
|
||||
//grib_copy命令
|
||||
String gribCopyCommandPath = systemStorageProperties.getPanguEnvPath()+File.separator+"grib_copy";
|
||||
//grib_set命令
|
||||
String gribSetCommandPath = systemStorageProperties.getPanguEnvPath()+File.separator+"grib_set";
|
||||
//公用gribProcessBuilder
|
||||
ProcessBuilder gribProcessBuilder = new ProcessBuilder();
|
||||
gribProcessBuilder.directory(new File(systemStorageProperties.getPanguModelExecPath()));
|
||||
//把grib文件切割成每6小时一份
|
||||
int step = 6;
|
||||
int i=this.weatherTask.getStartTime();
|
||||
while(i <= this.weatherTask.getLeadTime()){
|
||||
String gribCopyFileAddr = gribCopyTargetDir+File.separator+"panguweather_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+gribFileSuffix+".grib";
|
||||
//切割grib文件命令
|
||||
List<String> gribCopyCommand = new ArrayList<>();
|
||||
gribCopyCommand.add(gribCopyCommandPath);
|
||||
gribCopyCommand.add("-w");
|
||||
gribCopyCommand.add("step="+i);
|
||||
gribCopyCommand.add(outputFileAddr);
|
||||
gribCopyCommand.add(gribCopyFileAddr);
|
||||
gribProcessBuilder.command(gribCopyCommand);
|
||||
log.info("gribCopyCommand:{}", gribCopyCommand);
|
||||
Process gribCopyProcess = gribProcessBuilder.start();
|
||||
gribCopyProcess.waitFor();
|
||||
|
||||
//重新设置reftime信息
|
||||
String gribSetFileAddr = gribCopyTargetDir+File.separator+"panguweather_"+LocalDateTimeUtil.format(startTime,"yyyyMMddHH")+".grib";
|
||||
String date = LocalDateTimeUtil.format(startTime,"yyyyMMdd");
|
||||
String time = LocalDateTimeUtil.format(startTime,"HHmm");
|
||||
List<String> gribSetCommand = new ArrayList<>();
|
||||
gribSetCommand.add(gribSetCommandPath);
|
||||
gribSetCommand.add("-s");
|
||||
gribSetCommand.add("dataDate="+date+",dataTime="+time+",endStep="+0);
|
||||
gribSetCommand.add(gribCopyFileAddr);
|
||||
gribSetCommand.add(gribSetFileAddr);
|
||||
gribProcessBuilder.command(gribSetCommand);
|
||||
log.info("gribSetCommand:{}", gribSetCommand);
|
||||
Process gribSetProcess = gribProcessBuilder.start();
|
||||
gribSetProcess.waitFor();
|
||||
i+=step;
|
||||
startTime = startTime.plusHours(step);
|
||||
}
|
||||
}catch (Exception e){
|
||||
throw new RuntimeException("分割grib文件异常",e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 转换grib
|
||||
* @param gribFileSuffix
|
||||
* @param inputPath
|
||||
* @param outputPath
|
||||
*/
|
||||
private void convertGrib(String gribFileSuffix,String inputPath,String outputPath){
|
||||
try {
|
||||
// python3 pangu_reformat.py --ini_dir /export/xe_bk_data/weather/pangu_format_script --output_dir /export/xe_bk_data/weather/pangu2 --input_dir /export/xe_bk_data/weather/pangu2 --start_date 20251120 --end_date 20251120
|
||||
//调用python脚本转换flexpart能识别的气象数据文件
|
||||
//删除带有_not_grib_set名称的文件，此文件还未设置reftime属性
|
||||
List<File> files = FileUtil.loopFiles(inputPath, file -> file.getName().contains(gribFileSuffix));
|
||||
if(!files.isEmpty()){
|
||||
for(File file:files){
|
||||
file.delete();
|
||||
}
|
||||
}
|
||||
//构建转换命令
|
||||
String convertScriptPath = systemStorageProperties.getPanguFormatScriptPath();
|
||||
String startDate = this.weatherTask.getStartDate().format(DateTimeFormatter.ofPattern("yyyyMMdd"));
|
||||
String endDate = this.weatherTask.getStartDate().atTime(this.weatherTask.getStartTime(), 0, 0).plusHours(this.weatherTask.getLeadTime()).format(DateTimeFormatter.ofPattern("yyyyMMdd"));
|
||||
List<String> convertCommand = new ArrayList<>();
|
||||
convertCommand.add(systemStorageProperties.getFormatScriptPythonEnv()+File.separator+"python3");
|
||||
convertCommand.add("pangu_reformat.py");
|
||||
convertCommand.add("--ini_dir");
|
||||
convertCommand.add(convertScriptPath);
|
||||
convertCommand.add("--output_dir");
|
||||
convertCommand.add(outputPath);
|
||||
convertCommand.add("--input_dir");
|
||||
convertCommand.add(inputPath);
|
||||
convertCommand.add("--start_date");
|
||||
convertCommand.add(startDate);
|
||||
convertCommand.add("--end_date");
|
||||
convertCommand.add(endDate);
|
||||
|
||||
String adapterGribLog = "文件切割结束，开始调用python脚本适配flexpart模型，执行命令为："+convertCommand;
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),adapterGribLog));
|
||||
|
||||
ProcessBuilder adapterGribProcessBuilder = new ProcessBuilder();
|
||||
adapterGribProcessBuilder.directory(new File(convertScriptPath));
|
||||
adapterGribProcessBuilder.command(convertCommand);
|
||||
Process adapterGribProcess = adapterGribProcessBuilder.start();
|
||||
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(adapterGribProcess.getInputStream(), "UTF-8"));
|
||||
//读取输出日志
|
||||
String line;
|
||||
while ((line = reader.readLine()) != null) {
|
||||
if(StrUtil.isNotBlank(line)){
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),line));
|
||||
}
|
||||
}
|
||||
boolean finished = adapterGribProcess.waitFor((5*3600),TimeUnit.SECONDS);
|
||||
if (!finished) {
|
||||
// 如果在指定时间内进程没有结束，则强制销毁它
|
||||
adapterGribProcess.destroyForcibly(); // 强制终止进程
|
||||
String destroyForciblyLog = "python脚本适配flexpart模型进程超时，终止进程";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),destroyForciblyLog));
|
||||
} else {
|
||||
int exitCode = adapterGribProcess.exitValue();
|
||||
String progressFinishedLog = "python脚本适配flexpart模型进程结束，exitCode："+exitCode;
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),progressFinishedLog));
|
||||
|
||||
}
|
||||
}catch (Exception e){
|
||||
throw new RuntimeException("适配flexpart，转换grib异常",e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 保存生成的气象数据存储到数据库
|
||||
* @param inputPath
|
||||
* @param targetPath
|
||||
*/
|
||||
private void saveGribInfoToDB(String inputPath,String targetPath){
|
||||
String handleGribLog = "格式转换结束，处理grib文件数据入库";
|
||||
ProgressQueue.getInstance().offer(new ProgressEvent(this.weatherTask.getId(),handleGribLog));
|
||||
|
||||
List<File> files = FileUtil.loopFiles(inputPath,file -> file.getName().startsWith("pangu_"));
|
||||
List<File> targetFiles = new ArrayList<>();
|
||||
if(!files.isEmpty()){
|
||||
for(File file:files){
|
||||
File targetFile = new File(targetPath+File.separator+file.getName());
|
||||
FileUtil.move(file,targetFile,true);
|
||||
if(targetFile.exists()){
|
||||
targetFiles.add(targetFile);
|
||||
}
|
||||
}
|
||||
for(File targetFile:targetFiles){
|
||||
//获取文件数据开始日期
|
||||
String reftime = NcUtil.getReftime(targetFile.getAbsolutePath());
|
||||
if(StringUtils.isBlank(reftime)) {
|
||||
throw new JeecgFileUploadException("解析气象文件起始时间数据异常，此文件可能损坏");
|
||||
}
|
||||
//计算文件大小M
|
||||
BigDecimal divideVal = new BigDecimal("1024");
|
||||
BigDecimal bg = new BigDecimal(targetFile.length());
|
||||
BigDecimal fileSize = bg.divide(divideVal).divide(divideVal).setScale(2, RoundingMode.HALF_UP);
|
||||
//处理文件数据开始时间
|
||||
Instant instant = Instant.parse(reftime);
|
||||
LocalDateTime utcDateTime = LocalDateTime.ofInstant(instant, ZoneId.of("UTC"));
|
||||
//计算文件MD5值
|
||||
String md5Val = "";
|
||||
try (InputStream is = new FileInputStream(targetFile.getAbsolutePath())) {
|
||||
md5Val = DigestUtils.md5Hex(is);
|
||||
}catch (Exception e){
|
||||
throw new RuntimeException(targetFile.getName()+"MD5值计算失败");
|
||||
}
|
||||
//校验文件是否存在，存在删除从新新增
|
||||
LambdaQueryWrapper<WeatherData> queryWrapper = new LambdaQueryWrapper<>();
|
||||
queryWrapper.eq(WeatherData::getFileName,targetFile.getName());
|
||||
WeatherData queryResult = weatherDataMapper.selectOne(queryWrapper);
|
||||
if(Objects.nonNull(queryResult)){
|
||||
weatherDataMapper.deleteById(queryResult.getId());
|
||||
}
|
||||
//构建文件信息
|
||||
WeatherData weatherData = new WeatherData();
|
||||
weatherData.setFileName(targetFile.getName());
|
||||
weatherData.setFileSize(fileSize.doubleValue());
|
||||
weatherData.setFileExt(targetFile.getName().substring(targetFile.getName().lastIndexOf(".")+1));
|
||||
weatherData.setDataStartTime(utcDateTime);
|
||||
weatherData.setDataSource(weatherTask.getPredictionModel());
|
||||
weatherData.setFilePath(targetFile.getAbsolutePath());
|
||||
weatherData.setMd5Value(md5Val);
|
||||
weatherData.setShareTotal(1);
|
||||
weatherDataMapper.insert(weatherData);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取盘古数据存储路径
|
||||
* @return
|
||||
*/
|
||||
private String getPanguWeatherPath(){
|
||||
StringBuilder sb = new StringBuilder();
|
||||
sb.append(this.systemStorageProperties.getRootPath());
|
||||
sb.append(File.separator);
|
||||
sb.append(this.systemStorageProperties.getPangu());
|
||||
return sb.toString();
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取Graphcast数据存储路径
|
||||
* @return
|
||||
*/
|
||||
private String getGraphcastWeatherPath(){
|
||||
StringBuilder sb = new StringBuilder();
|
||||
sb.append(this.systemStorageProperties.getRootPath());
|
||||
sb.append(File.separator);
|
||||
sb.append(this.systemStorageProperties.getGraphcast());
|
||||
return sb.toString();
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user