Merge pull request #5 from wangerzi/dev

v1.4 update

Dockerfile

@@ -1,6 +1,7 @@
-# docker build -t wj2015/3d-model-convert-to-gltf:v1.3
+# docker build . -t wj2015/3d-model-convert-to-gltf:v1.4
+# startup docker by: docker run -d -p 8999:8999 wj2015/3d-model-convert-to-gltf:latest
 # you can debug by: docker run -it --rm -v `pwd`:/opt/3d-model-convert-to-gltf/ wj2015/3d-model-convert-to-gltf:latest /bin/bash
-# and run `conda activate pythonocc` to enter the environment.
+# you can also execute `conda activate pythonocc` to enter the environment.
 FROM continuumio/anaconda:2019.10
 LABEL [email protected]
 
@@ -25,4 +26,4 @@ COPY . /opt/3d-model-convert-to-gltf
 RUN cd /opt/3d-model-convert-to-gltf && \
     conda run -n pythonocc pip install -r server/requirements.txt
 WORKDIR /opt/3d-model-convert-to-gltf
-# CMD ['python', '/opt/3d-model-convert-to-gltf/server/main.py']
+CMD conda run -n pythonocc python /opt/3d-model-convert-to-gltf/server/rpc_server.py

README.md

@@ -1,6 +1,10 @@
 # 3DModelConvertToGltf - An Unified Model Format Conversion Tool
 
-The main reason for this project is that I encountered a scenario where **the STEP and IGES models need to be displayed on the Web**, but the web3d class libraries on the market do not support this format, and the direct display of STL files uploaded by users will consume a lot of bandwidth or CDN Traffic, converted to compressed gltf would be more appropriate.
+## Document
+
+English|[中文](README_ZH.md)
+
+The main reason for this project is that I encountered a scenario where **the STEP and IGES models need to be displayed on the Web**, but the web3d  libraries on the market do not support this format, and the direct display of STL files uploaded by users will consume a lot of bandwidth or CDN Traffic, converted to compressed gltf would be more appropriate.
 
 Demo assets model effect compare:
 
@@ -19,11 +23,7 @@ I organized my thoughts into a blog: [STEP and IGES models are converted to the
 
 > PS: My blog is write by Chinese, if you are non-Chinese native speaker, you should take a Google Translate tool for well.
 
-**Project status:** maintain
-
-## Document
-
-English|[中文](README_ZH.md)
+**Project status:** stable
 
 
 ## Mission
@@ -36,7 +36,23 @@ English|[中文](README_ZH.md)
 - [x] write easy to use convert.sh
 - [x] online convert preview
 - [ ] [bug] stp convert to gltf is too large
-- [ ] grpc support
+- [x] grpc support
+- [ ] rpc should response error detail
+- [ ] rpc docker server logs output problems
+
+## Version update
+
+##### v1.4 2021-06-11 17:20
+
+Support GRPC, convert code refactor, fix bugs.
+
+##### v1.3 2020-06-24 17:19
+
+Add English document, fix bugs.
+
+##### v1.0 2020-05-21 19:08
+
+Basic commit, core feature complete, support shell convert.
 
 
 ## Why not assmip
@@ -45,23 +61,43 @@ I tried to use [assimp](https://github.com/assimp/assimp), but the result under
 
 ## Why not implement API in this project
 
-Model conversion is a very performance-consuming and slow-speed service. The upload and download of the model will consume bandwidth. **If it is deployed directly on your own server, it will be a very bandwidth-intensive and CPU-consuming task**. For the most common method to upload and download large files is to **introduce OSS and CDN with dynamic expansion of queues and back-end services**, but the deployment cost and implementation cost will be relatively high, please contact [email protected] for business needs Commercial API support.
+Model conversion is a very performance-consuming and slow-speed service. The upload and download of the model will consume bandwidth. **If it is deployed directly on your own server, it will be a very bandwidth-intensive and CPU-consuming task**. For the most common method to upload and download large files is to **introduce OSS and CDN with dynamic expansion of queues and back-end services**, but the deployment cost and implementation cost will be relatively high.
 
 ## Quick Start
 
-Due to the trouble of environment configuration and other reasons, the command line mode **still needs to rely on docker**. **The command line mode is suitable for simple invocation on the server side.** The conversion process blocks the processes to be synchronized and cannot be deployed in a distributed manner to increase concurrency.
+Due to the trouble of environment configuration and other reasons, the command line mode **still needs to rely on docker**. **The command line mode is suitable for simple invocation on the server side.** The conversion process blocks the processes to be synchronized and cannot be deployed in a distributed manner to increase concurrency.The most recommended way is to use **grpc with docker** deployment to make rpc, which can be synchronous or asynchronous, and will be **easy to extend**.
 
 > PS：When there are too many simultaneous conversion models in the command line mode or a single model is too large, there is a risk that the server providing the web service is stuck
 
 ### Online convert previewer
 
 You can convert model online (<100MB) powered by [modelbox-sdk](https://github.com/wangerzi/modelbox-sdk)，preview link: [https://wangerzi.gitee.io/modelbox-sdk/examples/index.html](https://wangerzi.gitee.io/modelbox-sdk/examples/index.html)
 
+### GRPC Mode
+
+Based on GRPC, and it will be more convient to **build a dynamically expanded service cluster**, we support uploading zip/model source files, for the compatibility of each model, **response files are all in zip**, you need to decompress it after got it.
+
+You should run server-side rpc service by docker, please make sure 8999 port is usable and `wj2015/3d-model-convert-to-gltf:latest` image is up to date, command:
+
+```shell
+docker run -d -p 8999:8999 wj2015/3d-model-convert-to-gltf:latest
+```
+
+When using grpc in this project, please copy  `server/rpc/protos/converter.proto` , and generate a code template according to the language of caller and enjoy it. Official document: [Support Language](https://grpc.io/docs/languages/)
+
+#### Completed examples
+
+If this project is helpful to you, you can commit another examples and PR, such as php/golang/Nodejs rpc call examples.
+
+| name                            | code                                 | comments         |
+| ------------------------------- | ------------------------------------ | ---------------- |
+| Python rpc client usage example | server/examples/python/rpc_client.py | convert and save |
+
 ### Command Mode
 
 Download the  `convert.sh`, and grant execution authority, execute the following command, the second param should choose in `stl|stp|iges|obj|fbx`, please determine according to the file type 
 
-> The script depends on the docker environment, so you should prepare the Docker environment first.
+> The script depends on the docker environment, so you should prepare the Docker environment first. **Command mode is not support zip file convert**, beacause docker volume will auto sync picture or mtl assets to docker container.
 
 ```shell
 convert.sh stl inputpath.stl outputpath.glb # convert to glb single bin file

README_ZH.md

@@ -1,5 +1,9 @@
 # 3DModelConvertToGltf 统一模型格式转换工具
 
+## 文档
+
+中文|[English](README.md)
+
 此项目产生的主要原因是工作中遇到了需要**在Web中展示 STEP 和 IGES 模型的场景**，但是市面上的web3d类库均不支持此格式，并且用户上传的STL文件直接展示会占用大量带宽或CDN流量，转换为压缩后的gltf会比较合适。
 
 样例文件压缩效果如下表：
@@ -17,11 +21,7 @@
 
 本项目即采用了博客中总结的思路：[STEP和IGES模型转换为适用Web的glb格式](https://blog.wj2015.com/2020/03/08/step%e5%92%8ciges%e6%a8%a1%e5%9e%8b%e8%bd%ac%e6%8d%a2%e4%b8%ba%e9%80%82%e7%94%a8web%e7%9a%84glb%e6%a0%bc%e5%bc%8f/)
 
-**项目状态：** 维护中
-
-## 文档
-
-中文|[English](README.md)
+**项目状态：** 稳定维护中
 
 ## 待完成任务
 
@@ -33,7 +33,23 @@
 - [x] 一键转换脚本封装
 - [x] 在线转换预览
 - [ ] [bug] stp 转 gltf 最终文件太大
-- [ ] 支持以 grpc 形式调用
+- [x] 支持以 grpc 形式调用
+- [ ] rpc 接口优化，返回具体的错误信息
+- [ ] rpc server 日志问题
+
+## 版本说明
+
+##### v1.4 2021-06-11 17:20
+
+更新了 GRPC 的支持，清理 aiohttp 和配置代码，重构了转换部分的代码并修复了 BUG
+
+##### v1.3 2020-06-24 17:19
+
+新增英文文档，修复 BUG，尝试 aiohttp 写接口
+
+##### v1.0 2020-05-21 19:08
+
+基础提交，基本功能开发完毕，支持脚本调用
 
 ## 为什么不用 assmip
 
@@ -45,19 +61,39 @@
 
 ## 快速上手
 
-由于环境配置麻烦等原因，命令行模式依旧需要依赖docker，**命令行模式适合服务端简单调用**，转换过程阻塞进程同步进行，无法分布式部署增加并发量等
+由于环境配置麻烦等原因，命令行模式依旧需要依赖docker，**命令行模式适合服务端简单调用**，转换过程阻塞进程同步进行，无法分布式部署增加并发量；最推荐的方式就是用 **grpc 配合容器化部署做调用**，可同步可异步，可方便的做扩展。
 
 > PS：命令行模式同步转换模型过多或者单个模型过大时，有把提供Web服务的服务器卡住的风险
 
 ### 在线转换预览
 
 可以使用 [modelbox-sdk](https://github.com/wangerzi/modelbox-sdk) 在线转换模型(<100MB)，链接：[https://wangerzi.gitee.io/modelbox-sdk/examples/index.html](https://wangerzi.gitee.io/modelbox-sdk/examples/index.html)
 
+### GRPC 模式
+
+基于 GRPC 实现了服务内部的 RPC 通信，构建动态扩容的服务集群会更加方便，**支持上传 zip/模型源文件**，为各模型的兼容性考虑，**响应的文件都是 zip 格式的**，调用后需自行解压。
+
+首先需要**通过 docker 运行 RPC 服务端**，直接运行的指令如下，注意保证 8999 端口未被占用且 `wj2015/3d-model-convert-to-gltf:latest` 镜像最新
+
+```shell
+docker run -d -p 8999:8999 wj2015/3d-model-convert-to-gltf:latest
+```
+
+使用时，请将 `server/rpc/protos/converter.proto` 复制出来，并根据调用方变成语言生成模板并使用，官方网址：[支持的语言](https://grpc.io/docs/languages/)
+
+#### 已实现示例
+
+如果本项目对您有帮助，您可继续补充其他示例并提交 PR，比如常见的 php 、 golang 、Nodejs 的调用示例
+
+| 示例名称        | 目录                                 | 备注           |
+| --------------- | ------------------------------------ | -------------- |
+| Python 调用示例 | server/examples/python/rpc_client.py | 调用封装和保存 |
+
 ### 命令行模式
 
 下载代码中的 `convert.sh`，赋予执行权限，执行如下指令即可，第二个参数可支持 `stl|stp|iges|obj|fbx`，根据文件类型而定。
 
-> 脚本依赖于docker环境，所以 Docker 环境先准备好吧。
+> 脚本依赖于docker环境，所以 Docker 环境先准备好吧，而且命令行依赖的是 docker 的 -v 映射本地目录从而将贴图和模型一起导入到容器中执行转换，所以**不接受压缩包**，请直接指定需要转换的模型文件。
 
 ```shell
 convert.sh stl inputpath.stl outputpath.glb # 生成二进制glb文件
@@ -92,25 +128,6 @@ if (file_exists($out)) {
 }
 ```
 
-## 配置说明
-
-下列为默认配置 `server/config/app.yaml` ，请按需更改，如果使用docker需要映射配置文件
-
-```yaml
-app:
-    # 保存临时文件（原模型文件）
-    save_upload_temp_file: 1
-    # 保存转换过程文件（模型格式转换文件）
-    save_convert_temp_file: 0
-    # 后台并发处理数量（仅api）
-    background_process_num: 3
-# 上传路径配置（仅 API）
-upload:
-    path: uploads/
-    # 单位： Mb
-    maxsize: 30
-```
-
 ### Docker运行
 
 在宿主机安装好 `docker` 的条件下，运行如下指令获取镜像（大约4G）
@@ -123,7 +140,7 @@ docker pull wj2015/3d-model-convert-to-gltf
 
 ### 简单负载示意图
 
-如果有多机负载的需求，可借助 nginx 的反向代理做一下简单的负载均衡或者辅助消息队列以及生产者消费者来做，HTTP 服务或队列部分需要自己实现逻辑。
+如果有**多机负载**的需求，可借助 nginx 的反向代理、微服务的服务注册和调用轮训来做简单的负载均衡，还可以辅助消息队列以及生产者消费者，**其中 grpc 已内置实现并支持容器化部署**，如需使用 HTTP 服务或队列需要自己实现逻辑。
 
 ![1583754967257](assets/1583754967257.png)
 

convert.sh

@@ -10,4 +10,4 @@ outPath=$(
   pwd
 )
 outFile=$outPath/`basename $3`
-docker run -v $inputPath:$inputPath -v $outPath:$outPath wj2015/3d-model-convert-to-gltf:latest /bin/bash -c "cd $inputPath && conda run -n pythonocc python /opt/3d-model-convert-to-gltf/server/convert.py $1 $inputFile $outFile"
+docker run -v $inputPath:$inputPath -v $outPath:$outPath wj2015/3d-model-convert-to-gltf:v1.4 /bin/bash -c "cd $inputPath && conda run -n pythonocc python /opt/3d-model-convert-to-gltf/server/convert.py $1 $inputFile $outFile"

environment/docker-compose.yaml

@@ -2,13 +2,12 @@
 version: '3'
 services:
   app:
-    image: wj2015/3d-model-convert-to-gltf
+    image: wj2015/3d-model-convert-to-gltf:v1.4
     container_name: 3d-model-convert-to-gltf-app
     volumes:
       - ../:/opt/3d-model-convert-to-gltf:cached
     privileged: true
+    ports:
+      - 8999:8999
     command:
-      'conda run -n pythonocc python server/main.py'
-  redis:
-    image: redis
-    container_name: 3d-model-convert-to-gltf-redis
+      'conda run --no-capture-output -n pythonocc python server/rpc_server.py > /dev/stdout'