Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ビルド時のコアのバージョンを 0.12.0-preview.3 に上げる #401

Merged
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
209c1d8
各プラットフォーム別に新しく決められたアセット名のzipをダウンロード
PickledChair May 12, 2022
08ec35b
共有ライブラリの名称変更やmetas.json・モデルファイルがなくなったことを反映
PickledChair May 12, 2022
8a12fd4
VOICEVOX_CORE_ASSET_NAMEの定義場所を変更
PickledChair May 12, 2022
2a6e7dc
trivial change
PickledChair May 12, 2022
530c13f
Merge remote-tracking branch 'VOICEVOX/voicevox_engine/master' into b…
PickledChair May 12, 2022
77cb4d4
zip解凍後のフォルダ名を正しく指定する
PickledChair May 12, 2022
5f5e426
glob パターンはダブルクォートの外に出さないと機能しなかったので修正
PickledChair May 12, 2022
328fa1d
coreのアセットのzip解凍をunzipではなく7zで行う
PickledChair May 12, 2022
b354fcf
VERSIONファイルの移動を復活
PickledChair May 12, 2022
ed0b12e
現在のアセットにはREADMEが含まれていないのでREADMEのcatを削除
PickledChair May 12, 2022
4b65a33
Windows版におけるコアのzip解凍方法に関するコメントを追加
PickledChair May 12, 2022
ed2549a
build-docker.ymlをコア0.12.0-preview.3に対応させる
PickledChair May 12, 2022
14b16ee
version 0.12 以降のコアの読み込み時にPathをstrに変換する
PickledChair May 13, 2022
a910098
format
PickledChair May 13, 2022
027f2d2
CDLLクラスのコンストラクタの引数についてコメントを追加
PickledChair May 13, 2022
1d69a69
CDLLの引数についてさらに追記
PickledChair May 13, 2022
a44f48e
後でREADMEの追加・表示をしたい旨をコメント
PickledChair May 14, 2022
cb65954
コメントを修正
PickledChair May 14, 2022
ac5ce92
READMEのビルド方法の記載を更新
PickledChair May 14, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions .github/workflows/build-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:
env:
IMAGE_NAME: ${{ secrets.DOCKERHUB_USERNAME }}/voicevox_engine
PYTHON_VERSION: "3.8.10"
VOICEVOX_CORE_VERSION: "0.11.4"
VOICEVOX_CORE_VERSION: "0.12.0-preview.3"
VOICEVOX_ENGINE_VERSION:
|- # releaseのときはタグが、それ以外はlatestがバージョン名に
${{ github.event.release.tag_name != '' && github.event.release.tag_name || 'latest' }}
Expand All @@ -37,44 +37,44 @@ jobs:
target: runtime-env
base_image: ubuntu:focal
base_runtime_image: ubuntu:focal
voicevox_core_library_name: libcore_cpu_x64.so
voicevox_core_asset_prefix: voicevox_core-linux-x64-cpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-linux-x64-1.10.0.tgz
- tag: cpu
target: runtime-env
base_image: ubuntu:focal
base_runtime_image: ubuntu:focal
voicevox_core_library_name: libcore_cpu_x64.so
voicevox_core_asset_prefix: voicevox_core-linux-x64-cpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-linux-x64-1.10.0.tgz
- tag: cpu-ubuntu20.04
target: runtime-env
base_image: ubuntu:focal
base_runtime_image: ubuntu:focal
voicevox_core_library_name: libcore_cpu_x64.so
voicevox_core_asset_prefix: voicevox_core-linux-x64-cpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-linux-x64-1.10.0.tgz
- tag: nvidia
target: runtime-nvidia-env
base_image: ubuntu:focal
base_runtime_image: nvidia/cuda:11.4.2-cudnn8-runtime-ubuntu20.04
voicevox_core_library_name: libcore_gpu_x64_nvidia.so
voicevox_core_asset_prefix: voicevox_core-linux-x64-gpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-linux-x64-gpu-1.10.0.tgz
- tag: nvidia-ubuntu20.04
target: runtime-nvidia-env
base_image: ubuntu:focal
base_runtime_image: nvidia/cuda:11.4.2-cudnn8-runtime-ubuntu20.04
voicevox_core_library_name: libcore_gpu_x64_nvidia.so
voicevox_core_asset_prefix: voicevox_core-linux-x64-gpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-linux-x64-gpu-1.10.0.tgz
# Ubuntu 18.04
- tag: cpu-ubuntu18.04
target: runtime-env
base_image: ubuntu:bionic
base_runtime_image: ubuntu:bionic
voicevox_core_library_name: libcore_cpu_x64.so
voicevox_core_asset_prefix: voicevox_core-linux-x64-cpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-linux-x64-1.10.0.tgz
- tag: nvidia-ubuntu18.04
target: runtime-nvidia-env
base_image: ubuntu:bionic
base_runtime_image: nvidia/cuda:11.4.2-cudnn8-runtime-ubuntu18.04
voicevox_core_library_name: libcore_gpu_x64_nvidia.so
voicevox_core_asset_prefix: voicevox_core-linux-x64-gpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-linux-x64-gpu-1.10.0.tgz

steps:
Expand All @@ -100,6 +100,7 @@ jobs:
format('{0}:{1}-{2}', env.IMAGE_NAME, matrix.tag, env.VOICEVOX_ENGINE_VERSION)
) || format('{0}:{1}', env.IMAGE_NAME, env.VOICEVOX_ENGINE_VERSION)
) }}
VOICEVOX_CORE_ASSET_NAME: ${{ matrix.voicevox_core_asset_prefix }}-${{ env.VOICEVOX_CORE_VERSION }}
with:
context: .
builder: ${{ steps.buildx.outputs.name }}
Expand All @@ -109,8 +110,8 @@ jobs:
BASE_RUNTIME_IMAGE=${{ matrix.base_runtime_image }}
PYTHON_VERSION=${{ env.PYTHON_VERSION }}
VOICEVOX_ENGINE_VERSION=${{ env.VOICEVOX_ENGINE_VERSION }}
VOICEVOX_CORE_ASSET_NAME=${{ env.VOICEVOX_CORE_ASSET_NAME }}
VOICEVOX_CORE_VERSION=${{ env.VOICEVOX_CORE_VERSION }}
VOICEVOX_CORE_LIBRARY_NAME=${{ matrix.voicevox_core_library_name }}
ONNXRUNTIME_URL=${{ matrix.onnxruntime_url }}
target: ${{ matrix.target }}
push: true
Expand Down
56 changes: 34 additions & 22 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ env:
IMAGE_NAME: ${{ secrets.DOCKERHUB_USERNAME }}/voicevox_engine
PYTHON_VERSION: "3.8.10"
VOICEVOX_RESOURCE_VERSION: "0.11.4"
VOICEVOX_CORE_VERSION: "0.12.0-preview.0"
VOICEVOX_CORE_VERSION: "0.12.0-preview.3"
VOICEVOX_ENGINE_VERSION:
|- # releaseのときはタグが、それ以外はlatestがバージョン名に
${{ github.event.release.tag_name != '' && github.event.release.tag_name || 'latest' }}
Expand All @@ -26,7 +26,7 @@ jobs:
- os: macos-11
python_architecture: "x64"
pip_cache_path: ~/Library/Caches/pip
voicevox_core_library_name: libcore_cpu_universal2.dylib
voicevox_core_asset_prefix: voicevox_core-osx-universal2-cpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-osx-universal2-1.10.0.tgz
artifact_name: macos-x64

Expand Down Expand Up @@ -143,11 +143,16 @@ jobs:
- name: Download VOICEVOX Core release
if: steps.voicevox-core-cache.outputs.cache-hit != 'true'
shell: bash
env:
VOICEVOX_CORE_ASSET_NAME: ${{ matrix.voicevox_core_asset_prefix }}-${{ env.VOICEVOX_CORE_VERSION }}
run: |
# extracted like download/core/metas.json
curl -L "https://github.com/VOICEVOX/voicevox_core/releases/download/${{ env.VOICEVOX_CORE_VERSION }}/core.zip" > download/core.zip
ditto -x -k --sequesterRsrc --rsrc download/core.zip download/
rm download/core.zip
# extracted like download/core/libcore.dylib
curl -L "https://github.com/VOICEVOX/voicevox_core/releases/download/${{ env.VOICEVOX_CORE_VERSION }}/${{ env.VOICEVOX_CORE_ASSET_NAME }}.zip" > download/${{ env.VOICEVOX_CORE_ASSET_NAME }}.zip
ditto -x -k --sequesterRsrc --rsrc download/${{ env.VOICEVOX_CORE_ASSET_NAME }}.zip download/
mkdir -p download/core
mv download/${{ env.VOICEVOX_CORE_ASSET_NAME }}/* download/core
rm -rf download/${{ env.VOICEVOX_CORE_ASSET_NAME }}
rm download/${{ env.VOICEVOX_CORE_ASSET_NAME }}.zip

- name: Download PyOpenJTalk dictionary
shell: bash
Expand Down Expand Up @@ -193,9 +198,7 @@ jobs:
--include-data-file=../licenses.json=./ \
--include-data-file=../presets.yaml=./ \
--include-data-file=../default.csv=./ \
--include-data-file=../download/core/*.bin=./ \
--include-data-file=../download/core/metas.json=./ \
--include-data-file=../download/core/${{ matrix.voicevox_core_library_name }}=./ \
--include-data-file=../download/core/libcore.dylib=./ \
--include-data-file=../download/onnxruntime/lib/libonnxruntime.dylib=./ \
--include-data-file=${{ env.pythonLocation }}/lib/python*/site-packages/scipy/.dylibs/*.dylib=./scipy/.dylibs/ \
--include-data-file=${{ env.pythonLocation }}/lib/python*/site-packages/_soundfile_data/*=./_soundfile_data/ \
Expand Down Expand Up @@ -250,7 +253,7 @@ jobs:
target: build-env
base_image: ubuntu:bionic
base_runtime_image: ubuntu:bionic
voicevox_core_library_name: libcore_cpu_x64.so
voicevox_core_asset_prefix: voicevox_core-linux-x64-cpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-linux-x64-1.10.0.tgz
artifact_name: linux-cpu
nuitka_cache_path: nuitka_cache
Expand All @@ -259,7 +262,7 @@ jobs:
target: build-env
base_image: ubuntu:bionic
base_runtime_image: nvidia/cuda:11.4.2-cudnn8-runtime-ubuntu18.04
voicevox_core_library_name: libcore_gpu_x64_nvidia.so
voicevox_core_asset_prefix: voicevox_core-linux-x64-gpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-linux-x64-gpu-1.10.0.tgz
artifact_name: linux-nvidia
nuitka_cache_path: nuitka_cache
Expand Down Expand Up @@ -311,6 +314,7 @@ jobs:
env:
IMAGE_TAG: ${{ env.IMAGE_NAME }}:${{ matrix.tag }}${{ (matrix.tag != '' && '-') || '' }}latest
RUNTIME_IMAGE_TAG: ${{ env.IMAGE_NAME }}:${{ matrix.runtime_tag }}${{ (matrix.runtime_tag != '' && '-') || '' }}latest
VOICEVOX_CORE_ASSET_NAME: ${{ matrix.voicevox_core_asset_prefix }}-${{ env.VOICEVOX_CORE_VERSION }}
with:
context: .
builder: ${{ steps.buildx.outputs.name }}
Expand All @@ -320,8 +324,8 @@ jobs:
BASE_RUNTIME_IMAGE=${{ matrix.base_runtime_image }}
PYTHON_VERSION=${{ env.PYTHON_VERSION }}
VOICEVOX_ENGINE_VERSION=${{ env.VOICEVOX_ENGINE_VERSION }}
VOICEVOX_CORE_ASSET_NAME=${{ env.VOICEVOX_CORE_ASSET_NAME }}
VOICEVOX_CORE_VERSION=${{ env.VOICEVOX_CORE_VERSION }}
VOICEVOX_CORE_LIBRARY_NAME=${{ matrix.voicevox_core_library_name }}
ONNXRUNTIME_URL=${{ matrix.onnxruntime_url }}
target: ${{ matrix.target }}
load: true
Expand Down Expand Up @@ -372,7 +376,7 @@ jobs:
# Windows CPU
- os: windows-2019
architecture: "x64"
voicevox_core_dll_name: core_cpu_x64.dll
voicevox_core_asset_prefix: voicevox_core-windows-x64-cpu
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-win-x64-1.10.0.zip
ccache_url: https://github.com/ccache/ccache/releases/download/v4.4.1/ccache-4.4.1-windows-64.zip
artifact_name: windows-cpu
Expand All @@ -381,7 +385,7 @@ jobs:
# Windows DirectML
- os: windows-2019
architecture: "x64"
voicevox_core_dll_name: core_gpu_x64_directml.dll
voicevox_core_asset_prefix: voicevox_core-windows-x64-directml
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/Microsoft.ML.OnnxRuntime.DirectML.1.10.0.zip
directml_url: https://www.nuget.org/api/v2/package/Microsoft.AI.DirectML/1.8.0
ccache_url: https://github.com/ccache/ccache/releases/download/v4.4.1/ccache-4.4.1-windows-64.zip
Expand All @@ -391,7 +395,7 @@ jobs:
# Windows NVIDIA GPU
- os: windows-2019
architecture: "x64"
voicevox_core_dll_name: core_gpu_x64_nvidia.dll
voicevox_core_asset_prefix: voicevox_core-windows-x64-cuda
onnxruntime_url: https://github.com/microsoft/onnxruntime/releases/download/v1.10.0/onnxruntime-win-x64-gpu-1.10.0.zip
cuda_version: "11.4.2"
cudnn_url: https://developer.download.nvidia.com/compute/redist/cudnn/v8.2.4/cudnn-11.4-windows-x64-v8.2.4.15.zip
Expand Down Expand Up @@ -659,10 +663,20 @@ jobs:
- name: Download VOICEVOX Core
if: steps.voicevox-core-cache.outputs.cache-hit != 'true'
shell: bash
run: |
curl -L "https://github.com/VOICEVOX/voicevox_core/releases/download/${{ env.VOICEVOX_CORE_VERSION }}/core.zip" > download/core.zip
unzip download/core.zip -d download/
rm download/core.zip
env:
VOICEVOX_CORE_ASSET_NAME: ${{ matrix.voicevox_core_asset_prefix }}-${{ env.VOICEVOX_CORE_VERSION }}
run: |
curl -L "https://github.com/VOICEVOX/voicevox_core/releases/download/${{ env.VOICEVOX_CORE_VERSION }}/${{ env.VOICEVOX_CORE_ASSET_NAME }}.zip" > download/${{ env.VOICEVOX_CORE_ASSET_NAME }}.zip
# NOTE: Windows 版コアのみ PowerShell の Compress-Archive コマンドレットを用いて zip を作成している(デフォルト状態では zip コマンドが存在していないため)。
# このコマンドはバージョンによっては作成した zip 内のパスの区切り文字がバックスラッシュになる。 (cf. https://github.com/PowerShell/Microsoft.PowerShell.Archive/issues/48)
# unzip コマンドはこのような zip ファイルを解凍できるものの、終了コード 1 を報告して CI が落ちてしまう。
# 回避策として、unzip コマンドの代わりに 7z コマンドを用いて zip ファイルを解凍する。
# unzip download/${{ env.VOICEVOX_CORE_ASSET_NAME }}.zip -d download/
7z x -o"download" download/${{ env.VOICEVOX_CORE_ASSET_NAME }}.zip
mkdir -p download/core
mv download/${{ env.VOICEVOX_CORE_ASSET_NAME }}/* download/core
rm -rf download/${{ env.VOICEVOX_CORE_ASSET_NAME }}
rm download/${{ env.VOICEVOX_CORE_ASSET_NAME }}.zip

- name: Generate licenses.json
shell: bash
Expand Down Expand Up @@ -709,10 +723,8 @@ jobs:
--include-data-file="licenses.json=./" \
--include-data-file="default.csv=./" \
--include-data-file="presets.yaml=./" \
--include-data-file=download/core/*.bin=./ \
--include-data-file="download/core/metas.json=./" \
--include-data-file="download/onnxruntime/lib/onnxruntime.dll=./" \
--include-data-file="download/core/${{ matrix.voicevox_core_dll_name }}=./" \
--include-data-file="download/core/core.dll=./" \
--include-data-dir="speaker_info=./speaker_info" \
--include-data-dir="manifest_assets=./manifest_assets" \
--msvc=14.2 \
Expand Down
24 changes: 10 additions & 14 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -21,25 +21,25 @@ RUN <<EOF
EOF

# assert VOICEVOX_CORE_VERSION >= 0.11.0 (ONNX)
ARG VOICEVOX_CORE_VERSION=0.11.4
ARG VOICEVOX_CORE_LIBRARY_NAME=libcore_cpu_x64.so
ARG VOICEVOX_CORE_ASSET_NAME=voicevox_core-linux-x64-cpu-0.12.0-preview.3
ARG VOICEVOX_CORE_VERSION=0.12.0-preview.3
RUN <<EOF
set -eux

# Download Core
wget -nv --show-progress -c -O "./core.zip" "https://github.com/VOICEVOX/voicevox_core/releases/download/${VOICEVOX_CORE_VERSION}/core.zip"
unzip "./core.zip"
rm ./core.zip
wget -nv --show-progress -c -O "./${VOICEVOX_CORE_ASSET_NAME}.zip" "https://github.com/VOICEVOX/voicevox_core/releases/download/${VOICEVOX_CORE_VERSION}/${VOICEVOX_CORE_ASSET_NAME}.zip"
unzip "./${VOICEVOX_CORE_ASSET_NAME}.zip"
mkdir -p core
mv "${VOICEVOX_CORE_ASSET_NAME}"/* core
rm -rf $VOICEVOX_CORE_ASSET_NAME
rm "./${VOICEVOX_CORE_ASSET_NAME}.zip"

# Move Core Library to /opt/voicevox_core/
mkdir /opt/voicevox_core
mv "./core/${VOICEVOX_CORE_LIBRARY_NAME}" /opt/voicevox_core/

# Move Voice Library to /opt/voicevox_core/
mv ./core/*.bin ./core/metas.json /opt/voicevox_core/
mv "./core/libcore.so" /opt/voicevox_core/

# Move documents to /opt/voicevox_core/
mv ./core/README.txt ./core/VERSION /opt/voicevox_core/
mv ./core/VERSION /opt/voicevox_core/

rm -rf ./core

Expand Down Expand Up @@ -249,8 +249,6 @@ COPY --chmod=775 <<EOF /entrypoint.sh
#!/bin/bash
set -eux

cat /opt/voicevox_core/README.txt > /dev/stderr

exec "\$@"
Hiroshiba marked this conversation as resolved.
Show resolved Hide resolved
EOF

Expand Down Expand Up @@ -333,8 +331,6 @@ RUN <<EOF
--include-data-file=/opt/voicevox_engine/licenses.json=./ \
--include-data-file=/opt/voicevox_engine/presets.yaml=./ \
--include-data-file=/opt/voicevox_engine/default.csv=./ \
--include-data-file=/opt/voicevox_core/*.bin=./ \
--include-data-file=/opt/voicevox_core/metas.json=./ \
--include-data-file=/opt/voicevox_core/*.so=./ \
--include-data-file=/opt/onnxruntime/lib/libonnxruntime.so=./ \
--include-data-dir=/opt/voicevox_engine/speaker_info=./speaker_info \
Expand Down
6 changes: 5 additions & 1 deletion voicevox_engine/synthesis_engine/core_wrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,11 @@ def check_core_type(core_dir: Path) -> Optional[str]:
def load_core(core_dir: Path, use_gpu: bool) -> CDLL:
if is_version_0_12_core_or_later(core_dir):
try:
return CDLL((core_dir / CORENAME_DICT[platform.system()]).resolve(True))
# NOTE: CDLL クラスのコンストラクタの引数 name には文字列を渡す必要がある。
# Windows 環境では PathLike オブジェクトを引数として渡すと初期化に失敗する。
return CDLL(
str((core_dir / CORENAME_DICT[platform.system()]).resolve(strict=True))
)
Comment on lines +295 to +299
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

前に #388 を実装したときに、Windows 版の新しいコアが読み込めないバグを埋め込んでいたことに気づいたので、合わせて修正しました(最後のエンジンの実行テストに Windows 版だけ失敗したので気がつきました……)。

except OSError as err:
raise RuntimeError(f"コアの読み込みに失敗しました:{err}")

Expand Down