[WIP] エンジンのモック作成＋それを使ったコンポーネントテスト #2152
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
テーマ周りで気になった挙動まとめ
|
|
とりあえずスナップショットテストができた！ |
Hiroshiba
force-pushed
the
エンジンのmockを作る
branch
from
e9a6b07 to
1240838
Compare
Hiroshiba
force-pushed
the
エンジンのmockを作る
branch
from
a69ad2a to
5a80811
Compare
|
いろいろ試して、とりあえずトーク＆ソングのモックを実装し、トーク＆ソングエディタで音声（電子音）を再生できるところまで作りました。
SingEditor/TalkEditorをテストするのは流石にe2eになるかなぁという印象です。playwrigthとか。 エンジンのmock（正確にはVuex内のエンジン関数のmock）はいろいろ役立つと思うので、何かしらの形で実装しようかなと思ってます。 |
|
ただ消すのもちょっともったいないので、コードとしてここに残しておこうと思います。 `.storybook/main.ts`import type { StorybookConfig } from "@storybook/vue3-vite";
import { assetsPath, dicPath } from "@/mock/engineMock/constants";
const config: StorybookConfig = {
stories: ["../src/**/*.mdx", "../src/**/*.stories.@(js|jsx|mjs|ts|tsx)"],
addons: [
"@storybook/addon-links",
"@storybook/addon-essentials",
"@chromatic-com/storybook",
"@storybook/addon-interactions",
"@storybook/addon-themes",
],
core: {
builder: "@storybook/builder-vite",
},
framework: {
name: "@storybook/vue3-vite",
options: {
docgen: "vue-component-meta",
},
},
staticDirs: [
// モックエンジン用のファイル
{ from: "../node_modules/kuromoji/dict", to: dicPath },
{ from: "../tests/assets", to: assetsPath },
],
};
export default config;`.storybook/test-runner.ts`import { type TestRunnerConfig } from "@storybook/test-runner";
const config: TestRunnerConfig = {
async preVisit(page) {
// テスト用のスナップショット関数を追加する。
// *.stories.ts内で`window.storybookTestSnapshot`を使って呼び出せる。
if (await page.evaluate(() => !("storybookTestSnapshot" in window))) {
await page.exposeBinding(
"storybookTestSnapshot",
async (_, obj: unknown) => {
expect(obj).toMatchSnapshot();
},
);
}
},
};
export default config;`.storybook\preview-head.html`<!-- %BROWSER_PRELOAD% -->
<!-- FIXME: 色取得のために必要。DIできるようにしたい。 -->
<script type="module" src="/src/backend/browser/preload.ts"></script>`src\components\Talk\TalkEditor.stories.ts`import { userEvent, within, expect, fn, waitFor } from "@storybook/test";
import { Meta, StoryObj } from "@storybook/vue3";
import { provide, toRaw } from "vue";
import TalkEditor from "./TalkEditor.vue";
import { createStoreWrapper, storeKey } from "@/store";
import { HotkeyManager, hotkeyManagerKey } from "@/plugins/hotkeyPlugin";
import { createOpenAPIEngineMock, mockHost } from "@/mock/engineMock";
import { proxyStoreCreator } from "@/store/proxy";
import {
CharacterInfo,
defaultHotkeySettings,
DefaultStyleId,
EngineId,
EngineInfo,
SpeakerId,
StyleId,
ThemeConf,
} from "@/type/preload";
import { getEngineManifestMock } from "@/mock/engineMock/manifestMock";
import {
getSpeakerInfoMock,
getSpeakersMock,
} from "@/mock/engineMock/speakerResourceMock";
import { setFont, themeToCss } from "@/domain/dom";
import defaultTheme from "@/../public/themes/default.json";
import { cloneWithUnwrapProxy } from "@/helpers/cloneWithUnwrapProxy";
import { assetsPath } from "@/mock/engineMock/constants";
const meta: Meta<typeof TalkEditor> = {
component: TalkEditor,
args: {
isEnginesReady: true,
isProjectFileLoaded: false,
onCompleteInitialStartup: fn(),
},
decorators: [
(story, context) => {
// CSS関連
themeToCss(defaultTheme as ThemeConf);
setFont("default");
// ショートカットキーの管理
const hotkeyManager = new HotkeyManager();
provide(hotkeyManagerKey, hotkeyManager);
hotkeyManager.load(defaultHotkeySettings);
hotkeyManager.onEditorChange("talk");
// setup store
const store = createStoreWrapper({
proxyStoreDI: proxyStoreCreator(createOpenAPIEngineMock()),
});
provide(storeKey, store);
// なぜか必要、これがないとdispatch内でcommitしたときにエラーになる
store.replaceState({
...cloneWithUnwrapProxy(store.state),
});
context.parameters.vuexState = store.state;
// エンジンの情報
const engineManifest = getEngineManifestMock();
const engineId = EngineId(engineManifest.uuid);
const engineInfo: EngineInfo = {
uuid: engineId,
host: mockHost,
name: engineManifest.name,
path: undefined,
executionEnabled: false,
executionFilePath: "not_found",
executionArgs: [],
isDefault: true,
type: "path",
};
store.commit("SET_ENGINE_INFOS", {
engineIds: [engineId],
engineInfos: [engineInfo],
});
store.commit("SET_ENGINE_MANIFESTS", {
engineManifests: { [engineId]: engineManifest },
});
store.commit("SET_ENGINE_SETTING", {
engineId,
engineSetting: {
outputSamplingRate: engineManifest.defaultSamplingRate,
useGpu: false,
},
});
store.commit("SET_ENGINE_STATE", { engineId, engineState: "READY" });
// キャラクター情報
const speakers = getSpeakersMock();
const characterInfos: CharacterInfo[] = speakers.map((speaker) => {
const speakerInfo = getSpeakerInfoMock(speaker.speakerUuid, assetsPath);
return {
portraitPath: speakerInfo.portrait,
metas: {
speakerUuid: SpeakerId(speaker.speakerUuid),
speakerName: speaker.name,
styles: speakerInfo.styleInfos.map((styleInfo) => {
const style = speaker.styles.find((s) => s.id === styleInfo.id);
if (style == undefined) throw new Error("style not found");
return {
styleName: style.name,
styleId: StyleId(style.id),
styleType: style.type,
iconPath: styleInfo.icon,
portraitPath: styleInfo.portrait ?? speakerInfo.portrait,
engineId,
voiceSamplePaths: styleInfo.voiceSamples,
};
}),
policy: speakerInfo.policy,
},
};
});
store.commit("SET_CHARACTER_INFOS", { engineId, characterInfos });
store.commit("SET_USER_CHARACTER_ORDER", {
userCharacterOrder: store.state.characterInfos[engineId].map(
(c) => c.metas.speakerUuid,
),
});
// デフォルトスタイルID
const defaultStyleIds: DefaultStyleId[] = speakers.map((speaker) => ({
engineId: engineId,
speakerUuid: SpeakerId(speaker.speakerUuid),
defaultStyleId: StyleId(speaker.styles[0].id),
}));
store.commit("SET_DEFAULT_STYLE_IDS", { defaultStyleIds });
return story();
},
],
};
export default meta;
type Story = StoryObj<typeof meta>;
export const Default: Story = {
name: "デフォルト",
play: async ({ args }) => {
// 準備が完了するまで待機する
await waitFor(
() => expect(args.onCompleteInitialStartup).toHaveBeenCalled(),
{ timeout: 5000 },
);
},
};
export const NowLoading: Story = {
name: "プロジェクトファイルを読み込み中",
args: {
isProjectFileLoaded: "waiting",
},
};
export const TextInput: Story = {
name: "テキスト入力のテスト",
play: async ({ context, canvasElement, parameters }) => {
await Default.play?.(context);
const canvas = within(canvasElement);
// テキスト欄に入力
const textInput = await canvas.findByLabelText("1行目");
await userEvent.type(textInput, "こんにちは、これはテストです。{enter}");
const { audioItems, audioKeys } = parameters.vuexState;
await window.storybookTestSnapshot?.({ audioItems, audioKeys });
},
};
export const TextPaste: Story = {
name: "テキストペーストのテスト",
play: async ({ context, canvasElement, parameters }) => {
await Default.play?.(context);
const canvas = within(canvasElement);
// テキスト欄に入力
const textInput = await canvas.findByLabelText("1行目");
await userEvent.click(textInput);
await userEvent.paste("改行で改行\n読点で改行。最後の読点は改行しない。");
const { audioItems, audioKeys } = parameters.vuexState;
await window.storybookTestSnapshot?.({ audioItems, audioKeys });
},
};`src/type/globals.d.ts` // Storybookのtest-runnerのみで使用できるスナップショット関数
storybookTestSnapshot?: (obj: unknown) => Promise<void>;
```</details> |
|
メモ
|
AudioContextはシーケンスの作成で使用しています、この処理を関数化して外に出すPR（#2275）を作ってみました。 |
|
おおなるほどです！！ありがとうございます！！ |
|
適当にノートを足してRENDERしてみた感じ、ちょっとさすがAudioContextがないと大変そうだなとなりました！！ ちなみに書いたコードはこんな感じです。 import { createStoreWrapper } from "@/store";
import { NoteId, TrackId } from "@/type/preload";
import { resetMockMode, uuid4 } from "@/helpers/random";
import { cloneWithUnwrapProxy } from "@/helpers/cloneWithUnwrapProxy";
import { createDefaultTrack } from "@/sing/domain";
import { proxyStoreCreator } from "@/store/proxy";
import { createOpenAPIEngineMock } from "@/mock/engineMock";
import { SandboxKey, Sandbox } from "@/type/preload";
const store = createStoreWrapper({
proxyStoreDI: proxyStoreCreator(createOpenAPIEngineMock()),
});
const initialState = cloneWithUnwrapProxy(store.state);
beforeEach(() => {
store.replaceState(initialState);
resetMockMode();
});
describe("RENDER", async () => {
// FIXME: あとで汎用的にする
// @ts-expect-error mockのためにreadonlyに代入している
window[SandboxKey] = {
logInfo: (...args: unknown[]) => {
console.log("[logInfo]", ...args);
},
logWarn: (...args: unknown[]) => {
console.warn("[logWarn]", ...args);
},
logError: (...args: unknown[]) => {
console.error("[logError]", ...args);
},
};
it("空のトラックをレンダリングできる", async () => {
const { trackId, track } = await store.actions.CREATE_TRACK();
store.mutations.INSERT_TRACK({
trackId,
track,
prevTrackId: undefined,
});
await store.actions.RENDER();
await vi.waitFor(() => {
if (store.state.nowRendering) {
throw new Error("now rendering");
}
});
});
it("ノートがあるトラックをレンダリングできる", async () => {
const { trackId, track } = await store.actions.CREATE_TRACK();
track.notes.push({
id: NoteId(uuid4()),
position: 0,
duration: 1,
noteNumber: 60,
lyric: "あ",
});
store.mutations.INSERT_TRACK({
trackId,
track,
prevTrackId: undefined,
});
await store.actions.RENDER();
await vi.waitFor(() => {
if (store.state.nowRendering) {
throw new Error("now rendering");
}
});
});
}); |
|
ノート IDのプルリクを分けました！ テーマ周りもプルリグ分けました！ |
Hiroshiba
commented
Oct 12, 2024
|
📝 kuromojiはデフォルトで（デバッグ用に）入ってる感じで良さそう。 |
|
playwrightの通信をモックする形でエンジンAPIを偽装していたけど、VuexにDIする形にするならこれが不要になるはず。 await page.route(/\/version$/, async (route) => {
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify("mock"),
});
});
await page.route(/\/engine_manifest$/, async (route) => {
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(EngineManifestToJSON(getEngineManifestMock())),
});
});
await page.route(/\/supported_devices$/, async (route) => {
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(
SupportedDevicesInfoToJSON({ cpu: true, cuda: false, dml: false }),
),
});
});
await page.route(new RegExp(`/${assetsPath}/`), async (route) => {
const filePath = path.join(
__dirname,
"..",
"..",
"..",
new URL(route.request().url()).pathname,
);
const body = await fs.readFile(filePath);
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "image/png",
},
body,
});
});
await page.route(/\/speakers$/, async (route) => {
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(speakers.map(SpeakerToJSON)),
});
});
await page.route(/\/speaker_info\?/, async (route) => {
const query = new URLSearchParams(route.request().url().split("?")[1]);
const speakerUuid = query.get("speaker_uuid");
if (speakerUuid == null) {
throw new Error("speaker_uuid is required");
}
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(SpeakerInfoToJSON(getSpeakerInfoMock(speakerUuid))),
});
});
await page.route(/\/singers$/, async (route) => {
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(singers.map(SpeakerToJSON)),
});
});
await page.route(/\/singer_info\?/, async (route) => {
const payload = new URLSearchParams(new URL(route.request().url()).search);
const speakerUuid = payload.get("speaker_uuid");
if (speakerUuid == undefined) {
throw new Error("speaker_uuid is required");
}
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(SpeakerInfoToJSON(getSpeakerInfoMock(speakerUuid))),
});
});
await page.route(/\/is_initialized_speaker/, async (route) => {
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(true),
});
});
await page.route(/\/initialize_speaker/, async (route) => {
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
});
});
// NOTE: 空のユーザ辞書を返す
await page.route(/\/user_dict$/, async (route) => {
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify([]),
});
});
await page.route(/\/audio_query/, async (route) => {
const payload = new URLSearchParams(new URL(route.request().url()).search);
const text = payload.get("text");
const speaker = Number(payload.get("speaker"));
if (text == undefined || speaker == undefined) {
throw new Error("text, speaker is required");
}
const accentPhrases = await textToActtentPhrasesMock(text, speaker);
return route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(
AudioQueryToJSON({
accentPhrases,
speedScale: 1.0,
pitchScale: 0.0,
intonationScale: 1.0,
volumeScale: 1.0,
prePhonemeLength: 0.1,
postPhonemeLength: 0.1,
outputSamplingRate: getEngineManifestMock().defaultSamplingRate,
outputStereo: false,
}),
),
});
});
await page.route(/\/accent_phrases/, async (route) => {
const payload = new URLSearchParams(new URL(route.request().url()).search);
const text = payload.get("text");
const speaker = Number(payload.get("speaker"));
if (text == undefined || speaker == undefined) {
throw new Error("text, speaker is required");
}
const isKana = payload.get("is_kana") === "true";
if (isKana) {
throw new Error("AquesTalk風記法は未対応です");
}
const accentPhrases = await textToActtentPhrasesMock(text, speaker);
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(accentPhrases.map(AccentPhraseToJSON)),
});
});
await page.route(/\/mora_data/, async (route) => {
const payload = new URLSearchParams(new URL(route.request().url()).search);
const speaker = Number(payload.get("speaker"));
const accentPhraseRaw = route.request().postData();
if (accentPhraseRaw == undefined || speaker == undefined) {
throw new Error("accent_phrase, speaker is required");
}
const accentPhrase = (JSON.parse(accentPhraseRaw) as []).map(
AccentPhraseFromJSON,
);
replaceLengthMock(accentPhrase, speaker);
replacePitchMock(accentPhrase, speaker);
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(accentPhrase.map(AccentPhraseToJSON)),
});
});
await page.route(/\/synthesis/, async (route) => {
const payload = new URLSearchParams(new URL(route.request().url()).search);
const speaker = Number(payload.get("speaker"));
const enableInterrogativeUpspeak =
payload.get("enable_interrogative_upspeak") === "true";
const audioQueryRaw = route.request().postData();
if (audioQueryRaw == undefined || speaker == undefined) {
throw new Error("audio_query, speaker is required");
}
const audioQuery = AudioQueryFromJSON(JSON.parse(audioQueryRaw));
const frameAudioQuery = audioQueryToFrameAudioQueryMock(audioQuery, {
enableInterrogativeUpspeak,
});
const buffer = synthesisFrameAudioQueryMock(frameAudioQuery, speaker);
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "audio/wav",
},
body: Buffer.from(buffer),
});
});
await page.route(/\/sing_frame_audio_query/, async (route) => {
const payload = new URLSearchParams(new URL(route.request().url()).search);
const speaker = Number(payload.get("speaker"));
const scoreRaw = route.request().postData();
if (scoreRaw == undefined || speaker == undefined) {
throw new Error("score, speaker is required");
}
const score = ScoreFromJSON(JSON.parse(scoreRaw));
const phonemes = notesToFramePhonemesMock(score.notes, speaker);
const f0 = notesAndFramePhonemesToPitchMock(score.notes, phonemes, speaker);
const volume = notesAndFramePhonemesAndPitchToVolumeMock(
score.notes,
phonemes,
f0,
speaker,
);
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(
FrameAudioQueryToJSON({
f0,
volume,
phonemes,
volumeScale: 1.0,
outputSamplingRate: getEngineManifestMock().defaultSamplingRate,
outputStereo: false,
}),
),
});
});
await page.route(/\/sing_frame_volume/, async (route) => {
const payload = new URLSearchParams(new URL(route.request().url()).search);
const speaker = Number(payload.get("speaker"));
const raw = route.request().postData();
if (raw == undefined || speaker == undefined) {
throw new Error("score, speaker is required");
}
const { score, frameAudioQuery } =
BodySingFrameVolumeSingFrameVolumePostFromJSON(JSON.parse(raw));
const volume = notesAndFramePhonemesAndPitchToVolumeMock(
score.notes,
frameAudioQuery.phonemes,
frameAudioQuery.f0,
speaker,
);
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "application/json",
},
body: JSON.stringify(volume),
});
});
await page.route(/\/frame_synthesis/, async (route) => {
const payload = new URLSearchParams(new URL(route.request().url()).search);
const speaker = Number(payload.get("speaker"));
const frameAudioQueryRaw = route.request().postData();
if (frameAudioQueryRaw == undefined || speaker == undefined) {
throw new Error("frame_audio_query, speaker is required");
}
const frameAudioQuery = FrameAudioQueryFromJSON(
JSON.parse(frameAudioQueryRaw),
);
const buffer = synthesisFrameAudioQueryMock(frameAudioQuery, speaker);
await route.fulfill({
status: 200,
headers: {
"Access-Control-Allow-Origin": "*",
"Content-Type": "audio/wav",
},
body: Buffer.from(buffer),
}); |
|
📝 とりあえずデフォルトエンジンURLに |
|
📝 Nodeのときもkuromojiのブラウザの方のクラスを使いたい。 あと実装を外のリポジトリに配置したい。 |
|
（プレビュービルドでモック使えるかと思ったけど、ビルド時の.envのエンジンパスをまだ変えてないのでダメだった） |
|
🚀 プレビュー用ページを作成しました 🚀 更新時点でのコミットハッシュ： |
This was referenced Nov 9, 2024
|
📝 とりあえず第1弾として、engineMockディレクトリと、そのモックを使ったスナップショットテストだけプルリクエストを送信する。 その後モックを使ったいろんなコード変更をプルリクエストしていく。 |
|
📝
|
|
とりあえずできるかわかりませんが、VOICEVOX orgにforkして、それを取り込む方針で行こうと思います！ |
This was referenced Dec 7, 2024
|
ブラウザでkuromojiが正しく動くように、VOICEVOX/kuromoji.jsを作りました！！ これでプルリクを作っていこうと思います。 |
Hiroshiba
force-pushed
the
エンジンのmockを作る
branch
from
a4d3e02 to
2009e2d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
内容
の解決を目指したプルリクエストです。
ついでにストーリーブック上でコンポーネントテストする方法を色々試そうとしてます。
TalkEditorの表示と、モックエンジンを使ったピッチ推論までできたのですが、なぜかscssが読み込まれずにスプリッターの色指定がうまくいってないです。
Viteとかの設定な気がしないでもないので、詳しい方いらっしゃったらヘルプいただけると助かります 🙇
追記：わかりました！！！たぶん色の初期化をしてないからでした！！
関連 Issue
fix #2144
スクリーンショット・動画など
こんな感じで境界線がない。多分正確には透明になってる。
その他